Correspondence should be addressed to Yehouda Harpaz. e-mail: yh@maldoo.com
To the editor,
PET and fMRI are used heavily to investigate the human brain. Substantial part of this work falls inside what I call "cgnitive Brain Imaging", which I define as studies on healthy humans that try to elucidate the mechanisms of human cognition. These studies try to map locations in the cortex at higher resolution than the gross layout of sensory input areas, motor areas and the Wernicke and Broca areas, which were found by brain damage studies. The interpretation of these studies is based on the assumption that their results are reproducible across all healthy individuals.
However, a search of the literature reveals that there is no evidence for such reproducibility. At the moment, researchers in the field cannot point to a single experimental setting which gives results at resolution higher than the gross layout which are reliably reproducible across individuals. This lack of evidence for reproducibility has already been noted earlier in the visual recognition literature (Kosslyn and Bly 1997, Farah and Aguirre 1999), but it is true in other areas of cognitive brain imaging too. Thus, currently, we don’t have a reason to believe that the results of these studies are reproducible, and the lack of reproducible settings suggests that they are not. In addition, there are sporadic examples in the literature which also suggest that these studies are not reproducible.
For example, in a recent study (Miller et al, 2002) the authors say:
"In the present study, we demonstrate striking individual differences in brain activity associated with episodic retrieval, differences that go beyond the expected variations due to individual differences in cytoarchitectonics and warping due to spatial normalization.
And a study of auditory word processing (Burton, Noll, & Small 2001) says:
"However, the individual subject data revealed variability in the location of the temporal and frontal activation."
At the moment, studies that look at reproducibility either test reproducibility within subject, or compare whole images, which always show correlation because of the similarity of the gross layout of the cortex. These studies cannot tell us whether the results of cognitive brain imaging, which area peaks of activity in the cortex, are reproducible across individuals. Thus we are left in the dark about the reproducibility of the vast majority of cognitive brain imaging.
That is obviously an untenable situation. Therefore, I urge that an action should be taken with high priority to investigate the reproducibility of cognitive brain imaging across individuals. For example, Nuron may select three published studies (either randomly or using some criteria), and then ask other groups than the original authors to repeat the experiments exactly, and then compare the results.
While repeating experiments may seem like a waste, reproducibility of the results is important enough to justify even much larger effort. The number of these studies that are published each year is in the hundreds at least (maybe more than 1000), and at the moment we don’t have a reason to believe that any of these is reproducible across individuals in resolution above the gross layout of the cortex.
REFERENCES
1. Kosslyn, S.M. & Bly, M. 1997. Functional anatomy of object recognition in humans: evidence from positron emission tomography and functional magnetic imaging. Current Opinion in Neurology 10, 5-9.
2. Farah, M. J. & Aguirre, G.K. 1999. Imaging visual recognition: PET and fMRI studies of the functional anatomy of human visual recognition. Trends Cogn Sci. 3(5), 179-186.
3. Miller, M.B. et al. 2002. Extensive individual differences in brain activations associated with episodic retrieval are reliable over time. Journal of Cognitive Neuroscience 14, 1200-1214.
4. Burton, M.W., Noll, D.C. & Small, S.L. 2001. The anatomy of auditory word processing: individual variability. Brain Lang. 77(1), 119-31.