This is the exact text as I got it. My comments are here.
Editor Letter :
===========================================================
We have received two reviews of your article, "reproducibility of cognitive imaging.." that you subnitted to Cerebral Cortex. As you will see while the reviewers were not unsympathetic to your point of view, they have raised a number of issues about the criteria for selection of articles to support your position. A number of other weknesses in the manuscript were also identified. In view of their concerns, the review has not reached a high enough priority for publication. One of the reviewers has suggested that the article, assuming revision, might be more appropriate for a journal like Trendes in Cognitive Neuroscience or Critical Review in Neurobiology.
Thank you for considering Cerebral Cortex for your work. I hope the reviews provided will be helpful to you as you may consider to redesign your study for publication.
Sincerely,
Patricia S. Goldman-Rakic, Ph.D.
Prof. of Neuroscience, Neurology & Psychiatry.
===========================================================
REVIEWER#1
14798
1. It is unclear exactly how the comparisons have been made. What criteria does he have for lack of reproducibility and how far do differences in test paradigms account for the inconsistencies? How many studies have actually been carried out as intended replications of other results using different PET scanners? It might be easy to arrive at the conclusion of variability in results merely because there is variability in methods. This is a common problem, in neuroscience, not restricted to functional imaging. We would need to see much stronger case that there was something 'special' about these procedures.
2. It is probably a bad idea to have included all possible (125) studies in the survey without imposing some degree of quality control. Tighter criteria should have been used fort studies to have been included.
3. The writing is poor, in some places too informal or colloquial for a journal of the quality of Cerebral Cortex. For example, on p5, top, does the author mean 'giry and sulcy', actually gyri and sulci?!
4. By comparing within-subject variation, between study variation and between imaging modality variation, the author has spread himself little thin on the details of the analyses, which appears superficial. The structure of rationale of the analysis is also not always quite clear, as a result. It should be also much more specific about what it adds to the recent review by poline et al. Overall, while I have much in sympathy with the author's objectives, he should probably submit it to another journal for publication such as TICs or Critical reviews in neurobiology, or even a specialist nneuroimaging journal, where the technical aspects of his argument can be debated by a more expert audience.
========================================================================
REVIEWER#2
Referee's report on "reproducibility of cognitive imaging of the cerebral cortex PET and fMRI: A survey of recent literature", by Yehouda Harpaz (C14798).
The author addresses a very important issue in functional brain imaging: the issue of reproducibility of cortical activations in particular, and cerebral and cerebellar activations. This is an issue which has been (intentionally?) overlooked by scientists doing functional brain imaging. Unfortunately it is extermely rarely that scientists doing functional brain imaging try to replicate the findings of others. Also the publication system may discourage this most necessary practice.
The author has gone through over 100 published functional imaging articles and reports an evaluation of some of these from two perspective: reproducibility across studies and within studies. It is not clear how these studies are selected. The author uses the cognitive imaging without actually defining it. Usuaully cognition is taken different from somatosensory and motor activities, but occasionally the author discusses activation in premotor cortex.The domain of investigation should be clearly defined, as there are several example of high reproducibility of motor paradigms. For example, more than 50 studies and the same amount of abstracts have verified reproducible activations of the SMA and M1 when people perform motor sequences. Further it is unclear how the author defines reproducibility. The author should present a rigorous definition of what is a match of activations. In case centers of gravity it is, the author merely gives a judgement of reproducubility on unclear grounds. A third major deficit is lack of statistical tools to judge the reproducibility. One would hope that the author would sharpen the weapons to attack their apparent lack of reproducible findings in cognitive neuroimaging. Cognitive here is used in its usual meaning of perceptual attentional and purely cognitive tasks.
The manuscript has several spelling mistakes and omissions.
I have checked some of the examples given, and the description in the text is usually fair, but lacks precision, which is a pity since this is a most worthwhile.