Dear Roger Highfield,

This is the matter I talked with you on phone on monday.

The case demonstrates that in the field of brain scanning for investigating cognitive function (as opposed to clinical use):

1) Reproducibility, which is essential ingredient of science, is not regarded as important.

2) Fundamental assumptions with wide implications in the field are taken for granted, and are not discussed.

People that take this attitude can get much more impressive `results`, and hence dominate the field. By now, this attitude is prevalent enough to dictate what is published, and publications that challenge it are blocked, thus leading to a large body of `scientific research` which is not scientific.

The story is both interesting sociologically (how this can happen inside modern science establishment), and has public interest, because public money is used to generate bogus data.

The case itself: The original paper, with an impressive conclusions, was published in 'Current Biology'. The authors themselves admit their results are not reproducible, and based their analysis on unfounded assumptions.

I sent a comment (around a page) analyzing some of the unfounded assumptions, and stating that together with the irreproducibility of the article the data that was presented does support their conclusions. This was rejected by two reviewers. Both of the reviewers ignore completely the question of irreproducibility, and their review of my analysis was very faulty. The main faults can be easily understood by a non-expert in the field, and far below the standard of scientific texts. One of the reviewers clearly did not read significant parts of my comment, while the other distort one of the central statement of my analysis to change its meaning, and argue against the changed meaning.

Both of the reviewer agree that one of the points I raised (the unfounded assumption of equivalence between activity and function) has wide implications in the field, yet neither of them could come up with a reasonable response to my analysis.

My comment was rejected based on these reviews, with the editor of 'Current Biology' claiming that he is not an expert and must rely on the opinions of the reviewers. I then sent the matter to several members of the editorial board of 'Current Biology'. Two of them answered, both suggesting that I should submit my comment in another journals. One of these also agreed that the equivalence assumption is of wide implications, but did not offer any response.

I tried to submit to 12 other journals, and they all refused to consider my comment without reviewing it.

This is a single case, but it clearly demonstrates points (1) and (2) above, because none of the authors of the original article, the reviewers of the original article, the editor of 'Current Biology', the reviewers of my comments, the members of the editorial board of 'Current Biology', or any of the editors of other journals thought that irreproducibility and unfounded assumptions are serious problems.

One the editorial board members and some of the other editors said that the equivalence assumption deserve a wider review. However, the a single case has the advantage that it focuses the discussion, and the main points are clearly demonstrated.

How to proceed: I would suggest that you select one or more senior researcher in the field (I can suggest the names myself), send him/her the matter, and ask him/her to give an opinion, in particular concentrating on the points of:

1) Irreproducibility
2) The equivalence assumption (point 2 in my original comment).

That is approximately what I did when I sent it to the editorial board members, but you have the advantage that you can put more pressure on the researcher, by the potential threat of publishing my point of view without a response. You will see that nobody can come out with a reasonable answer to my points.

If you are interested in this case, the easiest way to find more is to lookup my page in http://human-brain.org/zeki-index.html . Alternatively, I can send you the actual papers.

Yehouda Harpaz
http://human-brain.org/