This is a response from the first author of an article that suggest
the possibility of lack of replication in imaging studies to my
message suggesting that they try to reproduce their results at another
site. My comments below.

<p>
*********************************************************
<pre>

From: "Michael Miller" &lt;miller_psych.ucsb.edu>
To: "Yehouda Harpaz" &lt;yh_maldoo.com>
Cc: &lt;todd.c.handy_Dartmouth.EDU>; &lt;scott.grafton_Dartmouth.EDU>;
&lt;michael.s.gazzaniga_Dartmouth.EDU>; &lt;george.wolford_Dartmouth.EDU>
Sent: Thursday, November 07, 2002 4:24 PM
Subject: Re: Reliability of individual differences


Dear Yehouda Harpaz,

Thank you for your suggestion. I was going to refer you to a paper by
Casey et al., 1998, that looks at spatial working memory activations
across 4 institutions, but then I read your website listed below and
saw that you've already given this topic considerable thought,
including a nice critique of the Casey article. Another paper you
might consider is McGonigle et al 2000 in Neuroimage. They don't look
at reliability across institutions, but they do a nice job of
reliability in individual subjects across multiple sessions on simple
cognitive tasks. I see you've also considered the use of the fMRI data
center as a source for reliability studies. We are currently working
on such a project using the analysis that we developed in our most
recent paper. I think it's important not to rely on comparisons of
statistical maps, which can have relatively arbitrary thresholds, but
to rely on quantitative comparisons of signal intensities. My main
focus right now is to look at reliability across a variety of memory
tasks. My experience has been that more constrained memory tasks like
working memory are quite reliable across subjects while more
strategically -oriented memory tasks like episodic retrieval are quite
variable across subjects. Anyway, I appreciate your interest.

Mike Miller

> On Thursday, November 7, 2002, at 02:31 AM, Yehouda Harpaz wrote:
>
> > Dear Michael Miller,
> >
> > I just read your artiacle "Extensive Individual Differences in Brain
> >  Activations Associated with Episodic Retrieval Are Reliable Over Time",
> > And I think it is very interesting.
> >
> > One suggestion for future studies: get _another research center_ to do
> > the
> > same
> > experiments on the _same individuals_, to see if they can reproduce the
> > results.
> > Brain imaging is notorious for being irreproducible across research
> > centers,
> > (presumably because of individual variability), so it is going to be
> > useful
> > to establish
> > that what you got is reproducible across research centers.
> >
> > Yehouda Harpaz
> > http://human-brain.org/imaging.html

</pre>
***************************************************** <br>
<p>

Things to notice are:
<p>

<li> This guy, for a change, actually read at least part of my page.<p>

<li> He was going to refer to Casey <i>et al</i>, which I <a
href=imaging.html#casey>discuss on my page<a> and show that even
though they claim to find "reliable activity", they clearly did not.
He doesn't actually say whether he was intending to give it as an
example of replication across sites, or as an example of failed
replication.  However, if he agrees with me that it was a failure,
there is no reason why he shouldn't say it, so it is more likely that
he thought it was an example of replication. <p>

<li> In his future research, he is going to look for replication across
memory tasks. Since any lack of replication in such test is naturally
attributed to the differences in the memory tasks, this research is
"safe" from the possibility of uncovering lack of replication across
sites or subjects. Thus for him, the possibility of lack of
replication across sites and subjects is not something worth pursuing.
<p>

I tried to ask him whether he agrees with my discussion of Casey et
al, and about cross-site reproduction, but he didn't answer.