Yehouda Harpaz
yh@maldoo.com
6 Jan 96 [last updated 22 Mar 2012]
 related texts 

Reasoning Errors In The Current Models of Human Cognition

1. Introduction

[1.1] Cognitive Psychologists (CogPsys) and researchers in related areas hypothesize many models of the way the human cognition, or part of it, works. Since we cannot, yet, test these models directly, their validation is based on interpretation of experimental evidence. This interpretation, however, is commonly fraught with fundamental errors in reasoning, and it is difficult to find a model which does not suffer from this problem.

[1.2] In this text I try to list the most important reasoning errors that occur in the arguments which are used to support models of human cognition. I first discuss the error which I believe to be the most fundamental, the conclusion-validation error, which is a 'second-order' error, and then list the 'first-order' errors. However, the first of these (the 'assumption of sameness', [3.1]), is probably as important as the conclusion-validation error, and can be described as The Dogma Of Cognitive Science.

[1.3] In most of the cases, each error is harmless on its own, but in combination with other errors can lead to serious mistakes in the interpretation. Commonly, an interpretation which supports a model is wrong because it is based on several of these reasoning errors.

More fundamental problems in cognitive science (including the idea of consciousness) are discussed in the methodological points. Misconceptions and myths are discussed separately. Direct comments on an online text representing the current thinking in cognitive psychology (MITECS) are here. The general problem of nonsense-arguments is discussed here.

2. The Conclusion-Validation Error

[2.2] This is an error because it can be used to support a conclusion which is not supported by any sound argument, and to support unsound arguments. (By 'sound argument' I mean argument which is made of good assumptions (supported by evidence) and good logical inferences). For example, assume that the wrong conclusion X is supported by lines of argument A,B and C, each of which is unsound (contains some unsupported assumption or wrong inference)(see the sketch below, the lines mean 'supporting'). Now, when the soundness of A is argued, it can be supported by claiming that its conclusion (X) is plausible, based on lines of argument B and C. In the same way, the lines of argument B and C can be supported by claiming their conclusion is plausible, based on arguments A and C or A and B respectively.

   Argument A  ___________
                          \
                           \
                            \
   Argument B ---------------> Conclusion X
                            /
                           /
   Argument C ____________/ 

[2.3] The 'obvious' remedy, of dealing with all the lines of argument at the same time, is realistic only if each line of argument is very simple. In all the interesting cases, the complexity of the lines of argument prohibits dealing with more than one of them at a time {1}. Thus as long as the plausibility of the conclusion of a line of argument is regarded as supporting the argument, there is no way to show that X is implausible.

[2.4] It can be argued that the convergence of A, B and C to the same conclusion supports them, but this is wrong. Unsound lines of arguments can be modified to reach any conclusion, without affecting their {un}soundness. Therefore, there is no problem to form several unsound lines of arguments which converge on the same conclusion. Only sound arguments are difficult to modify without affecting their soundness, so convergence of lines of argument is significant only if at least two of them are sound on their own. Hence there is no way to go around checking that each line of argument (at least two of them) is sound independently of the conclusion.

[2.5] The other way of using the conclusion-validation error is to support a line of argument. For example, assuming conclusion Y is supported by the sound arguments D, E. Then an argument F is presented, which also supports Y. This is regarded as supporting both Y and F. This an error, because if F is unsound, it can be made to support any conclusion, including Y, so the fact that it leads to Y does not show that it is sound. Thus before accepting the validity of the argument F we must check its validity, and it is not enough the check the validity of Y.

[2.6] The soundness of the conclusion is not necessary for using the conclusion-validation error, and it is enough that a person believes in the soundness of the conclusion for them to use the error. For example, that deducing modularity from double dissociation is a logical error is pretty obvious to everybody that ever thinks about it, and the best explanation for the wide acceptance of this deduction is that the conclusion, i.e. modularity, is very popular, and this popularity "validates" the deduction via the conclusion-validation error.

[2.7] Most of the 'supportive evidence' for the current models of cognition is based on this flaw. The researchers take an established fact about human behavior (the conclusion Y), which is known through behavioral/psychological observations (the arguments D, E), and then build a model (argument F) that 'predicts' it. Then they use this 'prediction' as evidence that the model is sound. Even though this is a quite obvious error, it is very common.

[2.8] The conclusion-validation error is very powerful: when it is used to support some conclusion or argument, it is impossible to show that the conclusion or the argument is implausible, no matter how wrong it is. It is made even worse by the fact that most people will not admit that it is an error.

[2.9] CogPsys seem to be completely ignorant of this problem, and are happy to support their lines of argument by comparing their conclusions with other lines of arguments. In general, they regard convergence of lines of argument as more important than the soundness of each argument. This allows them to believe in implausible models of cognition, which are based on 'convergent evidence' rather than on sound lines of arguments, and to take nonsense arguments as real.

3. 'First-order' Reasoning errors

[3.0.2] However, the different errors still occur independently. For example, A researcher may be assuming simplicity without being confused about genetically-programmed/learned features, or be confused about genetically-programmed/learned features without assuming simplicity. In the list, I tried to put in a separate entry every reasoning error which may appear on its own.

[3.1] The assumption of sameness: The dogma of cognitive science

Almost all (if not all) CogPsys assume that the underlying mechanisms of specific features in human thinking would be the same across people, generations and sometimes species. This assumption is in general unjustified in learned feature (i.e. almost anything), and have to be supported by some evidence.

Sometimes it is claimed that the similarity of the underlying mechanism is supported by the fact that people behave the same in some sense. This is wrong, because people learn to behave the same, for various reasons. To support the sameness assumption, it is needed to find a behavior which is not learned, and is still similar between people. In general, this is not done.

The sameness error is quite commonly accompanied by the "operational modularity" error in a very tight way such that it is diffcult to separate between them. This reasoning error is the most pervasive in Cognitive Psychology and related sciences, and may well be regarded as The Dogma of Cognitive Science. It is so strongly entrenched that in some cases it seems that the researchers lost the ability to consider the possibility that it is not true. The following is a partial list of lines of research that are based on it:

• Building models that behave like the average of the behavior of many people.

  The validity of this is normally assumed, by looking at other domains of research. However, in other domains it is much easier to know if the underlying mechanism are the same or not. If the underlying mechanisms by which different people perform a task are variable, and the target of the research are these mechanisms, it is not a valid method to build models that match the average behavior, because they tell us nothing about the actual mechanisms.

• Generalizing from single case studies. Obviously, if thinking is variable across people, this is not a valid approach.

• Looking for the 'Architecture of the mind' (e.g Soar, ACT-R, etc.), and also trying to define the 'neural circuits' in the cortex.

• Accepting irreplicable results as if they were replicable (see in A general discussion of cognitive brain imaging for details). The best explanation for this phenomenon is that people assume these studies must be replicable, because of the sameness assumption.

• The notion of 'qualia' (and hence 'consciousness') is for most people dependent (implicitly) on the sameness assumption (see in Methodological points).

• Most of The philosophical arguments about "type identity" and "token identity" don't make sense without the sameness assumption.

To a large extent, this error is motivated by wishful thinking. People that study cognition want to find generalization about human thinking, so they assume that there are such generalities, even if there is no real evidence for it.

It is worth noting that people that claim that humans are born "blank slate", and that it is possible to educate a child to anything, are doing the sameness error too. They assume that all humans start the same ("blank slate"), and that their learning mechanisms will respond in the same way to their environment.

[3.2] Evidence about memory and perception

CogPsys commonly test 'memory' and 'perception'. However, there is no way to test memory or perception directly (at least currently). All that can be tested is behavior, which is a result of cognitive processes which 'integrates' (in some way) past information, current information and emotional state (at least). Often, CogPsys assume the ability to uncouple the effects of the different systems (like memory, perception, etc.). Without at least basic understanding of the logic of cognitive processes, this assumption has no foundations, and it is nonsensical: a-priori, there is no reason to assume that it is possible, even in principle, to uncouple the effects of different systems.

[22 Mar 2012] There is now much better technology to look at brains than there was when I originally wrote this. But the statement above, that "there is no way to test memory or perception directly", is still 100% correct.

The 'computational' approach (following Marr) is the extreme example of this error. In this approach is it not only assumed that the systems can be uncoupled, but also that the interfaces between them can be known a-priori. The trend of investigating 'visual awareness' as separated from general awareness is also a version of this error

[26 Nov 2003] By now I concluded that this is a special case of the "Operational modularity" error.

[3.3] Model space size

It is common to claim that a model is supported by observations that are compatible with it. This, however, is true only if the number of possible models is small, so it is unlikely that there is a false model (i.e. a model that is not compatible with further observations), which is compatible with the original observations. It is thus essential to be sure that the model space is small before claiming that the observations support the model.

In most of areas of research, it is possible to take for granted that the model space is small, but this is not true when dealing with models of human thinking. This is because our understanding of human thinking is very rudimentary, so there are no strong constraints on the possible models.

CogPsys seems to be completely ignorant of this problem, and allow themselves to include in theirs models 'magical' units, which perform complex cognitive operation without plausible underlying mechanism. This effectively expand the model space to infinite size, and this undermines all the logic of experimental evidence.

[ 15Jan2002] Just found this: Roberts, S. & Pashler, H. (2000). How persuasive is a good fit? A comment on theory testing in psychology. Psychological Review, 107, 358-367. (Check here if the link is broken). A very critical article which makes good points. However, even such a critical discussion, which explicitly blame psychologists of "Neglect of basic principles" (last two paragraphs), ignores the model space size problem. They implicitly assume that once psychologists use their solutions, a good fit will give a strong support for the model, which is not true as long as the number of possible models is large.

[3.4] Model validation

In the long term, the only real test for a model is whether the predictions that it makes are correct. This criterion, however, is widely ignored in CogPsy, and there is a strong tendency to simply fix the model as needed to match new observations. Newell, for example, explicitly state this approach (Unified Theories of Cognition, 1990. P.14), and in psycholinguistics it was elevated to a formal theory (Principles and Parameters). This eliminates the possibility of refuting any model, no matter how bad it is.

Many CogPsys regard a model as validated when several (very simplified) aspects of human cognition have been modeled using it. This is obviously invalid, because there is no reason to assume that it is a problem to model selected aspects on a wrong model.

[3.5] Relying too much on macro-structure knowledge

CogPsys tend to try to support theirs models by evidence about the macro-structure of the brain. However, from the little that we know about information processing systems (e.g. computers), it is obvious that you cannot understand the logic of the system from the macro-structure. (For example, consider figuring out how a word processor works from the physical structure of the computer it runs on). The main problem is that currently we only know about the existence of connections between various brain structures, but we don't have a clue about their activity in various situations.

New imaging techniques give a more dynamic picture of the brain, but they don't tell us anything about the interaction between various parts of the brain. In addition, they can only show places where there is a large change in activity, which do not necessarily correspond to the most important activity for cognitive function. On top of this, the actual results don't show consistent (across experiments) correlation between brain images and psychological variables. (There is in the field an illusion that imaging techniques do give significant results, but this is based on overinterpretation of the data. Here is a fuller discussion).

As a result of our limited knowledge of the activity in the connections between various structures, and the complexity of these connections, the knowledge we do have imposes very little constraints on the possible models. Thus this knowledge cannot be used to guide model building, and can be only used for constraining them. CogPsys, however, believe that this knowledge is very useful, and give large weight to 'evidence' from it. Since the 'evidence' from macro-structure is not actually useful for building models, this 'evidence' is normally actually based on some of the other reasoning errors.

The most abused macro-structure features of the brain tend to be:

Lobes .

The borders between the lobes in the cortex are completely arbitrary, and do not correspond to any real functional border. In the best cases, they correspond to sulci (fold in) of the cortex, but there is no reason to believe that these sulci are dividing the cortex to functional units. In many cases, the borders are simply straight lines. Thus the 'Lobes' do not correspond to functional units.

The belief in the significant of lobes goes further than assuming that they are separate units. In Brain Basics: know your brain, which is an introductory text writtent by a US government agency, it says:

When you plan a schedule, imagine the future, or use reasoned arguments, these two lobes are working.

That strongly implies that the other lobes are not working at that point. Anybody with any expertise knows that this is false, and all the lobes work all the time. However, this text is intended for non-experts, and these are going to take the impplication in, because they don't know that it is wrong. The text continues in this way for several paragraphs.

Areas (regions) in the cortex.

These are normally defined by cytoarchitectonic observations. This has more validity than the division to lobes, and the parameters of neural activity may be different between areas. However, there are no borders between the areas, in the sense that information flow across area borders is the same way as inside areas. In particular, the connections (at the level of neurons) between areas are variable between individuals in a stochastic way. Thus these areas do not correspond to separate modules.

This paper takes the ideas of regions to ridiculous extremes. They say in the second paragraph:

For example, even though the size of Brodmann's area 17 (now called V1) can vary from one individual to the next by 2- to 3-fold (3, 5), area 17 (V1) is always next to area 18 (V2) and never has area 19 (V3) as a nearest neighbor. Why are the many recognizable cortical areas arranged the way they are?

Which is plain stupid. Area 18 is around area 17 by definition, so it is stupid to wonder why it is always there. The interesting thing is that it tends to always have the same "microscopic structure" (in the authors terms, "histological structure" is more commonly used). Thus the actual question is "why is the area around area 17 always has this microscopic structure?". The authors completely miss this because they regard the regions as separate modular units that can be (in principle) moved around.

Their actual finding, that the arrangement of regions is optimal, is equivalent to the statement that the probability of connections between neurons goes down with distance, which is the obvious natural assumption. It is therefore completely uninteresting finding, and is made to look interesting only by lousy thinking.

[29 jan 2004] This paper is following the same logic. This paper has some interesting features:

• It is a neuroscience paper, but three of the four authors, including the corresponding author, are from the "Committee for Philosophy and the Sciences, Department of Philosophy".
• It was communicated by Noam Chomsky.

Projections.

It is common for CogPsys to support some theory, which hypothesize that two areas, A and B, are connected, by asserting that there is a projection from A to B. However, in the cortex there are many projections, and each area is connected to many areas. Thus the probability that two areas are connected is quite high, so it is easy to find the 'appropriate' connections for any theory.

Columns.

Columns now have their own page

Ocular dominance columns

These are not realy columns, they are stripes. Since a person with a single eye doesn't have these "columns", but is able to see the world almost as well a two-eyed person, these "columns" cannot have much significance in vision in general. They may be important in being able to use the two eyes at the same time. However, they are relatively easy to visualize, so researchers assume they must be important in vision in general, and there is a large body of research dedicated to their development and role (search the net for "Ocular dominance columns").

A point worth mentioning is that the pattern of the stripes is variable between individuals, a fact that is rarely mentioned when ocular dominance columns are discussed. Typically, a drawing (corresponding to one individual animal) is shown, but it is not mentioned that this is one specific individual. The reader is left with the impression that the pattern is the same in all animals (a highly attentive reader can figure it out from the fact that the features in the pattern are not given names. If they were reproducible across individuals, they would be given names. I don't think there are many such attentive readers, though). Crick (p. 141, footnote) is a notable exception. [11mAY2004] This encyclopedia article (Cortical Columns, Carreira-Perpinan & Goodhill, Encyclopedia of Cognitive Science, Macmillan Publishers Ltd., 2002) also says it: "in fact each ocular dominance pattern is apparently as unique as a fingerprint".

It is worth also mentioning that the transition between left- and right-sensitive stripes is not as sharp as the figures normally suggest.

[30 Jan 2003] A new article in Nature Neuroscience (Capricious expression of cortical columns in the primate brain. Daniel L. Adams & Jonathan C. Horton, Nature Neuroscience 2003, 6(2), 113-114) shows the high variability in squirrel monkeys, and explicitly makes the point that it shows that they are insignificant, at least in squirrel monkey.

"Hypercolumns"

This is the most stupid of the column concepts, and deserves its own page.

Apart from anatomical structures, CogPsys also try to use other features that can be describe as 'macro-features', like global oscillations, e.g. gamma oscillations. However, the amount of information in these oscillations is so small that it cannot have any significant part in the information processing of the brain. Other 'macro-features' that Cogpsys try to use are 'transmitter-system' (e.g. dopamine-system). This is obviously nonsense, as the generation and sensing of dopamine is obviously not a functionally separated system, so it makes no sense to investigate it as such.

The reason that CogPsys (and researchers in related areas) pay so much attention to the macro-structure is that it is relatively easy to see, compared to more detailed neural activity.

[3.6] Ignoring knowledge about neurons

While giving too much weight to macro-structure, CogPsys completely ignore knowledge about single neuron characteristics. The common explanation for this is that we don't know enough about neurons. It is true that we don't know enough about neurons to build models from this knowledge, but we can use this knowledge to restrict the possible models (decreasing the model space size). For example, this knowledge tells us that symbolic models are extremely unlikely, because they need indirection, which neurons cannot do (see brain symbols for full discussion).

It is not obvious why CogPsys ignore this knowledge. It seems to me that the best explanation is that:

• It requires the CogPsys to learn something (neurobiology) that is off their main line of work .
• When they do learn it, they find that this knowledge contradicts their assumptions, so they drop it (the knowledge).

Even researchers in neuroscience are actually guilty of this error. For example, it is quite common to find models of the cortex activity which ignore inhibition, short distance connections and 'back-projections'(connections that go in a direction that does not agree with the model). The situation is somewhat improving lately, but slowly.

[6 Jan 2003] By now it is probably over the top to say the CogPsys ignore knowledge about neurons ([ 17 Dec 2003] At least sometimes they still do. Example: this article(doi)) ). However, they still don't systematically take it into account. Instead, they pick bits of data that seem to fit their ideas, and use them, ignoring the rest. Most commonly, they pick some imaging studies which they believe fit what they say, e.g. here and here.

[3.7] The lack of genetic/learned distinction

CogPsys commonly ignore the question whether a feature is learned or genetically coded. This distinction is important, because it determines what should be assumed. A feature that is assumed to be genetically coded must be independent from elements in the environment which are too new to affect genetics, and stable across individuals and generations. It can also be assumed to be simple, and can be homologous to features in other species. For a learned feature, these assertions are not true. The next two mistakes are partly the result of the lack of distinction between learned and genetic features.

In the related area of evolutionary psychology, there is now a tendency to intentionally blur the distinction between genetic and learned mechanisms. See the discussion about Evolutionary psychology, section 2.7, for example.

[3.8] Assuming too much power to genes

Some CogPsys don't ignore the genetic/learned distinction, but tend to assume too much about the what can be coded by genes. These assumptions can be divided into several groups:

• Assuming genetic codes for features that are related to entities that have not existed for long enough, and hence did not exert enough selective pressure, to evolve genetic code. This is normally done implicitly, and is very difficult to distinguish it from the 'lack of genetic/learned distinction' error.

• Assuming genetic codes for complex operations and knowledge

  The development of the connectivity in the brain is based on gradients of factors, which can be soluble, cell bound or part of the extra cellular matrix. These factors cannot code for the accurate and specific connectivity which is required for complex operations and knowledge.

  A confusing factor here is the fact that the neurobiologists tend to use the term 'specific' in a very weak sense. For example, the axons of some cell population are said to make specific connections when they connect to a specific layer in the cortex. The specificity required for complex operations is much stronger, and the synapses from each axons has to formed on a specific set of neurons, in a specific location in their dendrite trees. This kind of specificity cannot be achieved during development.

  Thus, the very complex processing that is going on in the dendrite tree of each neuron and inside each small group of neurons, cannot be specified by the genes, and is in effect starting as random (and later modified by learning). The only things that can be coded by genes are the tendencies of the cell populations, and these cannot account for complex operations and knowledge, because their number is relatively small. {3} (see Stochastic Conectivity for a discussion of the stochastic connectivity of neurons).

• Assuming genetic codes for abstract concepts.

  It is commonly argued that some abstract knowledge is innate, and in most of the cases this is supported by saying it is useful. However, because these concepts are complex, they cannot be coded genetically, as explained above.

  The argument that innate abstract concepts are useful is also wrong. An abstract knowledge is useful only if it is associated with the real world in some way. In other words, the sensory input which comes from some concrete entities has to be associated with the corresponding abstract concept(s). There is no reason to assume that learning these associations is easier than using as the concepts random neuronal patterns which happen to be activated as a result of the sensory input, and forming the correct associations between them by Hebbian mechanisms.

  The illusion that coding for abstract concepts is useful normally stems from ignoring the problem of applying the concepts to the real world, or assuming that some magic operation performs the appropriate associations. The latter is strongly supported by analogies to computer models, where the associations to the real world are given by the programmer.

CogPsys commonly look for the simplest model possible, and regard simplicity as a virtue in a model. For a learned feature, however, this assumption is not justified. The learning is done by a complex system, and it is a complex process, and there is no reason to assume that the result will be simple.

A confusing factor is comparison with physics, because the word 'simple' is used to denote different things in these areas. In physics, the 'simplest model' is the least ordered, i.e. the highest in entropy. In CogPsy, the 'simplest model' is (usually) the most ordered, i.e. the lowest in entropy {4}. Thus a 'simpler model' in physics (high in entropy) is more likely, while in CogPsy a 'simpler model' (low in entropy) is less likely.

An additional confusing factor is the assumption that simple behavior is necessarily mediated by simple mechanisms. This assumption is incorrect in cases that the simple behavior is optimal, because the mechanisms behind it will try to perform the optimal, and hence simple, behavior, independently of the complexity of the mechanisms themselves.

The simplicity assumption appears explicitly sometimes, but mostly it is implicit. For example, fitting a set of points in a graph (e.g reaction time against some parameter) to a simple function is, in most of cases, justified only if the underlying mechanisms are assumed to be simple.

Another example is the quite common assumption that the various cognitive functions would tend to degrade linearly and in a correlated fashion with the damage to the brain. This false assumption 'allows' CogPsys to deduce things from quantitative comparing the degradation of various aspects of cognition, without any real foundation.

Here (Edward M. Callaway, LOCAL CIRCUITS IN PRIMARY VISUAL CORTEX OF THE MACAQUE MONKEY, Annu. Rev. Neurosci. 1998. 21:47-74) is an example of the simplicity assumption in neuroscience. The author says (p.49, top):

If neighboring neurons receive inputs from similar sources and provide output to similar targets, it is unnecessary to resolve circuits at the level of individual neurons: Coarser but more fruitful methods can often be used.

The simplicty assumption goes hand in hand with the modularity assumption, because modular systems, in general, are simpler.

Here (in "research projects" of this guy) is another typical example of the simplicity assumption. First, there was a simple model which accounted for some of the data. When data was found that did not fit it, the researchers simply postulated another mechanism. In this page, the logic is made explicit in the "signal-processing section":

That a neuron can show different preferred spatial and temporal frequency preferences, and differing degrees of direction selectivity, to luminance and envelope stimuli suggests that its firing reflects a combination of inputs from two distinct pathways or "streams", one mediating 1st-order and the other 2nd-order responses.

[3.10] The confusion about the meaning of 'memory' and implications

In cognitive psychology, The word 'Memory' has (at least) two distinct meanings:

1. A mechanism by which past experience (in the widest definition of the word, including internal processes) affect current thinking in some way.
2. A system in which units of information can be stored and later retrieved.

It is trivially obvious that humans possess memory in sense (1). However, there is no evidence {2} that they possess memory in sense (2) (and neurobiological knowledge make it extremely unlikely). Cognitive psychologists, however, often take for granted the existence of memory in sense (2), presumably by confusion with memory in sense (1). Thus large part of CogPsy theory is based on a quite strong assumption (that humans has a mechanism to store and retrieve units of data) which has no experimental support.

Sometimes researchers claims that when the changes in synapse strength is what they call "storing", and re-activating a similar pattern of the original one is "retrieving". This is "kind of" an example of the confusing and ambiguous usage of terms error, except that "storing" is not really an ambiguous word, and changes in synaptic strengthes are clearly not "storing".

[3.11] Relying on common sense

It is commonly claimed that it is reasonable to assume some assumption, because it 'makes sense'. However, common sense must be based on some experience with the system which is dealt with, or with similar systems. Since nobody has, as yet, any experience with the internals of an intelligent system, nobody has the common sense to deal with it.

A typical example of common sense error is the quite widely held belief that humans regularly compute distances to objects around them. Any sort of analysis of human performance. which is free from reasoning errors, clearly shows that they don't, and the only basis for this belief is that 'it is obvious', or 'it make sense'. {5}

When researchers try to deduce features of cognition from evolution, it may be regarded as this kind of error. At the moment, we do not have a way of predicting the way complex features will evolve in complex environment, so any conclusion from 'evolutionary consideration' about cognition is actually a common sense error in disguise (or plain hand waiving){6}.

[3.12] relying on computer models

Some CogPsys regard computer models as a major guide for understanding human cognition. However, computers have fundamentally different mode of action, at the implementation level. Thus there is no reason to assume that a good program on a computer is also a good program for neurons (see Computer models of cognition for a fuller discussion).

A common counter-argument is that at higher levels computers and humans operate in the same way, but this is not based on any evidence. This argument is commonly based on model space size error: the computer can do X, humans can do X, so they do X in the same way (implicitly assuming only one possible way of doing X).

Modeling on computer is regarded as a useful way of 'flushing out' all the details of the model, thus eliminating the 'ignoring complexity' problem. This would be true if it was actually possible to program real cognitive processes, which is currently out of question. Instead, what the computer models try to model are simplified versions of the real processes, so the details are left hidden in the simplification operation.

Another version of this error is the projection of behavior that is found in artificial neural networks (ANN) programs to the brain. The ANN are minute compared to the brain, and their learning algorithms are clearly different from the mechanisms of the brain. Thus there is no reason to believe that there is any correlation between the behavior of ANN and the brain. This does not prevent some researchers to assume that this correlation exist, and use the results of ANN simulations to 'prove' things about the brain.

In many cases, it is claimed that the computer models, even if they are not doing the same as the brain, give us insights about the brain. There are two ways in which this idea is supported:

• implicitly assuming that the brain and the computer are doing the same thing, even after admitting that they don't. This may sound absurd, but this line of argument is actually quite common.

• Attributing the 'insight' to the computer model even when it clearly did not come from the computer model. Typically, some researcher has some idea of how the brain works, he/she implements it in a computer model, and then it is claimed (by the researcher or by other people) that the idea comes from computer modeling. Clearly, the researcher had the idea before starting to write the computer model, so the idea is independent of the computer model. Because of the difference between the low-level implementation, the model cannot be regarded as confirmation of the idea, either. During writing the model, some new ideas have arisen, but it cannot be assumed that these are relevant to the brain.

[3.13] relying on 'good engineering principles'

Sometime an assumption is supported on the basis that it is 'good engineering'. This would make sense if the relevant consideration are done, which include:

• Consideration of the constraints imposed by real neurons. This is rarely done. Even when the analysis is based on neural networks, the nodes commonly have features that neurons don't have. e.g. backward propagation, reliable timing, reproducible computation of mathematical functions etc.

• Consideration of the process by which the feature has arisen. For a feature that is assumed to be genetically programmed, this means to consider the constraints imposed by the development of the real brain. For example, the currently known processes that direct development are not capable of coding for a complex network of specific connections. In addition, the evolution of a feature have to be considered, i.e. whether it had enough selective pressure for enough time to be evolved.

  For a learned feature, the process by which it was learned have to be considered. In particular, it is important not to assume learning that is relying on magical capabilities.

[3.14] Detached models

Many psychological models are detached, in the sense that their constituents do not correspond to any specific features in the real system (brain). The problem with such models is that they don't tell us anything about the real system, and are unrestricted (have infinite model space).

The normal justification for these models is that they are useful for analyzing human behavior. However, this claim must be supported by checking prediction power of these models, which is rarely done (model validation error). Thus these models can be very 'successful' (agreeing with many pieces of data), yet useless (cannot predict any further data).

Another common justification is that these models point to further experiments. Again, the claim that the experiments which were pointed by these models are more useful than other experiments is rarely (if ever) supported by any evidence.

A common extension of detached models is the 'interpreted detached models', where a simple detached model is used a base of different interpretations. In this way it can 'explain' wide range of phenomena, without actually explaining any of them. A typical example is the 'auditory loop'.

[3.15] The modularity assumption

CogPsys commonly assume that cognition is modular. This assumption is 'classically' supported by two arguments (Marr):

1. It is easier for a designer to work with a modular system
2. In a modular systems, any module can be replaced without affecting the rest of the modules.

Other 'support' for the modularity assumption is that it is easier to evolve a modular system. That is nonsense, because evolution does not have a way of plugging in new modules. Instead, it modifies existing systems for new roles. Looking at the evolution of the human brain, it is clearly a case of a general-purpose module (the cortex) expanding and taking more and more functionality from the more specialized modules.

Thus the modularity assumption has no logical basis or supporting neuroanatomical evidence. However, the modularity assumption is very popular, because it simplifies model building. Therefore, there is a lot of 'evidence' for it, which is mostly based directly (but implicitly) on the assumption itself.

For example, if ablation of a specific region of the brain consistently causes deficit in some cognitive function, the only valid conclusion is that this region has an essential (but possibly small) part in this function. However, researchers frequently conclude that this region performs the function, which is reasonable only if modularity is already assumed.

This conclusion, which is implicitly based on the modularity assumption, is then used to support the modularity assumption. This line of reasoning has led to accumulation of large body of pseudo-evidence for modularity, which serves to increase its popularity (see also the discussion of dissociation below).

Another 'support' for modularity is that the visual system has regions which are more sensitive to one attribute (e.g. color), and damage to these regions cause larger deficit in processing this attribute. This, however, does not show a modular system, because these regions, like the rest of the cortex, interact directly with many other regions. Thus their processing is integrated with the other regions, and they do not form a separate module.

The confusion about modularity is increased by bogus definitions of the word. For example, Fodor in 'The modularity of mind' says (p. 37): 'So what I propose to do instead of defining "modular" is to associate the notion with a pattern of answers to such questions as 1-5. Roughly, modular cognitive systems are domain specific, innately specified, hardwired, autonomous, and not assembled." This notion lacks the most important characteristics of modules, i.e. that they have well defined borders and interfaces. This allows Fodor to argue for 'modularity' without bothering about modules.

Shallice in 'From neuropsychology to mental structure' goes a step further, and equates modularity with double dissociation (p. 21). Thus he can demonstrate 'modularity' by demonstrating double dissociation, without the need for demonstrating innateness or well-defined borders and interfaces.

The popularity of 'modules' is such that in some cases researchers use it even though it makes no sense at all. For example, Mountcastle (1997, Brain, V.120, 701-722) discusses the columnar organization of the cortex, and calls the columns 'modules', even though they don't have any of the characteristic of modules. Many other people do the same error.

[13Jul2004] Just found this article (Crutch and Warrington, Brain, Vol. 126, No. 8, 1821-1829, August 2003), which is quite stunning example. They found in a single patient that the knowledge of places depends on grographical proximity, and then say in the end of the abstract:

It is argued that information about geographical proximity cannot be encoded in purely verbal or visual terms. Consequently, we propose the existence of a separate module of spatially encoded information within conceptual knowledge.

Disappointingly, even a serious researcher like Kathleen Rockland contributes to the confusion, because she uses "module" to mean "patch", and hence support the existence of modularity by showing patchiness. For example, see this article (Ichinohe and Rockland, Cerebral Cortex 2004 14(11):1173-1184), which apart from the confusing usage of modularity, is a good science and contains quite interesting observations.

[3.16] Ignoring Complexity

CogPsys commonly include in their models a complex feature (e.g. comparator, rule-resolver), and (mostly implicitly) assuming that it is simple. In its outcome, this error is equivalent to the model space size error, but it stems from different source: in the model space size error the researcher knows that the feature is complex, but is unaware of the model space size problem, while here the researcher is unaware of the complexity of the feature.

A common example is the distinction between conscious and non-conscious processes, when it is assumed that the person is aware of all the important steps in a conscious process. This assumption s possible only because the large number of steps in any cognitive process (in other words, the complexity of thinking) are simply ignored.

[3.17] 'Knowing' what humans do

In many cases CogPsys assumes that they know something about how human do something, even if they don't have a basis for this. The 'computational' approach may be an explicit version of this error, depending on what is assumed in the computational model. However, in many cases this error is done implicitly, and is based on errors like the simplicity assumption, modularity and sameness assumptions and common sense. I list this as a separate error because in many cases it is difficult to identify the underlying error.

[3.18] Confusing and ambiguous usage of terms

Many texts about cognitive function use terms in a confusing or ambiguous ways. 'The confusion about the meaning of memory' error (above) can be regarded as such an error. The confusing definitions of modularity ([3.15] above) are another example. See here for a discussion of the effect of this confusing usage.

The primary confusing word is 'consciousness', which many people explicitly avoid giving any meaning. This allows them to write all kind of nonsense, because to show that a text which uses the word 'consciousness' is nonsense, you have to iterate through all the potential meaning of the word, and show that with each meaning the text is still nonsense. Even if the number of possible definitions of 'consciousness' was small, people in general are not ready to go through this kind of exercise. Therefore, a text with uses the term 'consciousness' cannot be shown to be nonsense, independently of whether it is nonsense or not. This fact is heavily used in discussions of human cognition. See in Methodological points for a fuller discussion of the term 'consciousness'.

The word 'image' is another example. In terms of the brain, 'image' does not have any meaning, because the definition of 'image' does not give any way to identify if a pattern of activity in the brain is an image or not. Therefore, it is not possible, even in principle, to confirm or refute any theory about the brain that is formulated in terms of images. In other words, all these theories are empty waffle. The fact that 'images' are used in what are supposed to be scientific texts is one of the clearest indications of how nonsensical the field of cognitive psychology is.

A more tractable example is the usage of the term 'to retrieve'. Retrieval forms an important part of many models of cognitive functions, yet it seems that neurons cannot perform this operation (At least nobody succeeded to come with a reasonable model of how they do). One of the suggested 'solutions' is the idea that retrieval is done by activation in a neural network.

However, the term 'to retrieve' implies that the retrieved item is available for further processing, and this is essential for the models that use retrieval. Activation inside a network does not make the 'item' (whatever that is in this context) available for further processing, so this is not actually a retrieval operation. Hence the 'retrieval by activation' hypothesis does not explain how models that rely on retrieval actually do it, and it seems a good explanation only because it attaches a different meaning to the term 'to retrieve'.

The term 'content-addressable memory' is a related confusing term. It is used to refer to two distinct entities:

1. A database system with a clever and fast search algorithm. These systems can perform retrieval in the normal sense (proper retrieval), but are not implementable by neurons, because they invariably use pointers (see brain symbols for discussion).
2. A system of activation in a network. These are implementable by neurons, but cannot perform proper retrieval.

The term 'specific' (as in 'specific connection') is also used in a confusing manner. Neurobiologists commonly use it where other domains would use terms like 'weakly constrained'. For example, a neurobiologists may say that an axon forms specific connections to the fourth layer of the cortex. However, this means that the axon may form connections with any of the neurons which has dendrites in the fourth layer in the region it innervates the cortex. The number of these is the order of magnitude of several tens of thousands, and the selection between them seems to be stochastic. On top of this, the location the axon innervates the cortex is not defined accurately either. Thus the connection are not actually specific, in the sense that they would differ between different brains in a stochastic manner {7}.

However, when researchers from other domains are considering possible models, they commonly hang on the assertion that there are many specific connections in the cortex. This allows them to build models which rely on specific connections in the computer science sense, and these models are biologically nonsense.

'Representation' is another example of a confusing usage. In one extreme, a 'representation' of an entity is any feature in the brain that corresponds (in some sense) to thinking about this entity. With this definition, the brain necessarily has representations. In the other extreme, "A representation is a formal system for making explicit certain entities or types of information, together with a specification of how the system does this." (Marr, Vision, P.20. See in Vision[I.1] for full discussion).

Thus it can be 'proved' that the brain has a formal system, by saying that the brain has representations (the first sense), and representations (the second sense) are formal systems. This argument, or somewhat weaker version of it, is normally implicit in the assumptions that people make, although sometimes half of the argument appears, with the other half being implicit. The example in Vision in the previous paragraph is the only case I found of both halves of the argument in the same text, but even there they are separated( [26 Jul 2003] more about representations).

'Representation' has another confusion associated with it. In many cases, people assume that 'Representations' have to be manipulated by some 'user', which leads to the necessity of a symbolic system. However, if the 'representations' are active entities (i.e. they affect parts of the system directly), for example if they are patterns of neural activity, then the 'representations' can do the manipulation themselves. That possibility eliminates the need for the 'user', and hence for a symbolic system. {8}

'Awareness' also has a mixed bag of meanings. It is used to refer to the ability to recall the thinking process and the ability to report about internal states, in the sense that this is how it is measured. However, most of the people that discuss awareness seem not to believe that an explanation of these two abilities explains awareness. For example, when Francis Crick discusses visual awareness, he does not ask what are the mechanisms that allow us to recall what we see and the mechanisms that allow us to report what we see. Hence, he does not understand 'awareness' as the abilities to recall and report, but something else, which he does not make explicit.

'Free will' is another example, as discussed in the the methodological problems page. I suspect that for many people 'free will' implicitly means that it is free from everything, including the activity of the brain, and they actually believe that we do have this freedom, but they don't dare to admit that. 'Intentionality' is a related confused term, without any real meaning.

'Specialization' is also used in a confused way. Normally, the meaning of "specialization of some entity for some task" is that the entity has some features that are specific for performing the task, which other entities of the same type don't have. In cognitive science it is used in many cases to describe a situation where all that is known is that the entity perform (at least some part of) the task. This usage implies (because of the normal meaning) that the entity has some specific features for the task, even if it is not intended. This usage means that "specialization" can be proved simply by showing that some part of the brain perform some task, and hence to "prove" (by implication) specific features for the task.

A typical example is the assertion that "the Broca area is specialized for language productions", which probably most cognitive scientists will endorse. The truth is that there isn't any evidence for special features in the Broca area compared to the rest of the cortex, and by now we know enough to be sure that there aren't such features (All the data can be explained by the Broca area being in the right position to interact with all the motor areas that control the vocal tract). Nevertheless, the assertion strongly implies such modifications, and people that don't know enough about the morphology of the cortex (which include most of cognitive scientists) would take it to mean that the Broca area has specific features for language production.

"Reorganization" also used in neuroscience in a confusing way. When the word is used about a network of neurons, the obvious meaning is changing the connections between them. However, neuroscientists many times claim to show reorganization of some part of the brain, even though their data just show changes in the patterns of activity in this part. What the neuroscientists think about this is not obvious: I think in some cases they really believe that changes in patterns of activity necessarily means changes in connections. In the other cases, they simply use "reorganization" with a different meaning, but do not bother to tell the readers. In both cases it creates the impression that there is a lot of evidence for changing of connections when there isn't.

"Hierarchy" is a source of confusion too. The normal definition of "hierarchy" implies well-defined relations between nodes and their super-nodes and sub-nodes. In neuroscience, the word is used many times even when there aren't such well-defined relations. For example, the fact that as we proceed away from the location of the sensory input we find more complex responses is quite frequently presented as showing hierarchy in the cortex, even though this observation tells us nothing about the organization of the cortex (because any network will show this feature). It is not always obvious whether the writer/speaker really thinks in that case that the observation proves a hierarchy, or just uses it as a buzzword.

[3.19] The dissociation (and double dissociation) myth

By Dissociation I mean differential response of the performance in two tasks to the same manipulation. Here I discuss the case where the 'manipulation' is brain damage, because this the most important case, but the same arguments apply to other cases.

In a case of brain damage, the dissociation means that the performance in one task (Task-A) don't change (or only slightly) and the performance in other task (Task-B) deteriorates. Double dissociation is the combination of the latter with the reverse case, i.e. deterioration of Task-A while Task-B is unchanged.

brain damage dissociation, particularly double dissociation, is regarded by many as almost absolute proof that the two tasks are handled by separate systems (modularity). This is based, usually implicitly, on the belief that if the two tasks where handled by the same system, they would deteriorated by the same rate.

This belief, however, is wrong. Even if the tasks are handled by the same system, the difference between them will cause the system to use a different set of resources for each task (in the brain, that means different sets of neurons). These sets may overlap extensively, but there would be a subset of resources which are used only for Task-A, and the same for Task-B. If the Task-A-specific subset is damaged badly , but the rest of the system is damaged only lightly, then the performance in Task-A will deteriorate, but the performance in Task-B will not change. The same can happen the other way. Thus for each pair of tasks, which are handled by the same system, we should expect some cases of a dissociation in both ways, i.e. double dissociation.

Because dissociation (and double dissociation) is expected to occur for tasks that are handled by the same system, its occurrence cannot be used as evidence for or against modularity.

The way I explained this error, it is a typical case of mis-analyzing the null hypothesis [3.21, below]. This mis-analysis is sometimes (maybe mostly) a result of the 'ignoring complexity' error, when it is assumed that a system is an entity without internal features. Alternatively, it may be based on the modularity assumption (which is what it is supposed to support). Another possibility to get this error is a failure to grasp the nature of evidence, in particular that it is not enough that an evidence will be compatible with a proposition to support, it also has to be incompatible with all other resonable propositions.

To actually deduce anything from the double-dissociation error, it also requires the sameness assumption.

[17 Mar 2003] There is a discussion of the validity of double dissociation in the February edition of Cortex (article by Dunn and Kirsner at the top and many comments in the discussion at the bottom of the page). This guy actually makes the same point that I make above. The comment by Baddeley makes the "of mis-analyzing the null hypothesis" error explicitly, when it says: "If two plausible measures of underlying function behave differently, then it does provide at least prima facie evidence for their separability." If the "their" refers to the "measures", this sentence is empty of contents, so it must be refering to the "functions". With this interpretation, this is a baseless assumption. Baddeley simply takes his intuition of the conclusion as the null hypothesis. ( This one also contains useful discussion, though not really about double dissociation. )

[3.20] Relying on introspection

To actually do this error, a person also needs to ignore the complexity of thinking, i.e. do the 'ignoring complexity' error.

[3.21] Misanalyzing the 'Null Hypothesis'

CogPsys commonly misanalyze the 'Null Hypothesis', i.e. what would be expected if their hypothesis is wrong. For example, some CogPsys assume that there is a built-in mechanism that makes humans regard other humans as special entities, based on the observation that all humans do it. This ignores the fact that because of their abilities, humans are special, and for a human, this is enhanced by the physiological similarity. Thus we would expect humans to regard other humans as special in any case, so the fact that they do it does not support a built-in mechanism for it.

The dissociation myth [3.19] is another example. The same line of argument is commonly used for various arguments. Some examples are:

• Penrose argues that the brain is not a sound machine, and therefore is based on unknown physical principles, ignoring the fact that as far as we know (from neurobiology and developmental studies) the brain cannot be a sound system with the known physical principles anyway.
• The treatment of the Self as special is sometimes argued to be built-in, ignoring the fact the Self is necessarily special.
• The 'unity of experience' is sometimes argued as a special attribute of humans, even though this is the obvious natural state, because the world we experience behave as a single world, so humans learn to perceive it this way. For example, if a person is shown a confusing picture which can be interpreted in several conflicting ways, he/she tends to interpret it in one way, because in the real world the conflicting interpretations cannot exist simultaneously.
• Linguistic similarities are taken as proof for innate language mechanism. For example:
  • The recursive nature of language - this ignores the fact that all mentally healthy human can easily cope with recursive ideas, and therefore need language that can express them (and by definition any language that can express recursive ideas is recursive).
  • The fact that languages are open ended, with no restriction on nesting and combinations - this ignores the fact that lack of restriction is (obviously) the default state.

The argument of 'lack of negative evidence' which is used by Chomsky can also be regarded as this kind of error, because it ignores the fact that human can (and do) use more subtle information than direct instruction (e.g. failures to communicate, absence of structures from the language, their failure to find a coherent interpretation of a sentence, etc.). however, this is probably based on assuming some model of the way humans perform language tasks, so may be actually a 'Knowing what humans do' error.

[3.22] Matching model to optimal behaviour

A special case of the 'mis-analysis of null hypothesis' error (above) , which is very common, is matching a model to an optimal behavior. When an intelligent system performs a task that is relatively simple (for the system), the null hypothesis has to be that it will perform the task using optimal strategy, independently of the internal mechanisms of the system. Thus, when a system uses (approximately) an optimal strategy in performing some task, this cannot tell us anything about the internal mechanisms of the system, unless it can be shown that it is a complex task for the system. If it is a simple task for the system, the approximate optimal behaviour is the null hypothesis. This, however, does not prevent CogPsys from constructing models which match an optimal behavior of humans on simple tasks, and claim that this match supports their models.

[ 15 Jan 2004 ] This (Nature 427, 244 - 247 (15 January 2004)) is a typical example. What they find is that human performance fits with the best mathematical model, and therefore concludes that humans use it, which is simply lousy logic. All they can deduce is that humans don't use less effective models.

[3.22] Theories and testable hypotheses