Measuring Intelligence: Facts and Fallacies

by David J. Bartholomew

Overview

This book penetrates the thicket of controversy, ideology and prejudice surrounding the measurement of intelligence to provide a clear non-mathematical analysis of it. The testing of intelligence has a long and controversial history and whether intelligence exists and can be measured still remains unresolved. The debate about it has centered on the "nurture versus nature" controversy and especially on alleged racial differences and the heritability of intelligence.

Product Details

ISBN-13:	9780521544788
Publisher:	Cambridge University Press
Publication date:	08/26/2004
Edition description:	New Edition
Pages:	186
Product dimensions:	5.98(w) x 8.94(h) x 0.55(d)

About the Author

David J. Bartholomew is Emeritus Professor of Statistics, London School of Economics, Fellow of the British Academy and a former president of the Royal Statistical Society. He is a member of the editorial board of various journals and has published numerous books and journal articles in the fields of statistics and social measurement.

Read an Excerpt

Cambridge University Press
0521836190 - Measuring Intelligence - Facts and Fallacies - by David J. Bartholomew
Excerpt

1 The great intelligence debate: science or ideology?

The noise of battle

Almost everyone uses the word intelligence but it is one of those Humpty Dumpty words whose meaning is so elastic that it can cover virtually anything we choose. Lack of clarity does not make for rational discussion, but that simple fact is not enough to account for the ferocity with which the intelligence debate is often conducted. Every so often a spark ignites the dry tinder and the arguments flare up again. One of the most spectacular recent displays was triggered by the publication of The Bell Curve in 1994 by Herrnstein and Murray.¹ This book created quite a stir at the time and it has now become a point of reference for current exchanges - even if the temperatures are lower. It gave a new lease of life to a controversy with a long history going back at least to Sir Francis Galton towards the end of the nineteenth century.²

On the face of it, the book appeared to be a straightforward, thorough and clear exposition of a body of social science research. Its object was to explore how many of the problems affecting contemporary American society could be explained by variation in IQ (the intelligence quotient - a measure of intelligence) among individuals. It is clear from the flood of literature which followed in its wake, mainly highly critical, that there was much more to this than met the eye. The titles, alone, of some of the books which followed in response to The Bell Curve convey the strength of feeling which was engendered. Measured Lies was described on the cover as 'the most thorough and cogent analysis of the tangle of pseudo-science and moral dishonesty that comprises the frozen heart of The Bell Curve'. Inequality by Design: Cracking the Bell Curve Myth and The Bell Curve Wars: Race, Intelligence and the Future of America are two more examples of the same genre.³ Writing in Measured Lies, Joyce King could hardly contain her contempt for the statistical treatment when she wrote 'The Bell Curve paradigm employs a number of "conjuring tricks", including statistical associations, that magically become causal relationships' (p. 182) and, again, 'I categorically reject the specious premise of the Bell Curve argument: that science has demonstrated that human intelligence, like height or weight, is a measurable, normally distributed natural law like thing in itself '(p. 185). What has science demonstrated we may well ask? King's 'shot gun' approach could hardly fail to find a legitimate target somewhere. It is indeed true, as we shall see later, that the normality implied by the Bell Curve is a convention, not a fact, but the reason for this, and its implications, go much deeper than this superficial tirade comprehends. It is clear that The Bell Curve touched a raw nerve in some quarters and the reasons go very deep. Until we understand why this is so it will be impossible to get at the truth beneath the noise and fury.

It would be a mistake to jump to the conclusion that the storm occasioned by the publication of The Bell Curve was the product of the peculiar circumstances of America a decade or more ago and that it would blow itself out in due course. Surely, we might think, there are well tried methods by which such things can be investigated without the need for so much public excitement? One would have thought that the measurement of intelligence would have been one of those worthy but unexciting statistical activities - like the production of the Retail Prices Index - vital for the well-being of society but best left to those expert in these things. But this is not so, and the reason is that the whole question is bound up with our nature as persons. The 'hard' sciences like physics and chemistry seem to deal with hard facts about the physical nature of the world and scientists of all backgrounds can work together unencumbered by their ideological baggage. Everything changes as soon as we cross the frontier, and bring the human person into the picture. There is a deep cleavage in the approach to the scientific study of the human individual, not only between the scientific community and society at large but, more significantly, within the scientific community itself. In the 1960s the utterances of the Nobel prize-winning physicist William Shockley and the psychologist Arthur Jensen⁴ touched a similarly raw nerve, to such an extent that the debate on intelligence ceased to be a purely intellectual contest and resulted in large-scale protests on American university campuses and, even, in some instances, in physical violence. The 'giant killer' on that occasion was Stephen Jay Gould whose book, The Mismeasure of Man, sought to demolish the arguments of the 'biological determinists' with a mixture of detailed examination of the evidence and verbal fisticuffs.⁵ Gould returned to the fray with a new and enlarged edition in 1996 as his contribution to the debate on The Bell Curve. We shall return to this later, not only because it is one of the most cogent attacks, but because it seems to be widely regarded as the definitive refutation of the theoretical underpinnings of Herrnstein and Murray's case. Even Gould did not satisfy the likes of Joyce King because, to her way of thinking, his argument was conducted within a framework (paradigm was her term) which she saw as fundamentally flawed. Other, more modest but critical, treatments of the measurement of intelligence will be found in Richardson's The Making of Intelligence and Howe's IQ in Question. Among other contributions to the debate, less conspicuous perhaps, we shall take time to look at the arguments of Steven Rose in Lifelines where a number of widespread misapprehensions are given further currency.⁶ Writers like these are sure to be quickly on the scene whenever the media decide to give the topic of intelligence a fresh airing.

Why such sensitivity?

It is worth spelling out, albeit in outline form, why intelligence is such a sensitive issue. Those on the right of the political spectrum tend to start with the primacy of the individual and the individual's freedom. Society, as they see it, is an aggregate of individuals and the ills of society are to be understood as arising from individual choices. Human diversity is to be encouraged. Equality is to be seen as equality of opportunity rather than of ability or achievement. Intelligence, like virtually all other human attributes, varies. A healthy and just society depends on recognising that fact and on the arrangement of its affairs so as to harness this diversity to serve the common good. Those on the left start at the other end, with society. It is society which moulds individuals as it constructs roles for them. The ability of individuals and the contributions they can make to society depend primarily on nurture. It is an egalitarian philosophy which sees inequality as arising from unjust social systems rather than from lack of enterprise or ability. When things go wrong the blame falls on society not the individual.

To some at least, the mere suggestion that people might vary in their innate abilities is deeply threatening. It places a fundamental inequality where ideology posits equality. Whereas inequalities created by an unequal distribution of resources can be remedied, any which are innate cannot be changed. They are made more pernicious by the alleged fact that these innate factors are perpetuated as they are passed down the generations in the genes. Money spent on schemes to eradicate innate inequality is thus money wasted trying to change the unchangeable.

In reality, this highly simplified caricature is complicated by a climate of mistrust, greed, the desire for power and all the other motives which commonly lie behind the propagation of high minded principles. Little wonder then that science is quickly branded as pseudo-science when it presumes to threaten such deeply held convictions.

The problem actually goes deeper still. It is not variation as such which causes the trouble. After all, we vary in our tastes in food and music. Some of us are tall and others short. Some are athletic and others are not. Except in rather restricted circumstances, such as among school children of a certain age, such things are not equated with feelings of inferiority or superiority. What is it about intelligence that links it so intimately with human worth?

It is obviously something to do with our brains, the seat of our consciousness, but it is what we do with those brains that counts. Intelligence is held to influence our ability to acquire power, influence and wealth. In short, those who have it in good measure are seen as being better endowed and able to lead a fuller life. Perhaps the link derives from the ultimate source of our values and beliefs; it may also have to do with the fact that they have changed in much of western society as a whole. The Judeo-Christian belief in 'Man made in the image of God' provided a sufficient basis for equality. With that fact securely in place, it was easy to be relaxed about other inequalities which spoke of the diversity of the creation rather than differences in worth. Without that foundation, one has to cast around for some other basis for ascribing worth to individuals. Science is seen, by some, as providing the only objective basis of knowledge and hence as the obvious route to a rational assessment of the human person. What is more natural then than to turn to a measure of our most distinctive and impressive attribute - our intelligence? For many therefore, intelligence and intelligence testing impinge on their understanding of themselves at the deepest level. It is hardly surprising that they find it hard to be rational about it.

What is intelligence?

We start with the term intelligence itself. It is clear from the way the word is used that we think of it partly, at least, as quantitative. People are described as being 'more' or 'less' intelligent. Dictionaries are the arbiters of what words are supposed to mean. In this case the Shorter Oxford Dictionary gives a wealth of material which focuses on the notion of 'understanding'. Thus it speaks of intelligence as: 'the faculty of understanding' and, more helpfully, as 'quickness of mental apprehension'. The quantifiability of intelligence, in common usage, is attested in 'understanding as a quality admitting of degree'. So far so good but none of this helps us very much in constructing a scale of measurement. Some would argue that, in spite of the way we use the word, intelligence cannot be measured at all. According to them the project falls at the first fence because it is futile to try to measure the immeasurable.

Before examining this argument it will help to look beyond the dictionary to see what the founding fathers of intelligence testing thought they were trying to measure. All of those quoted below were, or are, closely involved in the practice of measuring intelligence so let us see what they have to say.

Starting with David Wechsler who gave his name to widely used tests, it is:

. . . the aggregate or global capacity of the individual to act purposefully, to think rationally, and to deal effectively with his environment.

Sir Cyril Burt, one of the founding fathers of the intelligence testing movement, called it:

. . . innate general cognitive ability.

Even Howard Gardner, who some see as an opponent of IQ testing and as the originator of a richer and more relevant approach to human abilities says, of any 'intelligence':

To my mind, a human intellectual competence must entail a set of skills of problem solving - enabling the individual to resolve genuine problems or difficulties that he or she encounters and, when appropriate, to create an effective product - and must also entail the potential for finding or creating problems - and thereby laying the groundwork for the acquisition of new knowledge.⁷

Herrnstein and Murray opted for

. . . cognitive ability.

One definition, which it is claimed would cover what is contained in the Handbook of Human Intelligence,⁸ is given by Sternberg and Salter in their introductory chapter as:

. . . goal-directed adaptive behaviour.

It seems clear enough that these various forms of words (and many others which could be quoted⁹) collectively justify the claim that we have a fairly clear idea, in general terms, of what we are talking about when we speak of 'intelligence'.

Unsympathetic writers have been more concerned to point to what they see as the triviality of some attempts at measuring intelligence by offering 'negative' definitions such as, for example, 'intelligence is not about doing tests'.¹⁰ As a sign of desperation - or, maybe, terminal cynicism - some have turned the normal order of things on its head and defined intelligence, tongue in cheek perhaps, as 'what intelligence tests measure'.¹¹ The trouble, of course, is that there are many such tests and they do not necessarily all measure quite the same thing. An unambiguous definition in these terms would therefore have to pre-suppose universal agreement on the definitive test.

However, those seeking to ridicule the whole notion that intelligence can be measured at all (Michael Howe,¹² for example) merely see this negative approach as an 'own goal'. This definition appears to be no more than a tautology defining intelligence in terms of its own definition! This is a too superficial analysis and we shall have to return to the deeper issues after the foundations of our own argument are in place. For the moment it is sufficient to observe that the response depends entirely on the process by which the measure is constructed.

Even this brief excursion into the stormy waters of definition shows how easy it would be for the whole enterprise to be shipwrecked. For if we cannot define what it is that we wish to measure with precision, how can we expect to find an agreed measure? The more serious researchers have recognised this hazard and have sought an alternative route to the scientific study of mental abilities. Charles Spearman¹³ was, perhaps, the first a century ago, and he proposed to abandon the word altogether and substitute, in its place, the symbol g which referred to a quantity appearing in the new theoretical framework which he was proposing. Merely replacing a word by a symbol does not, of course, change anything and some would interpret it as no more than a smoke screen to confuse those who panic at the first sign of mathematics. However, that was not what Spearman actually did, as we shall see in chapter 2. He defined a process - or a model as we might call it today - which gave meaning to the concept. Arthur Jensen¹⁴ followed Spearman in arguing that although the word 'intelligence' serves well enough in everyday language, it is too ambiguous for scientific use. He therefore called his major book on the subject The g-Factor.

There is a second quantity, usually abbreviated to IQ (for Intelligence Quotient), which figures more prominently than g in discussions of intelligence. It is often treated as if it were synonymous with 'intelligence' and, sometimes also, with g. These two quantities are central to the purpose of this book and, in separating the science from the ideology, it is crucial to understand that they are distinct entities. We shall return to this distinction shortly but, having made the point, there are some other issues to be dealt with first. The most important at this stage concerns whether intelligence can be measured at all. For if it cannot, there is little more to be said.

Can intelligence be measured?

Steven Rose¹⁵ has long been adamantly opposed to what he calls the psychometric approach to intelligence testing and the subject re-surfaces in his book Lifelines. His treatment is based on misunderstandings which are quite common and therefore worth examining carefully.

Rose makes two basic criticisms. The first, which he calls improper quantification, is simply that things like intelligence cannot be quantified. The second, to which we shall come in a later chapter, relates to the use of the normal distribution in intelligence testing. The assumption, he writes, 'that any phenomenon can be measured and scored, reflects the belief . . . that to mathematize something is in some way to capture and control it' (p. 184). Thus, for example, the fact that some people are more violent than others does not, according to Rose, imply that the degree of violence which they display can be expressed on a numerical scale. This is even more true, he would argue, of intelligence. According to Rose, intelligence is a far more subtle and many-sided thing which makes it absurd to force it into the narrow mould of a single dimension. The alleged fallacy is that 'reified and agglomerated characters can be given numerical values'.

However, it is hard to make Rose's case without slipping into the very quantitative language which his argument sets out to undermine. A recent issue of the Times Higher Educational Supplement, for example, published an article by a philosopher (David Best) entitled 'Why I think intelligence cannot be tested.' Definitions of intelligence, the author writes, 'tend to be both tendentious and fallacious'.¹⁶

The problem is that the concept is twisted so that it can be empirically tested. According to the author, intelligence is not about doing tests, which tend to be concerned with speed of performance and such like, but about creativity and willingness to grapple with ideas. Not for the first time, we are reminded that Einstein was slow as a child and therefore, by contemporary testing standards, not very intelligent. Nevertheless, it emerged that the author thought that some sorts of intelligence could be measured and he, himself, spoke of 'lower intelligence' which surely implies that some people have less of it than others. Few, perhaps, would go as far as the newspaper columnist, quoted in note 10 of this chapter, who inverted the usual order of things by claiming that those who possessed what he regarded as true intelligence would find themselves at the bottom of the class if they deployed that attribute in IQ tests! The real burden of these complaints, as with so many other criticisms, is that intelligence is too subtle to admit of measurement on a simple linear scale.

This is true, and it is difficult to imagine that any serious investigator would wish to defend the proposition that all that is meant by the word intelligence can be captured by a single number. Indeed, that is one reason why most serious scholars, from Charles Spearman onwards, have been wary of using the term 'intelligence' at all. What is equally true is that we constantly make such comparisons by speaking and behaving as though some individuals were in fact more intelligent than others. How are we to reconcile these apparently conflicting views of the situation?

To anticipate a discussion, which can only be fully developed later, the answer lies in the fact that intelligent behaviour is not a one-dimensional phenomenon - it has many facets. We are very familiar with this sort of thing in other spheres and, in most contexts, it is entirely uncontroversial. Judges of ice skating know that the quality of performance cannot be adequately measured on a single dimension. So they rate skaters on two dimensions - artistic performance and technical merit. Wine tasters recognise several dimensions along which wines vary and any assessment of overall quality must somehow take all into account. Intelligence, as generally understood, is no different. There can therefore be no serious scientific claim to have captured this phenomenon in a single number. To assert otherwise is simply mischievous.

Thus far Rose, and those who think like him, are right, but it could turn out to be the case that there is one dominant dimension along which individuals vary much more than any other. Furthermore, if this same dimension were to emerge almost regardless of what set of test items were used, providing only that they reflected what we commonly understand by intelligent behaviour, then we might begin to suspect that it described something fundamental and important. What psychometricians, from Spearman onwards, have done is to focus on this particular major dimension, which seems to be important in describing human variability, and give it a name. The reason for calling it g was precisely to avoid the legitimate criticism that the term intelligence was not precise enough and not, as Rose fancifully speculates, to give it the fundamental status of the gravitational constant - also known as g!¹⁷ Rose is wrong to claim that 'reified and agglomerative characters' cannot be given numerical values. They obviously can and, as we shall see, factor analysis provides one way of doing this. What can and must be questioned is whether those numerical values adequately capture the essence of the comparison which we are making when we say, for example, that one person is more intelligent than another. This is an empirical question which has to be answered by bringing the data into dialogue with the usage of the term in everyday language.

Do measures of intelligence have any use?

However satisfied we might feel at having constructed a measure of intelligence, the exercise is pointless unless there is something useful which we can do with it. There are two ways of approaching this question. One is at the pragmatic level, where we ask whether it enables us to predict anything which it would be useful to know. For example, does it make selection for particular jobs more efficient? One of the early such uses was in the US air force where it was necessary to identify those with the ability to become good pilots. Educational selection was, perhaps, the major application envisaged. Binet's¹⁸ motivation was to identify children who would benefit from extra help. Selection for different types of secondary education was the main use of intelligence tests in Britain after the Second World War. The fact that they are not now used on such a wide scale in education has more to do with changing ideas in education and society rather than the efficacy of the tests. In any case there is one important point to be made about the use of non-specific tests for general intelligence which is often overlooked. Selection for any particular task can probably be best made using a test designed specifically for that purpose. However, this may be a time-consuming and costly business. The value of a general test is that, although it will seldom be the best instrument available, it will serve a great variety of purposes adequately. It is useful to an employer in much the same way as is a university degree in literature or philosophy - it is an indication of general ability rather than a particular set of skills. As a general rule one might surmise that the less specific the test, the more widely useful it would be.

Testing has, traditionally, been seen as useful in the field of education, broadly interpreted. More recently its potential use in medicine has become apparent. For example, mental tests can be used to monitor the progress of degenerative brain conditions such as Alzheimer's disease.

There is a second and less pragmatic kind of usefulness. If intelligence is an important human characteristic, it will have far-reaching implications for the way that society works and the life experience of its members. The just and efficient ordering of society depends, among other things, on being able to trace the lines of cause and effect linking policy decisions to outcomes. The authors of The Bell Curve were concerned to show that intelligence was a causal factor in a great many situations of social interest like poverty, income and marital stability. Whether or not individual differences are innate is clearly important if one wants to eliminate them. We are a long way from being able to say what is feasible or possible in some cases but these remarks show how much is at stake. The kind of question we should be asking is: does our measure explain anything in a coherent and clear fashion which, before it was available, was obscure? Is it an economical way of structuring our thinking in this particular field? Does it enable us to do anything useful that we could not do beforehand? Such questions lie behind the summing up in chapter 13.

Ideology re-visited

Now that we can view them in a broader context, it is helpful to return to the ideological questions raised earlier. The statistical exercise of constructing a measure of anything only becomes threatening at the point when it comes to be used. Measures of IQ were used by Herrnstein and Murray to investigate how far variables of social interest, like income or socioeconomic class, depend upon IQ. At first sight this may seem an innocent enough matter, but it is clear that many of the critics saw it as a thinly veiled attempt to advance a right wing political agenda. The authors of The Bell Curve were accused of pursuing a right-wing policy under the guise of social science. The alleged pernicious nature of the publication was magnified by the false picture which it presented to the world, at least as seen by many of its critics. A particularly critical matter concerned the hardly noticed transition from an empirical quantity like IQ to a more permanent characteristic of an individual envisaged as an unchanging, and perhaps unchangeable, personal attribute. Here we are back to fundamental questions about the nature of the human person and the threat posed by social science to deeply held views.

To illustrate how far beyond the boundaries of science the animosities run, it is instructive to trace the battle lines on the subject of funding. Opponents of The Bell Curve made much of the financial support provided by the Pioneer Fund. The fund was apparently founded in 1937 to promote the procreation of white families living in America before the Revolutionary War. According to Giroux and Searls, writing in Measured Lies (p. 79), this had been described by the London Sunday Telegraph as a 'neo-Nazi organisation closely integrated with the far right in American politics'. The message which comes across is that Herrnstein and Murray's work is a pseudo-scientific attempt to bolster up a pernicious doctrine and that the involvement of the Pioneer Fund is evidence of this. In short, tainted money cannot produce credible science.

The other side, unsurprisingly, saw things rather differently. Charles Murray in his Afterword to The Bell Curve pointed out that today's Pioneer Fund is very different from its origins. He claimed that of over a 1000 scholars whose work was cited in The Bell Curve his opponents could identify only thirteen who were funded by this 'tainted' source. In any case, most of these 'tainted' articles were published in reputable peer reviewed journals. From that side, the Fund was seen as having the courage to support fearless researchers who were prepared ask questions which the politically correct establishment would prefer to sweep under the carpet.

Neither side seemed over much concerned with whether the research was good science!

Our primary object in this book is to enable the reader to distinguish the science from the ideology. This is not to deny a place in policy making to extra-scientific matters. People have different beliefs and value systems and this fact is recognised in democratic societies where procedures exist to resolve the conflicts resulting from them. Scientific method, on the other hand, exists to discover those things about the world on which, given sufficient evidence, rational people should be able to agree. In matters of science we do not resort to the ballot box to resolve differences but to the data and the methods of analysing them. For this we need the expertise of those with the qualifications and experience to analyse the data. But who are the experts in this field?

Who are the experts?

To what kind of expert should we turn to resolve the often bitter disputes which rage in this field? Broadly speaking, the social sciences are concerned with the behaviour of people. Psychologists obviously have a strong claim to be heard as they study individual behaviour and have a long tradition of measuring aspects of it. Psychometricians, in particular, have made it their business to construct measures of all kinds of human ability. Historically, they were first on the scene with Charles Spearman's investigations of intelligence. We have noted that it was he who introduced g as the representation of what he believed to be the major dimension of human ability. His successors developed and elaborated his ideas, giving birth to factor analysis which has provided the theoretical framework for investigating these matters and to which we shall return. Sociologists have also staked their claim on the grounds that the development of human abilities takes place in the context of society and is strongly influenced by social interactions.

---

© Cambridge University Press

Table of Contents

1. The great intelligence debate: science or ideology?; 2. Origins; 3. The end of IQ?; 4. First steps to g; 5. Second steps to g; 6. Extracting g; 7. Factor analysis or principal components analysis?; 8. One intelligence or many?; 9. The bell curve: facts, fallacies and speculations; 10. What is g?; 11. Are some groups more intelligent than others?; 12. Is intelligence inherited?; 13. Facts and fallacies.

Editorial Reviews

'This book represents a step forward in the debate on measuring intelligence. It is essential reading for anyone concerned with the 'intelligence debate'. It will also make excellent reading for anyone learning about factor analysis, and provides a perfect illustration of the Bartholomew school of measurement models.' International Statistical Institute

'… unquestionably well researched and very thorough.' Significance

'… the author manages to present his ideas in a way that is both accessible to the general reader and useful to scholars in the relevant fields.' Science Direct

'… commendable for its thoughtfulness and good judgment.' Intelligence 33

From the Publisher