What do IQ Tests Test?

If you think IQ tests test intelligence, you might ask yourself: How do you know?

What is intelligence?

Give us your definition.

Or by intelligence do you just mean IQ, in which case your understanding of intelligence is tautological: intelligence is performance on an IQ test, which is measured by performance on an IQ test.

IQ and wealth at low scale (outside the tail).
Mostly Noise and no strikingly visible effect
above $40K, but huge noise. Psychologists
responding to this piece do not realize that
statistics is about not interpreting noise.
From Zagorsky (2007) via Nassim Taleb.

Fact is, the IQ testers never have defined intelligence. Instead, they came up with a set of puzzlers and declared your score on this test is the measure of your intelligence.

Trouble is, the IQ-ists have never attempted to show how IQ test results relate to what is understood by the term intelligence as manifest by, say, the creative work of a poet, a painter, a composer, an architect, or a scientist, or the more mundane endeavors of a politician, a policeman, a prostitute or a peddler of illicit drugs. Their strongest claim is that IQ tests predict "career success", except as anyone who looks into the matter will find, the correlation is trivial.

The best that IQ tests can do is provide an indication of academic aptitude, but even in that case they don't work well, not even as well as traditional academic exams, which is why Harvard and other such places use a SAT test with a mathematical component and a verbal component, the results of which are by no means closely correlated

Better still, obviously, would be to break things down further, e.g., SAT Physics, SAT Biology, SAT English, SAT music, etc. Except that would be giving up, since it would be an acknowledgement that the whole idea of intelligence as unitary feature of mind measurable on a single linear scale is nonsense.

Related:
Nassim Taleb: The psychologists' construct, general intelligence, is based on a statistical error.

IQ-ism, a Fake Science Serving the Fascist New World Order

Psychologists generally assert that IQ tests provide the best predictive measure of individual life success, hence the need to give everyone an IQ label, the better for schools, employers and the world at large to judge their intellectual merit.

This notion is tremendously appealing to those intent on creating the Fascist New World Order, which is to say a bureaucratic global dictatorship, controlled by the Money Power, which requires a submissive populace brainwashed into a belief in its own mental inadequacy, and therefore, its own incapacity for democratic, national self-government.

What the promoters of IQ-ism assert by implication is that:

You have an IQ less than those set in authority over you, which means that your judgement is invariable inferior to that of those who dictate the conditions of your existence. Hence, cease your clamor, do as you are told, and be content with whatever rights and freedoms we, your rulers, in our wisdom, are prepared to grant you.

Here for instance is Jordan Peterson with expert-level hand-waving, brainwashing and bullying a University of Toronto undergraduate psychology class:

One of the things I have to tell you about IQ research is that if you don't buy IQ research, you might as well throw away all the rest of psychology, and the reason for that (blah, blah, blah (watch it here) Or in other words: Question what I have to say about IQ and you will be judged mentally unfit for education as a psychologist, and you might as well quit the course now.

In a discussion of the recent Boeing 737MAX crashes at the Unz Review, many commentators seemed content to attribute these disasters to the presumed low IQ of the Third World pilots flying the planes. Here, for example, is one that gets right to the point:

Boeing is great/ dumb Third World pilots.

In response to such views, I quoted a couple of paragraphs from an essay on IQ by the well known financial analyst, Nassim Taleb:

“IQ” is a stale test meant to measure mental capacity but in fact mostly measures extreme unintelligence (learning difficulties), as well as, to a lesser extent (with a lot of noise), a form of intelligence, stripped of 2nd order effects — how good someone is at taking some type of exams designed by unsophisticated nerds. It is via negativa not via positiva. Designed for learning disabilities, and given that it is not too needed there (see argument further down), it ends up selecting for exam-takers, paper shufflers, obedient IYIs (intellectuals yet idiots), ill adapted for “real life”. The concept is poorly thought out mathematically by the field (commits a severe flaw in correlation under fat tails; fails to properly deal with dimensionality; treats the mind as an instrument not a complex system), and seems to be promoted by: — racists/eugenists, people bent on showing that some populations have inferior mental abilities based on IQ test=intelligence; those have been upset with me for suddenly robbing them of a “scientific” tool, as evidenced by the bitter reactions to the initial post on twitter/smear campaigns by such mountebanks as Charles Murray. (Something observed by the great Karl Popper, psychologists have a tendency to pathologize people who bust them by tagging them with some type of disorder, or personality flaw such as “childish” , “narcissist”, “egomaniac”, or something similar). — psychometrics peddlers looking for suckers (military, large corporations) buying the “this is the best measure in psychology” argument when it is not even technically a measure — it explains at best between 2 and 13% of the performance in some tasks (those tasks that are similar to the test itself)[see interpretation of .5 correlation further down], minus the data massaging and statistical cherrypicking by psychologists; it doesn’t satisfy the monotonicity and transitivity required to have a measure (at best it is a concave measure). No measure that fails 80–95% of the time should be part of “science” (nor should psychology — owing to its sinister track record — be part of science (rather scientism), but that’s another discussion). — It is at the bottom an immoral measure that, while not working, can put people (and, worse, groups) in boxes for the rest of their lives. — There is no significant correlation (or any robust statistical association) between IQ and hard measures such as wealth. Most “achievements” linked to IQ are measured in circular stuff s.a. bureaucratic or academic success, things for test takers and salary earners in structured jobs that resemble the tests. Wealth may not mean success but it is the only “hard” number, not some discrete score of achievements. You can buy food with a $30, not with other “successes” s.a. rank, social prominence, or having had a selfie with the Queen. Read more

This prompted a response to me from University of London IQ psychologist, James Thompson:

Have you have also read my replies to Taleb? http://www.unz.com/jthompson/swanning-about-fooled-by-algebra/ http://www.unz.com/jthompson/in-the-wake-of-the-swan/

Which provided the opportunity to express more fully than before why I believe that IQ-ism is fake science:

I've had a look. But as I'm sure you will agree, to review your response to Taleb adquately would demand a lengthy paper, which I will not attempt to compose here. I will, though, address the first point that you make in your January 3, article. Taleb criticizes the poor statistics used by intelligence researchers... I have assumed he means that more than half of intelligence research findings are wrong, and for malicious reasons. If this is his point, he is factually wrong. Your assumption is surely incorrect. Taleb neither said nor implied that more than half of intelligence research findings are wrong, for malicious reasons. Rather, he was presumably drawing an inference about the invalidity of most intelligence research findings from the well known "replication crisis in psychology" and other fields of research, and the well known fact that across the board, the majority of research papers are so poorly designed and analysed that most research claims must be false. So no, Taleb is not accusing you or those who labor in the IQ field of malicious fraud. You attempt to bury Taleb beneath a mountain of technical details and journal references that few here will ever read, but you do not confront Taleb's key point, which is that, yes, IQ tests measure something, and yes whatever they measures correlates in some degree with behavior, success, income, whatever, but so what? The key questions Taleb raises to which you offer no answer are: does IQ usefully quantify intelligence as that term is generally understood and as it is defined by the dictionary? and, more fundamentally, is it even theoretically possible to quantify intelligence, as that term is generally understood, by a single number? Taleb answers both questions in the negative. I agree. Furthermore, I believe that if you stopped calling whatever it is that you measure with you tests intelligence, then no one would question your work. Indeed, they might pay it no attention at all, which does raise a question of whether some psychologists, by mislabeling their product, are deliberately selling a bill of goods.

In the event that that draws a crushing rebuttal, I promise to post it here.

So far, all I've had is reference to a fact-free rebuttal by Stephen Pinker:

Irony: Replicability crisis in psych DOESN'T apply to IQ.S. Pinker

Great to be a famous author innit. No need to argue a point. Just assert an opinion and the world will defer — LOL

Except:

Most Reported Genetic Associations with General Intelligence Are Probably False Positives. Psychol Sci. 2012 Nov 1; 23(11): 1314–1323.

Or if you prefer a more mainstream source: The Telegraph's Science Correspondent reports:

IQ tests 'do not reflect intelligence' 

Or something even more downmarket: Daily Mail:

IQ tests are 'meaningless and too simplistic' claim researchers 

And I like this from the Psychologist:

What intelligence tests miss 

It is a profound historical irony of the behavioural sciences that the Nobel Prize was awarded for studies of cognitive characteristics (rational thinking skills) that are entirely missing from the most well-known mental assessment device in the behavioral sciences – the intelligence test. Intelligence tests measure important things, but not these – they do not assess the extent of rational thought. This might not be such an omission if it were the case that intelligence was an exceptionally strong predictor of rational thinking. However, research has found that it is a moderate predictor at best and that some rational thinking skills can be quite dissociated from intelligence.

Perhaps others will join in the amusing quest for quotes sending up S. Pinker.

Related:
CanSpeccy: Posts From the Past: About Intelligence (12)