University of Cambridge Cognitive Ability Test

On scientific front, it's very useful. Its use outside of academia is has however been very problematic.

There are studies that empirically measure drop change cognitive ability from lead poising, oxygen deprivation, sleep deprecation, post-burnout, environment distraction, noise pollution, temperature, aging, drug and alcohol use during puberty, smoking, school teaching style, etc. etc.

Notable is that these are either population metrics or compare each individual with themselves. This is what IQ and other similar tests were meant for. Comparing one person with another is nonsensical and a flawed use of these metrics.

This is where where IQ has fallen and become a rather bad metric. People are familiar with the problems scewing results. IQ test performance and education level is highly correlated, which is supposed to be compensated for in the final score. But poor education quality in certain regions make the statistics easily used to argue quite unsavory ideas.

You can create a proxy for IQ by simply asking a group of people to invent their own questions, summing them together, randomly sampling from those questions, and then rank ordering the outcome. High IQ people will score in the top percentile for any set of questions. That's the whole point.

"Notable is that these are either population metrics or compare each individual with themselves."

Yes, for the purposes of that research. Why would a comparison between two people be a flawed use case? Do you just mean the colloquial use and understanding of IQ is flawed?

Over a population, outside factors affecting the score is smoothed out to create a normal distribution. Over one individual, most factors remain the same.

Over two different people, so many factors affect the score that making the claim "one person is more intelligent than the other" is statistically unsound without a massive score difference. This is even ignoring that a full IQ test involves FAR more than the usual online logic puzzles people tend to know, yet still have these flaws.

so yes; > colloquial use and understanding of IQ is flawed.

Do you have a citation for this? It sounds pretty unlikely to me.

I'm pretty sure if I asked a group of people to invent their own questions I'd get a load of general knowledge questions about music, sports, and popular culture.

> Comparing one person with another is nonsensical and a flawed use of these metrics.

Isn't this what IQ tests literally do, given that they transform raw scores to a normal distribution for comparability?

Let's say you had a caliper that added a random number between zero and one inch to every measurement. If you measured a trillion small peas and a trillion big peas, you would be able to conclude that one set was smaller than the other. If you compared two peas, it'd be a 50-50 guess driven by that random number.

You seem to be suggesting that IQ tests simply don't work, not that their point isn't to compare people.

I'd even suggest the point of these tests is more than simply comparing people. IQ tests, at least how they are commonly used, are used to determine the one's worth as a human-being. I am not saying I personally agree with those views, but merely that such views are reflected in many modern societies and how people treat one another.

I think that would be a fair summary of my opinion of them, especially when related to the somewhat problematic view of "IQ" in popular culture.

As far as I can tell, their only real purpose in the UK is to try to convince "intelligent" people to give money to MENSA.

This is a talking point from an old university class lecture interpreting factor analytic data on personality and IQ [1].

In practice, intelligence tests don’t depend on the specific questions asked. If you let a group of people generate their own items, pool them together, sample randomly, and then rank scores, the same individuals would tend to rise to the top. The high IQ people would cluster toward the top with a correlation of 0.9. This is because people with higher general cognitive ability perform better across virtually any cognitive task, a phenomenon first documented by Spearman (1904) and repeatedly confirmed in psychometrics research (e.g., Jensen 1998; Deary 2012).

[1] https://youtu.be/D7Kn5p7TP_Y?t=47m50s

IQ tests have a 95% confidence interval of about 10, so it's a bit more accurate than you're implying, but still a range of a good fraction of a standard deviation. (basically, the measurement is about 5 times sharper than the variation in the population, so it's for sure a blurry view but they do give you a meaningful idea of where an individual sits in that distribution)

My test only provided at a 90% CI. Not like it is a substantial difference, but I thought I’d throw that out there.

Then again, the same psychologist told me that the discrepancy between my scores was so large that a true FSIQ cannot be used. In a sense, he told me that based on that test, I didn't actually have an IQ that could be accurate measured based on that given test. Apparently, it’s not normal to have almost two SD between some scores...

Being a smart idiot isn't easy work, but someone has to do it.