Result percentiles: how do you stack up?

November 8, 2018

New to “Test Your Culture?”  📊 Take the quiz first!

Test Your Culture reports how many cultural items a test-taker knows out of the top 10,000 — for example, 3,000 or 4,000.

But the number by itself doesn’t mean very much — what matters is how it compares with others. And the standard way to measure that is using a percentile. A percentile tells you what percentage of a population has reached a given score, and therefore how any particular score “stacks up.”

We already report a percentile along with each test-taker’s result, but some of you have been asking to see all percentiles. So here you go — current* percentiles for Test Your Culture results, for Americans aged 18+, which is also our first reported research results.

* Percentile data is live, so while this blog post is dated November 2018, data is current as of December 17, 2018. Percentiles normalized to U.S. population distribution by age, gender, and educational level according to 2017 U.S. Census.

Because the above chart is live, it will grow more and more accurate (and therefore smoother) over time, but here’s what we can say about the data so far (as of November 2018, and rounding to the nearest 500 for simplicity):

  • The average American aged 18+ knows 4,000 of the top 10,000 cultural items. (Technically this is actually the “median,” or 50th percentile.)
  • A majority of Americans aged 18+ know between 2,500 and 5,000 of the top 10,000 cultural items. (This is the range between the 25th and 75th percentile, or half the population — the threshold for a bare majority.)
  • Nearly all Americans aged 18+ know between 1,000 and 7,000 of the top 10,000 cultural items. (This is the range between the 2nd and 98th percentiles, or two standard deviations.)

These percentiles are computed from all test-takers who filled out the optional survey at the end, who self-identified as American, and provided their age (18+), gender, and educational level. This is because we normalize our percentiles according to these three categories, using U.S. Census data. This means we weight each segment of test-takers not according to how many visited Test Your Culture, but how many there are in the U.S. — so a score comparison isn’t against other test-takers (who may be more likely to score higher than the American population overall), but against our estimate of the whole American population.

So back to the results — what’s fascinating here is the extent to which there’s such a significant degree of variation. And a natural question arises: how much variation is it? Does it vary to the same extent some other well-known measure of knowledge does?

The most straightforward comparison is to “evidence-based reading and writing” SAT scores (known as the “verbal” section until 2005), looking at the distance of a standard deviation — specifically raw SAT scores (the actual number of correctly answered questions before any normalization). Running the numbers, normalized SAT scores report a nationally representative median of 510, with a standard deviation below and above of 400 and 620 respectively. Converting these to raw scores produces a median of 25.5 correctly answered questions, with a standard deviation below and above of 16 and 36 respectively, or an absolute −37% to +40% around the median. In comparison, Test Your Culture shows a standard deviation of −42% to +39% around the median — nearly identical.

This in itself doesn’t mean a ton — SAT questions are written and chosen by human beings, and aren’t necessarily a perfect or linear measure of knowledge. (Additionally, SAT percentiles are based solely on 11th and 12th graders, while our own percentiles span all ages starting from 18.) But it does mean that cultural knowledge appears to vary approximately to the same degree as reading and writing ability — at least according to the best-recognized national exam in the U.S.

Check out more blog posts or 📊 take the quiz!