Media literacy
How can a poll of 1,000 people speak for a whole country?
By the Agora desk · Published · How we count player splits
Because of random sampling: if everyone has a fair chance of being picked, about 1,000 people will usually land within roughly 3 percentage points of what the whole country thinks, whether the country has 10 million people or 300 million. It's like tasting soup: one spoonful from a well-stirred pot tells you how the whole pot tastes, however big it is. The hard part isn't the maths; it's getting a sample that's genuinely random and representative.
In short
- A random sample of about 1,000 gives a margin of error of about ±3 points.
- The size of the population barely matters; how the sample is chosen matters a lot.
- The margin on the gap between two candidates is roughly double the headline figure.
- Small groups within a poll (young voters, one region) have much bigger margins.
- Margin of error only covers chance; bad questions and self-selected samples are separate problems.
Why a small random sample works
Think about flipping a fair coin 1,000 times. You won't get exactly 500 heads, but about 95 times in 100 you'll get somewhere between roughly 470 and 530. Chance evens out quickly. A poll works the same way: if 50% of a country holds a view and you ask 1,000 people chosen at random, your answer will usually be close to 50%.
The key word is random. Pew Research Center, for example, recruits its main US survey panel by picking random addresses from the US Postal Service's list and inviting one adult from each household. Everyone has a known chance of being included, so the sample ends up with a realistic mix of ages, incomes, regions and views.
That's also why the size of the country barely matters. A spoonful tells you about a pot of soup whether the pot holds one litre or a hundred, as long as it's stirred. Polling 1,000 people in the UK and 1,000 in the US gives about the same precision.
Margin of error, with real numbers
The margin of error says how far a result could plausibly be from the true figure just because of who happened to be picked. Pollsters usually quote it at a 95% confidence level. Pew's example: if a candidate is on 48% with a margin of ±3 points, their true support is probably between 45% and 51%. If you ran the same poll 100 times, about 95 of the results would fall within 3 points of the true value.
For a simple random sample, and a result near 50%, the margins look like this. Notice the diminishing returns: halving the margin takes four times as many people, which is why so many polls stop at around 1,000.
- 100 people: about ±10 points.
- 250 people: about ±6 points.
- 1,000 people: about ±3 points.
- 4,000 people: about ±1.5 points.
- 10,000 people: about ±1 point.
Reading a lead: when is it really a lead?
The margin applies to each candidate separately, so the margin on the gap between them is roughly double. Pew's worked example: a 5-point lead with a ±3 margin has a margin of about ±6 on the lead itself, so the true lead could be anything from about 1 point behind to 11 points ahead. That's what reporters mean by a "statistical tie". Changes between two polls are uncertain in the same way, so a single poll that jumps is usually less newsworthy than it looks. An average of several polls is more reliable than any one of them.
Subgroups shrink the sample. In a poll of 1,000, perhaps only 150 are under 30, so the figure for young people might be ±8 points or more. Be wary of headlines built on one small group.
When different signals point different ways, as in Agora's story on polls and markets clashing over the Senate majority, remember that each poll is an estimate with a range, not a single exact number.
Weighting: fixing a sample that isn't quite right
Real samples are never perfectly random. Some groups are harder to reach or less likely to answer. So pollsters weight their results: they compare the sample with official population figures, such as census data on age, sex, education and region, and count under-represented people a little more and over-represented people a little less. If a sample has half as many young adults as the population does, each young respondent might count roughly double.
Weighting reduces bias but adds some uncertainty of its own, which good pollsters build into the margin they report. It can only fix what the pollster knows to measure: if the people who refuse to answer differ in ways the weights don't capture, the error remains.
What the margin of error doesn't cover
The margin of error only measures luck of the draw. Other problems can be bigger:
- Who's missing: people without internet access, without a fixed address, or who never answer surveys.
- Question wording and order: loaded wording, questions that ask two things at once, and people giving the answer they think sounds best.
- Who counts as a voter: election polls have to guess who will actually turn out, and those "likely voter" models can be wrong.
- Opt-in online polls: when anyone can sign up, often for a reward, some respondents answer carelessly. In a 2024 analysis Pew found that 20% of under-30s in one opt-in poll agreed "the Holocaust is a myth", against 3% in its own randomly recruited panel; in an earlier test, 12% of under-30 opt-in respondents claimed to be licensed to operate a nuclear submarine.
- Size isn't a cure: in 1936 the Literary Digest magazine got more than two million replies, largely from its own readers and lists of phone and car owners, and forecast that Alf Landon would beat Franklin Roosevelt. Roosevelt won 46 of the 48 states.
A checklist for any poll you see
Agora's own player splits are not a poll: players choose to take part, so they're a self-selected group rather than a random sample. Here's how Agora counts its player splits. For a real poll, check:
- Who ran it and who paid for it? Reputable pollsters publish their full tables and methods; in the UK, members of the British Polling Council must do so.
- How were people chosen: random recruitment, or anyone who clicked a link?
- Who was asked: all adults, registered voters or likely voters? When, and how many?
- What was the exact question, and what came before it?
- Is the headline about the whole sample or a small subgroup?
- Is this one poll, or does it match the average of others?
Common questions
Why have I never been polled?
The odds are small. Pew estimates the chance of any given US adult being in a typical Pew survey at about 1 in 26,000. Millions of people are never asked, which is exactly how a random sample is supposed to work.
Are online polls accurate?
It depends how people were recruited. Online panels built from random samples, such as address-based recruitment, can be very good. Opt-in polls, where anyone can join, are more prone to errors, especially for small groups. Polls on social media, where anyone can vote, aren't polls in the statistical sense at all.
If polls are so clever, why do they sometimes get elections wrong?
Because sampling error is only one source of error. Turnout guesses, people who won't answer, late changes of mind and question wording can all push results further off than the stated margin. Polls are snapshots of the moment they were taken, not predictions.
Would polling more people always help?
Only up to a point. Going from 1,000 to 4,000 people halves the margin of error, but a big sample collected the wrong way is still wrong, as the 1936 Literary Digest poll showed.
Sources and further reading
- Pew Research Center: Understanding the margin of error in election polls
- Pew Research Center: Frequently asked questions about its surveys
- Pew Research Center: Online opt-in polls can produce misleading results (2024)
- Full Fact: How to spot misleading poll figures
- Random (University of Alabama in Huntsville): The 1936 Literary Digest poll
- British Polling Council
Read next
- How do I read a news story critically?A step-by-step checklist: spot news vs opinion, read past the headline, check sources and numbers, notice what's missing and check other coverage.
- How do I spot misinformation and fake news?Stop, check the source, find better coverage, trace it to the original. A step-by-step method fact-checkers use, with red flags and a worked example.
- How does the Electoral College work?Why Americans elect a president through 538 electors, how 270 wins, why the popular vote loser can win, and the arguments for keeping or scrapping it.
Put it into practice: latest stories
Written by the Agora desk from the sources above. See our editorial standards. More Learn articles.
