Illustration pertaining to polls

[JohnStOnge]

I've been shaking my head while watching TV as talking heads make a big deal about this candidate or that being ahead and how things have "changed" since the last poll. I decided to do a simulation to illustrate how foolhardy that is. I set up a scenario in that, in the true population, 45 percent plan to vote for Candidate A (A), 45 percent plan to vote for Candidate B (B), and 10 percent are undecided U). Then I simulated 10 random samples of 1000 each from the population. Here is what I got :

A 43.5%, B 45.5%, U 11%
A 46.6%, B 44.8%, U 8.6%
A 43.3%, B 45.9%, U 10.8%
A 45.6%, B 43.3%, U 11.1%
A 45.6%, B 45.7%, U 8.7%
A 44.7%, B 45.8%, U 9.5%
A 44.9%, B 43.3%, U 11.8%
A 42.7%, B 46.7%, U 10.6%
A 43.8%, B 46.0%, U 10.2%
A 46.1%, B 42.7%, U 11.2%

Each set of results is a legitimate estimate of the same real situation (45, 45, 10). Yet they vary from having Candidate A leading by 3.4 percentage points (46.1 to 42.7) to having Candiate B leading by 4 percentage points (46.7 to 42.7). That is a 7.4 percentage point swing between two sets of results when there is absolutely no difference in the true situation.

Also, what I did represents an idealized situation in which pollsters would be able to get pure, theoretically sound probability samples. That is not possible so the true error associated with each result is probably somewhat greater than that simulated.

Anyway, just a reminder that changes in polls don't necessarily mean changes in reality. At some point the change becomes large enough so that you can be confident that the reality has changed. But media people act as though polls indicate change on many occasions when it's not really possible to say that.

If the set of results above was a set of weekly poll results over 10 weeks, for example, media people might say Candidate A started off behind, went ahead for one week then back behind for the next week...so on and so forth. But in reality the situation never changed at all.

[93henfan]