False Precision

That’s the phrase of the day, courtesy Zombie (h/t Glenn).

That only 9% of those polled are responding is just incredibly devastating — that means the polls are essentially worthless, with that kind of massive, systemic sampling problem they can’t really be said to be accurate to within better than 10% or so.

Think for a moment about what that 9% means — if fewer than 1 out of 10 of those polled are responding, the pollsters must do one of two things: assume the 9% who answer are still a representative sample of the population, or attempt to adjust the sample of responders to the population. I have seen no indication the latter is happening. And note that this is a pretty new problem — response rates were four times higher only 15 years ago; it’s unlikely pollsters would even know how to adjust.

And while it’s possible selection ratio bias will be a small problem in the 2012 election, as one study of 81 surveys suggests, that’s really impossible to know in advance for a given election — at the very least, the margin of error must be considered somewhat larger than in the past.

All of which is too bad for Romney, because right now the polls say he’s winning (for a simple, accurate measure, disregard topline numbers and DRI split — whoever is winning independents is winning the election).

Upshot? No one will know who won the election until the votes are counted. Ignore anyone who says otherwise.

UPDATE: As usual, Iowahawk says it better in less than 140 characters:

There are 11 women in the bar. Conclude how drunk they all are, based on the only one who will talk to you. #polls #9percentresponserate