You all know about the issues of weighting samples to achieve representativeness. In polling this is an art. But even if you get to representativeness, depending on the average sample sizes the polls themselves will exhibit a distribution of outcomes about a mean. Therefore with a large enough sample space of polls you can find one at a tail of the distribution of outcomes. Today with the proliferation of polling this is getting to be more and more of a problem. Look at this musing by Joshua Micah Marshall: A Feingold Comeback? He notes: “A new independent poll has Feingold down by only 2 points.” A thickly polled state will come back with a range of results. Even non-internal polls (which are often curated for maximum effect on the press, and so shouldn’t be trusted) will exhibit a normal distribution of results, so if you want to hinge an argument on one poll, it is fast becoming a trivial task to find that poll to satisfy your needs. In 2008 the less intelligent set of conservative bloggers expressed ideologically motivated skepticism of polls (grounded in the fact that they were too stupid to know any better). What excuse does the self-proclaimed “reality based” party of pointy-headed intellectuals have in 2010?
Whatever happens, 2012 will be even worse. The number of polls is going to up, and verbally oriented bloggers and reporters will cherry-pick outliers to produce whatever narrative they wish to roll out.
This exchange will, no doubt, change some polls.
http://michellemalkin.com/2010/10/19/chris-coons-cant-name-the-five-freedoms-in-the-first-amendment/
I like the Onion findings, which predict the GOP will pick up something like 8,000 House seats and about 1,800 governorships.
In 2008, fivethirtyeight.com was routinely denounced on freerepublic as a den of libsymp ‘Rat spies out to undermine conservative morale with their evil poll massage.
This year, with fivethirtyeight predicting big Republican gains, there hasn’t been a peep.
That’s only human. I remember on liberal boards in 2004 how all the polls showing Bush ahead were discounted for one reason or another. For some mysterious reason by 2008 the same polls became much more reliable.
Rasmussen is typically vilified by the left, though his numbers are usually pretty good. He’s even conservative (small “c”) about Republican prospects this year. He forecasts something like 50 House seats and 6-8 Senate seats, mean House GOP control, but only a large, filibuster-capable minority in the Senate.
“In 2008, fivethirtyeight.com was routinely denounced on freerepublic as a den of libsymp ‘Rat spies out to undermine conservative morale with their evil poll massage.”
The turn around in the attitudes of 538’s commentators has been quite amusing as Nate Silver is now accused of ignoring all sorts of special factors that mean the Dems will do much better than predicted.