More on Polls

There's a good article over at HuffPo which discusses some of the internal problems with polls and the difficulty of sorting out sampling error and response bias. One key graf:

Gallup's tracking poll, USA Today and CBS News all show the Republicans with some kind of lead over the Democratic ticket. But, interestingly, all three polls were also conducted using a higher sampling of Republican voters than in July, thus raising a question of methodology.

In a year in which Democrats have a lead of 11 million registered-voters over Republicans, and have been adding to that advantage through a robust field operation, are pollsters over-sampling Republicans?

Despite a raft of advantages in the electorate for Democrats, in September's first Gallup Tracking poll, an equal number of Republicans and Democrats were surveyed (including "leaners") from Sept. 3-5, compared to a 10-point Democratic identification advantage two weeks ago. That partisan makeup of the polling pool resulted in a 5-point lead for McCain in Sept. 5 tracking poll. Meanwhile, the new CBS poll features a 6-point swing in partisan composition toward Republicans, which plays some role in the poll's two-point lead for McCain. Finally, the latest USA Today poll, which claims a four-point edge for McCain, was arrived at after a 10-point swing in partisan makeup toward GOP respondents.

The point here is not that the polls are wrong or inherently unreliable, but that there is a lot of statistical noise that gets rolled into the final 48-46 number. That this is an effect of the convention is not arguable, but the cause is not clear -- are republican leaners more energized and more likely to identify as republican? Are they just more likely to answer the phone? I'll leave you to read the article, but the take home point is that while the numbers at this point are certainly better for McCain, there's reason to believe that it may be temporary, and that it may not be accurate at all.