One Political Scientist On Judging The Validity Of Polls

Don’t trust the polls. Trust the average. That’s the general advice from most pollsters or politicos when reading presidential predictions.

But even so, not all polls are created equal.

Results differ based on who is being selected for a poll, whether it is a national or state poll, the number of candidates on the ballot and how close the poll is to the election.

Matthew Dickinson of Middlebury College speaks with Here & Now‘s Jeremy Hobson about what makes one poll better than the other and why it matters when it comes to predicting who will win an election.

Interview Highlights: Matthew Dickinson

On factors affecting polling results

“One of the things that I think a lot of listeners underestimate is that, even in a perfectly well-done poll, you’re going to have tremendous statistical variations from poll to poll. And that’s just simply a fact associated with [the fact that] we’re taking a sample from a population. So if you dip back into that population many different times, you’re going to come up with slightly different measures. So a lot of what you’re seeing here is just random fluctuations due to the process of sampling.

Now the other thing that’s going to affect it are man-made errors, and that has to do with how the polls are conducted, whether you offer third-party candidates, what your sample population is — is it registered voters or likely voters — and that explains a lot of the variations you’re seeing in these state polls as well.”

On likely-voter surveys vs. registered-voter surveys

“Typically in the end, likely-voter surveys are more accurate predictions of the final vote. But problem is this requires you to make estimates about who a likely voter is, and different polling organizations have different ways of doing this, typically through asking a series of questions — ‘Did you vote in 2012? Do you know who is running? Do you know where your polling place is?’ And based on that, you make an estimate of who a likely voter is.

Now when you move from registered voters from likely voters, you’re more likely to shift in recent elections and your results in a Republican direction. And that’s because likely voters tend to be a little wealthier, a little better educated than registered voters…. So the key here, compare apples to apples — if you’re looking at trends over time, don’t compare registered polls taking time Z to a one taking a week later that’s only likely voters.”

On a ‘gold standard’ poll

“Something that I’d recommend that all listeners do — don’t rely on any single poll. Any poll is a snapshot and is subject to both the problems with man-made wording and sampling errors. The way you want to get around that, look at polls in the aggregate, so do something like the RealClearPolitics polling average or the Huffington Post aggregate. That way the tendency is for polls in errors to cancel out and you get a more reliable assessment of the polling landscape.

Having said that, the gold standard ones are — I wouldn’t pick out anyone in particular, because in any given election cycle, depending on the assumptions they’re making about likely voter models, they could just have a bad election cycle, like Gallop, which is a very reputable polling thing. But the major media surveys are good, and major academic surveys generally are good. The ones you wanna worry about are the sort of fly-by-night, ones that you haven’t heard of that somebody comes up with automated poll that has a result that doesn’t fit with your instincts about what’s happening in this race — that is one candidate way ahead or way behind.”

On including third party candidates in the poll

“There’s no a prior reason to say one is better than the other — it all depends on whether you think these third party candidates are viable players in the election. We know in the years past at this time in the election cycle, third party candidates tend to poll better than they actually do in the final results, and that’s the reason why you find a lot of major pollsters not including these third party candidates because they think their support is going to dissipate.

On the other hand, there’s reason to think in this election cycle, third party candidates are going to play a little bit influential role. First, both of our major party candidates have incredibly high unfavorable. So the question now comes down, do you think these third party candidates are credible to the sense that they’re going to detract from support for major party candidates… But here’s the key, if they’re not in there, you still have to include a ‘don’t know’ option. You have to give voters respondents in alternative to saying they’re going to vote for one of the two major candidates. If you do that, then these polls are roughly comparable.”

Guest

Matthew Dickinson, professor of political science at Middlebury College. He tweets @MattDickinson44.

Copyright 2016 NPR. To see more, visit NPR.