The Challenges Behind Accurate Opinion Polls

Transcript

RENEE MONTAGNE, HOST:

And now to another big reversal, this one in the world of American politics. The shocking defeat of House Majority Leader Eric Cantor came after his internal polls showed him winning by a safe margin. Instead, he lost by double digits. NPR's national political correspondent Mara Liasson reports on the challenges of figuring out how people are going to vote.

MARA LIASSON, BYLINE: America is having a big data moment. Not only is everything in our lives subject to measurement analysis, but public opinion has never been so closely monitored by so many.

UNIDENTIFIED REPORTER 1: Well, brand-new Fox News poll shows that Americans trust...

UNIDENTIFIED REPORTER 2: Brand-new numbers releasing right now...

UNIDENTIFIED REPORTER 3: Three out of 5 Americans are very or moderately worried about having enough money for retirement.

UNIDENTIFIED REPORTER 4: Twenty-seven percent of those surveyed have an unfavorable opinion of him.

UNIDENTIFIED REPORTER 5: And a CBS news poll out this morning reveals most Americans do not like...

LIASSON: Many horserace polls, the ones that try to determine the outcome of an election, are accurate, but there's always an exception that proves the rule - like this week in Virginia, where Eric Cantor's pollster thought he would win by 34 points. Instead, he lost by 11. Getting a representative sample in an electorate that small - only 6,500 people voted in Cantor's primary - is really difficult. But even in a high turnout general election, it's tricky to get a sample of voters that mirrors the actual electorate. Neil Newhouse famously struggled with this when he forecasted a Romney win in 2012. Newhouse had created a turnout model of the electorate that included fewer young and minority voters than ended up turning out.

NEIL NEWHOUSE: We all recognize the growing challenges of pulling in a demographically changing country and especially one where white voters are proportionally less of the population as each year progresses.

LIASSON: J. Ann Selzer is a nonpartisan pollster who says the challenges the campaign grassroots machines are changing the electorate faster than pollsters can model.

J ANN SELZER: Pollsters that had likely voter models based on past elections weren't recognizing that the present and the future of getting people out to vote has changed. So they look to past elections, and they said well, younger people vote less often, minorities vote less often, so we're going to count them as less likely to vote.

LIASSON: Polling has other challenges. And to be a savvy consumer of polls, like any other product, it's important to read the fine print. Every poll includes information about how the poll was taken. How many people were surveyed? Was it a random sample? Did the questions seem fair? And, says Pew Research Center pollster Scott Keeter, was the poll conducted by human beings or a computer?

SCOTT KEETER: Robot polls have been very popular in the last several election cycles in particular because they're cheap. They're a way of getting a lot of interviews in a short space of time. But they have a couple of problems that they've had to confront.

LIASSON: One problem with automated polls where respondents are asked to answer questions by punching buttons on their phone is that federal law allows robocalls on landline telephones only.

KEETER: We really don't know who's answering them. There's, by definition, no interviewer present. So children could be responding to them if they wanted to. People that are not in the target population, such as registered voters, could be answering them.

LIASSON: And then there are Internet polls - even more unreliable because they allow people to opt in voluntarily from their favorite websites. But Pew and other polls continue to experiment with new ways to find people because the old ways just aren't working. Democratic pollster Peter Hart started his career by going door-to-door to survey votes, but now Hart says fewer and fewer people want to talk to pollsters.

PETER HART: That is the big challenge facing this industry - is there's a declining response rate. And the second challenge is we now have at least 35 percent of our population that has cell phones only. So that's the only way that you can reach people, and that's much more expensive.

LIASSON: More expensive because it takes a lot more effort to find people with cell phones. There's no cell phone phonebook, and there's no way of knowing where people live - not like in the old days when you move to a new city and got a new phone number.

HART: People get a cell phone and they move, and therefore, the person who was in Cleveland may now be in Boston but using a Cleveland area code.

LIASSON: Pollsters also suggest that consumers reading polls look at a lot of them and look at the same ones over and over again so they can see trends. That's because, says Neil Newhouse, public opinion is a moving target.

NEWHOUSE: Public opinion is fluid, and polls try to pick that up. They're a snapshot in time, but it's where voters stand at a certain point in time, and that can change.

LIASSON: And that's why designing a survey that captures those changes is truly a combination of art and science. Mara Liasson, NPR News, Washington.

MONTAGNE: You're listening to MORNING EDITION from NPR News. Transcript provided by NPR, Copyright NPR.