How polls work and why they’re often so wrong

Pollsters were applauded for predicting the Alberta election, and derided for being so off the mark in the U.K.

Thestar.com
May 8, 2015
By Robin Levinson King

If they paid any attention to the polls, gamblers who put their money on elections in Alberta and the U.K. would have broke even.

On one hand, the Alberta election became a victory not only for the NDP, who topped a 44-year Tory dynasty, but for the pollsters who predicted the shocking orange crush.

But just days later and an ocean away in the U.K., and surveyors were almost forced to eat their hats when what was supposed to be a neck-and-neck race between the Conservatives and Labour turned into a near-majority win for incumbent Prime Minister David Cameron.

The Star spoke to Derek Leebosh, a pollster at Environics, to better understand how number crunchers call an election, and why they can sometimes be way off the mark.

How do polls work before an election?

The principle is that you do a random selection of voters and you ask their opinions on different issues and how they’re going to vote. If you talk to enough people and they’re actually a random sample of the population, then what 1,000 people tell you can predict what the other 10 million people think.

How big a pool of people do you need to reach?

It actually doesn’t have anything to do with the size of the population. You can do a poll on the Toronto mayor and poll 1,000 people and it’s just as valid as doing a poll on the American presidency when you poll a 1,000 people.

What matters is just the number of interviews you do. It’s just as statistically accurate to predict for a population of one million as it is to predict for a population of 100 million - as long as people have been randomly selected.

You sometimes see polls of 500 people, sometimes 1,000. The margin of error obviously gets bigger the smaller the sample size is. You can do a poll of 500 people across Canada and it will give you an idea of what party support is across Canada. The problem is when you have a small sample then it gets dicey when it starts getting compared to Quebec versus the rest of Canada, or you start looking at different age groups or men against women.

The advantage of doing a large sample size is it means you can slice and dice the population a lot more.

When is the best time to do a poll, if you want to predict elections?

The closer you are to election day the more predictive it’s going to be.

If I do a poll 10 days before the election, lots of things can change. There are stories about how nobody thought American President Harry S. Truman was going to win in 1948, because the polls all showed him down by a huge margin. The trouble is, they all stopped polling six weeks before the election back in those days.

But polling doesn’t just exist just so you can find out whatever it is you’re going to find out a day before it happens - because obviously, the largest poll of all is election day.

It’s supposed to give us an idea, over the course of a campaign, of who is showing momentum, who isn’t, what way the wind is blowing, what are people’s views on the issues and why people are voting the way they do.

Sometimes we get too focused on just the horse race.

The volume of polls have never been so high, just the sheer number of polls we get over the course of an election campaign compared to what they would have been say twenty years ago. But often times I find it’s kind of a mile wide and an inch deep. It’s always just who’s ahead, and who’s behind.

It’s never why is this person ahead? There’s less and less analysis about what is driving people’s opinions.

Can polls influence the vote?

I don’t know that people necessarily jump on the bandwagon and vote for a winner. The NDP won this shock victory in the Alberta election, but they didn’t get any more votes than the final polls showed - if anything they got slightly less.

But one thing that polling does do, is if you start to move ahead, particularly if you were an underdog, it starts to send a signal to people that if I vote for this party it’s not a wasted vote anymore, because lots of other people are voting for them.

And it has a big impact on party moral, and fundraising starts to improve. And then of course, the media coverage starts to change, because if you seem to be more and more popular, the media tends to write more favourable stories about you. It can have an indirect impact in that way as well.

Alberta was one of those elections that vindicated a lot of pollsters. But Thursday’s U.K. election shocked everyone, because polls had showed the two parties neck and neck.

How can polls be both so right and so wrong?

In the vast majority of elections, the final polls are very close to what the final results were. I don’t know if there’s really an explanation for why the polls got it right. I think it’s more is there an explanation for why the polls got it wrong.

In Alberta, a lot of the final polls had the NDP at 43 or 44 per cent and the Conservatives at 24 per cent. But the popular vote ended up being 41 to 27. It didn’t matter because the NDP had such a huge lead that they still won by a huge margin anyway, so nobody cared.

But in the U.K, the problem is that the polls showed it was a dead heat. A flip of two per cent from one party to another can make the difference between a dead-even race or one party blowing the other out of the water in seats.

I’m sure the polling industry will have to do a whole inquiry into it.

In the last British election, in 2010, they were also off a bit. But people didn’t talk about it so much because the polls said Labour was going to lose and they lost. But what people don’t talk about is that the polls said the Conservatives were going to beat them by 10 points and they only beat them by five.

But people didn’t care as much because they got the right winner.

Do you trust polls the night before an election?

I know they’re not going to be perfect. But the final poll will certainly give you a very good idea.

If polls the night before the election in Alberta show the NDP have an 18 point lead, I know they’re going to win. They may not win by 18 points, but they’re definitely going to win because there’s a limit as to how far off the chart that could go.