The neurotic obsession with polls has hit a new height in the Toronto mayoral election. It’s as if the polls keep watch so that Ford can’t sneak up on us again.
It’s terrible that the polls dominate the discussion. Policies, experience, and character are forgotten when talk turns to trends and predictions.
Polls also reduce voter engagement. When one candidate’s appears destined to win, what’s the point in voting, or volunteering, or fund raising for any of the others?
Perhaps the most pernicious effect of polls in this election is how they appear to legitimize strategic voting. If people are going to vote strategically, rather than voting for who they think is best, they are allowing circumstance to dictate their choices. As circumstances change over the course of a campaign, their votes must change too. Early on we saw an “anyone-but-Ford” trend that favoured Tory over Chow. If Ford falls hopelessly behind, what will strategic voters do? If, in their hearts, they actually prefer Chow to Tory, but the polls indicate a Tory landslide, will they decline to vote at all? All bets are off when strategic voting takes hold.
That’s the fallacy of strategic voting. It assumes the public can learn something from the pollsters, and that the pollsters can learn something from the public. But if strategic voters mislead the pollsters about their intentions, then the pollsters inadvertently mislead strategic voters about where to place their votes. They are dependent on one another, but neither is reliable.
You’d expect the media to exercise some caution in their interpretation of poll results because of this. But elections are complicated. The issues are difficult to explain and the outcome is uncertain. To fill air time and line inches cheaply, you can’t beat the latest poll.
Polls allow journalists to delude themselves about telling stories before they happen. Although polls are backward looking by nature, there is an assumption that yesterday’s poll will tell you something about tomorrow’s. However it takes good data and consistent analysis, tracked over time, to establish meaningful trend.
Shockingly, to me at least, nothing is good or consistent in the way polls in this election are being reported. All the public polls are compiled and presented in a single chart by the Toronto Star on their website. Look closely and you’ll see problems that should prevent the Star and other news organizations from making headlines from these numbers.
They know that every polling company uses different sampling and weighting methods, so different results, on different days, by different pollsters, does not establish a trend. Yet after Forum Research laid down a baseline of results from January to June, 2014, the Star spiced things up by throwing in a half dozen polls by other companies in July, August, and September.
Those other companies provided dramatically different numbers, spawning stories about an insurmountable lead for Tory. Of course that was all strategic voters needed to hear in the “anyone-but-Ford” camp. The flocked to Tory.
Not only was this misleading, it was unforgivably sloppy. Check the poll tracker site for yourself and total the breakdowns for September 12th and 15th. They add up to 107% and 106% respectively.
On September 12th, it the Main Street poll made headlines because Tory appeared to be surging ahead at 45% while Chow languished at 27%. On September 15th, radio stations were leading with the story that one out of every two voters was behind Tory, based on a Nanos poll putting his support at 49%.
Obviously neither of these polls should have been included as part of the trend lines depicted on the Star’s site. First, total responses cannot exceed 100% by so much unless massive voting fraud is being predicted. Second, these polls were outliers in relation to the dozens of data points already established by Forum Research over the previos nine months. Whether because of sampling or weighting or both, they produced results significantly outside the range of values found just days before and after.
In the chart of Mixed Poll results provided below, there are two striking anomalies that any critical journalist should have noticed.
- Tory support tends to peak on days when Nanos and Main Street are polling.
- Only in Nanos and Main Street polls does the undecided vote jump into double digits.
I’m not saying that the Forum results were more accurate than Nanos, or vice versa. The point is that they shouldn’t both be represented graphically, and reported uncritically, as part of the same trend.
Yet that’s what happened. The poll-drunk media shamelessly piled on, anointing Tory as the probable winner and relegating Chow to third place.
Real trends can be stubborn, however. After this sloppy statistical mashup in mid-September, Tory support declined 11% from its peak at 49% (according to Nanos), and Chow support started trending upwards to 25% from her low of 19%. In other words, the gap between them had closed a total of 17% in one week, September 15th – 22nd.
Did this make the news? No, instead the reported poll results centred on how Tory was maintaining a lead over Ford, and how Chow was still languishing in third place. The emerging trend line was crushed by the story line established by previous polls.
To be clear, I don’t have a dog in this fight. My preferred candidate was eliminated by the polls already. I have little interest in polling or strategic voting, for that matter. I just want the media to be accountable for the effects of misreported polls on an impressionable electorate, especially during an election where the spectre of a Ford dynasty scares everybody. The way the polls have been used to build Tory support, by the Star especially, warrants examination by its editorial board at the very least.
To illustrate all of this, I have provided two charts based on the results published in the Star. Both cover the same nine months of polling, one that mixes all the results, and the other with results by only one company, using a consistent sampling and weighting method. They tell very different stories about the candidates and their supporters, including the undecided voters. Dotted trend lines have been added to help illustrate how the statistical stories deviate from one another.
The mixed poll chart is an implausible mess of anomalies, with peaks and valleys resulting, not from a change in voter intention, but from a change of pollster. This is what the media has been reporting on.
Because Nanos and Main Street found more Tory supporters than Forum could, the final month of the campaign looks hopeless for Chow. But when those half dozen data points are removed, the Forum numbers show the gap was closing fast within one week of Tories reported peak in popularity. As the dotted trend lines illustrate; the half dozen anomalous data points alter the trend mathematically, just as they've altered perceptions of the candidates.
It’s too late to do anything about this now. A fair election has been made less fair by misappropriated poll results, used by the media to make headlines. It’s also unfair to the pollsters, who each bring a different kind of rigour to their work, but are made to look incompetent when their results are blended.
Obviously Chow’s chances have been diminished by this sloppiness, but voters have been hurt too. You can’t exercise a choice you don’t believe you have. Persuaded that a vote for Chow might mean a return of Ford, or that a vote for Chow would be wasted in a Tory landslide, many ordinary people may be holding their noses and voting for Tory on election day, or declining to vote at all.
My reading of these charts: When media chase the polls and induce strategic voting, I think we all lose.