I rarely agree with John Ibbitson. I don't think he's an idiot. Far from it; he's an intelligent commentator on public affairs. I just normally think he's wrong so this time I should note when I think he's right or at least close to the mark:
Smith learns hard lesson in Alberta: Intolerance doesn't win elections - The Globe and Mail:
Alberta politics are always fascinating for those of us who study public life in Canada. Its history of third parties, for instance, is something I mention in every intro Canadian Studies course I teach. What makes this last election really interesting is two things:
1) just how wrong the pollsters and pundits were
2) the implications of the PC victory
On both these points, Ibbitson has some words that are worth paying attention to, even if he's mocking pollsters and pundits.
Much of the public discussion in the 24 hours after the Alberta election involved pundits and pollsters asking themselves this question "how would we be so wrong?" To a one, these people predicated a Wildrose government and very, very likely a majority (mea culpa). The general conclusion of the pundits, however, and oddly, is this: we were not wrong. Our polls accurately reflected Alberta voting intentions. it is just that those intentions changed in the last few days of the election campaign.
The problem with this is that the polls were wrong because they did not capture the shifting sand of public life. Even if the polls were correct -- up to three days before the election (and, frankly, I seriously doubt this) -- the problem is that the polls presented a white/black image of the electorate. They did not present the ambiguities of the voters or the fact that voters in large numbers were still thinking about for whom they were going to vote.
Let me explain it this way: imagine that you ask ten people what they want for lunch. Three say "hamburgers", two say "hot dogs" and one says "a salad". Four say "I don't know yet. I am leaning toward a hamburger but I am still thinking about it and I like the hotdogs at Bob's Diner and they do have a good salad too. Moreover, one of those people who selected a hamburger overhears this conversation and says "you know, a hotdog or a salad does not actually sound bad. Could you hold up for a second, I might change my mind." And, at the same time, the person who picked a salad says "you know, I like salads, but I don't want to be a pain in the butt. If I'm the only one picking salad and that makes an extra stop for you, I'll order to a hotdog."
If you were asked to report what people wanted for lunch, what would you say? Well, the pollsters came back and said "most people want hamburgers." Technically, that might be correct but it is not really correct, is it. The hamburger people had 50% of the decided menu option the first time people were asked about lunch but when people started talking about lunch, the orders quickly became messed up. The person picking salad was willing to change and one of the 50% of hamburger selectors was rethinking there choice while a number of people were still considering their options. What the pollsters and pundits did was forget about all this confusion. They assumed that the first pick people made for lunch was what they were going to stick with and that those people who had not yet decided, would decide in basically the same ratio as those who had decided. Thus, they came and said, OK, here is what the lunch orders will be 6 hamburgers, 3 hotdogs, and a couple of salads.
But, you can see from my example that a different type of order would be easily possible, and indeed probable now that folks were talking about lunch instead of just making a decision on the spur of the moment. Instead of 6-3-2 ratio, you could just as easily get 3-6-1.
The problem, of course, is not that the pundits did not explain their methods and their reasoning. The problem is that they did not think deeply enough about what the numbers mean. I read a column that said, in effect, this: there is no way the poll numbers could have been wrong because everyone had the same numbers. Following that logic (in the words of the old Meatloaf song): "the world would still be flat." Our pollsters have good math. They can mathematically sift through data and draw correlations that are so far beyond my ability to do so that it is not funny. The problem is not that their math was bad. The problem is that they began their analysis with the wrong assumption: our math is so good, we don't need to look at this election from other directions. We don't need to think about the complexities, the details, the way in which thinking can change people's minds, the way in which a fairly intense public discussion can lead people to say "you know what ... I changed my mind. I'm not 100% certain but I am thinking about voting differently then I was last week."
One thing we need to learn about punditry, then, is that you need to take that extra step.
No comments:
Post a Comment