Wednesday, 22 October 2008

The new MORI poll and why it doesn't make sense

Lib Dem Voice a few days ago assessed the crop of polls and argued that the ICM poll that had the Lib Dems on 21% was more accurate than those that had us on the mid teens. All based on sound psephological reasons, natch.

I'm now going to have a dig at the MORI poll out today which shows a 13% swing between Labour and Conservatives. Last month - Tory 52, Labour 24. This month - Tory 45, Labour 30. As PB argues - still a massive Tory lead, but a huge shift.

The reason why there should be such a shift is sort of understandable. A month ago Labour was in no end of turmoil. Brown had not yet has his successful conference and he had yet to usurp Sarkozy and the Swedes as the man with the plan for saving the world. So perfectly normal that people should change their minds you would think.

But here is what is bothering me. The MORI polling method includes just people who are certain to vote. To my mind, that would tend to narrow down the number of switchers. People who are less certain to vote, so my argument goes, are more likely to be flexible about which party to support and more likely to persuasion by political fortunes. Sure, there will be some for whom the civic duty of voting is absolute, but I feel there are many more who are certain to vote because they are certain they will always vote for a certain party.

Which brings me onto my biggest gripe with pollsters. Their refusal ever to say sorry. Every poll they have ever carried out is right, they will argue. You and I know that this cannot be right. Polls on the same day point to wildly different vote shares. Yet the pollsters, even in retrospect, will argue that their own poll showed what people were thinking at the time (no possibility of discovering the truth of course).

Of course the methodology suggests that 95% of polls should be accurate to within 3% (depending on sample size, but this figure is based on the usual size of just over a thousand). That means that one in twenty will be out by more than 3%. They are 'rogue polls'. Yet to ever infer that a particular poll might be a rogue is to invite threats of legal action (I know, I've been there). Yet you will never hear pollsters admitting which of their polls might have fallen into this category. Even when you look back the accumulated polls of all firms stretching back over a considerable period and see an unexplained outlier. Nope, says the pollster, that was wht people were thinking at the time. They just changed their minds straight after.

I'm not, of course, suggesting that the current or previous MORI polls are rogues. But it would be nice occasionally to hear pollsters admitting which they got wrong.

1 comment:

Mark Senior said...

It is always useful to look at the detailed data in these polls .
Looking at the Mori poll , the raw data is 136 people voting LibDem now compared to 106 ( both figures out of 1004 ) who voted LibDem in 2005 . Mori do not weight for past vote but ICM weight to a past vote figure of 13% . ICM would therefore automatically have weighted up the 136 figure to circa 155 and a headline LibDem figure of around 19-20 % .
What evidence do we have that the ICM approach is more likely correct than Mori ? We can again find some from the detailed Mori data .
They find 30 people who voted Labour in 2005 now saying they will vote LibDem and just 6 who voted LibDem saying they will now vote Labour .
They similarly find 15 switchers from LibDem to Conservative with 7 in the reverse direction . It is not possible to deduce the interchanges with Others .
Clearly these figures indicate a much better performance for the LibDems compared to 2005 than the Mori published figures for this poll of 17% or just 14% on their absolutely certain to vote figure .