Opinion Polling In 2019: More Art Than Science?



IT WAS ANDREW COOPER wot won it! Or so might the Brexiteers have crowed, paraphrasing the Sun’s famous headline of 1992, when its concerted assault on Neil Kinnock turned what looked like a dead cert for Labour into a surprise win for the Tories’ John Major.

Who is Andrew Cooper?  He was the chief pollster for the “Remain” campaign. The man whose assumptions, about how many Britons would turn out to vote in the European Union Membership Referendum of 23 June 2016, lulled Prime Minister David Cameron into a false sense of security. Allowing the “Leave” campaign’s chief wizard, Dominic Cummings, to steal a victory his enemies had, quite literally, not seen coming.

That’s the thing about polls. It’s not simply a matter of assembling the data: whether by calling landlines, mobile phones, or, acquiring people’s opinions online; it is also about how you interpret the data. What are the assumptions you make about how the hundreds, or thousands, of people you have interviewed will behave when the time comes to cast their votes – or not.

TDB Recommends NewzEngine.com

That was Andrew Cooper’s big mistake. According to Tim Shipman, author of “All Out War: The Full Story of How Brexit Sunk Britain’s Political Class”, Cooper assumed that the same percentage of Britons would sit out the 2016 Brexit Referendum as had sat out the General Election of the year before. Take these ‘can’t be arsed’ voters out of the equation, and – Bingo! – “Remain” emerges as the referendum winner every time.

He was, as well all know now, quite wrong in that assumption. The turnout for the Referendum was well up on the 2015 General Election. Cummings had successfully mobilised the forgotten, the marginalised, and the very-bloody-angry. The Britons who had given up voting, because, after Tony Blair, the largely indistinguishable politicians of the Tory and Labour parties always won. This time, however, said Cummings, their votes would count. This time, they could deliver an almighty “Fuck You!” to the entire British Establishment, and there would be nothing – absolutely nothing – they could do about it. “Take Back Control”, bellowed Cumming’s inspired slogan. And, they did.

It is something like this. Something in the interpretation of the raw data. Something about the assumptions they’re making, that explains how Reid Research and Colmar Brunton were able to come up with such wildly different descriptions of New Zealand’s current political landscape. And that ‘something’ is very important. With so few publicly available opinion polls being conducted in this country, we really do need to know how one reputable polling agency can put National on 46 percent, while the other puts it on 37 percent. Nine percentage points is a hell of a gap – well beyond the usual statistical tolerances. The public is entitled to know what happened – and which pollster got it right!

But, is that even possible anymore? Getting it right? After Brexit, Trump, the British snap-election of June 2017, and the Australian Federal Election of May 2019, can “The Polls” be trusted to tell us anything at all reliable about the voting intentions of the public?

Is it possible that opinion polling, like black-and-white television, was something peculiar to a particular historical era? To a time when everybody had a landline telephone; read the same newspapers; watched the same shows, at the same time, on television; and were limited to a choice between two very large and broad-based political parties?

Back then, when people still had a huge amount of faith in the political system, polling struck many voters as a welcome, and highly scientific, addition to the process of determining their electoral preferences. When a pollster called, the person picking up the phone felt under a heavy obligation to co-operate fully. To refuse, was tantamount of declaring oneself a bad citizen. Answering the pollster’s questions honestly and comprehensively was the randomly chosen interviewee’s civic duty.

How things have changed. Fewer and fewer voters have landlines. As a consequence, the phonebook is no longer sufficient to the task of preparing a statistically valid survey. And, unfortunately, there is no handy, publicly available, register for cell-phone subscribers. Even if there was, the dutiful citizenry required to make the survey truly representative has largely disappeared.

Because, while the pollsters were learning every little quirk about the voting (and buying) public, the public was also learning a thing or two about polling. What it was for; how it was being used; and by whom. With painful slowness, more and more voters realised they were being used. That what they were participating in wasn’t a matter of civic duty. It was a marketing exercise, impure and simple. They were helping the politicians to manipulate the electorate – i.e. themselves. Clever “communications specialists” were persuading them to buy back their own opinions.

This is the reason why polling agencies have so much difficulty in persuading the people they contact to participate in their surveys. For every one person who agrees to be interviewed, there are dozens who refuse. So many, that in statistical terms, the participating subjects have become unusual in themselves, and their opinions have to be weighted accordingly. Then, there are the respondents who deliberately mislead their interviewers. The “rat-fuckees” who become “rat-fuckers”.

Confronted with these formidable obstacles, opinion polling is becoming less and less a science, and more and more an art. The trick now lies in assessing how much of the sample to write-off as tricksters; how many as atypical political junkies; how many as persons who enjoy being asked for their opinions, but who will never make it off the sofa on election day. These questions cannot be answered scientifically, they can only be guessed at. Those guesses will be guided by experience; by ex post facto analyses; and, crucially, by intuition.

One pollster will say: “Oh sure, people love Jacinda – but will they turn that love into a bona fide ballot-paper stuffed into a ballot-box? Young people may revel in Jacinda’s stardust, but its old people who vote. We need to be conservative in our assumptions about the relative weighting of these two groups.”

Another pollster might say: “Normally, I’d apply a pretty heavy discount to the under-25 preferences. But, you know what? I’m betting that, in 2020, the under-25 vote is going to spike. I’m looking at all those secondary-school kids on the street demanding action on climate change. I’m thinking about the influence they’re going to have on their big brothers and sisters. Heck! The effect they will have on their parents! That’s why I’m assuming a significantly higher turnout.”

The first pollster might predict a solid vote for National. The second, a game-changing surge towards Labour and the Greens. They both can’t be right – but I’m hoping that more than 50 percent of voters would like to see the second prediction proved correct.


  1. Roy Morgan has scarpered it seems, David Farrar withdrew from Colin James “poll of polls” before the 2017 General Election, and there are not enough polls still standing or conduced frequently enough, to even have a “poll of polls” anymore. The “Trend” was how differing polls were accommodated, ever more uncomfortably it has to be said, in recent times.

    After 30 years of neo liberal psychology penetrating (and atomising, as academics used to say) the population, there are now so many micro demographics and niches liable to do anything. Precariat, generation rent, well off boomers, struggling boomers, millennials, Xers, newish migrants, and on and on.

    From a class analysis, NZ has a large underclass–thank you Roger ’n’ Ruth…, thousands of low paid workers, a taxpayer supported middle class via WFF and Accom. Supplements, a sizeable petit bourgeois of self employed, franchises and SMEs, and still a provincial urban divide–much complexity in a small population.

    So yes the polls are up against it alright.

  2. Great analysis, Chris! I think the message for the polling companies is never to do a poll at nearly exactly the same time as their rivals. They have inadvertently exposed the limitations of their craft.
    Colmar Brunton and Reid Research need to look after each other’s interests by agreeing to have at least a month between polls so any sudden shifts can be explained by some intervening event or another… however flimsy the explanation that might be.

  3. who care if we have two different polls the ones common theme in the two polls say soimon is extremely unpopular

  4. Word has it that those Schrodinger’s Cat boxes increasingly contain quite grumpy Raccoons that attack your face, yet this considerable health and safety hazard is routinely ignored by the Cattery Committee.

  5. I lied to a Curia phone poll about 6 months ago, but just for fun as I’ve always thought Curia a bit politically suspect.

    I’m flabbergasted to learn that I am now a ‘rat-fucker’.

    I won’t do it again. I always thought that only men were rats, and in the autumn of my life I am having to face the fact that women can also be rats, and that apparently I am one of them. Oh lordy.

    Still think Simon the best of a bad bunch – except for Maggie Barry – let Maggie loose with some rusty hedge-clippers and a basket for all the chopped-off heads – and another basketfull of abortion leaflets from
    nrly-Rev Simon O’Connor- and see what the pollsters make of all that.

    • I always lie to polls, giving the opposite of my real view and I encourage everyone to do the same. They’re all invalid nonsense with their self important sounding ‘margin of errors’ and it’s fun to bugger them as someone called Jim once said. ( I once did my own poll – random sample of 5 people with a margin of error of 98 % )

  6. The reality is that Kiwis trust our political process less than ever, and trust pollsters even less than that.
    The polls have become just another form of marketing, and brand formation with little relevance to what the electorate actually thinks, other than being a demented sales pitch.

    We all know the pollsters have political bias and that the questions are rigged. We have no real interest in the results of polls, or at least less of an interest in the polls than the latest gossip about some random rugby hero’s girlfriend or boy toy.
    Sure the right is prepared to spend millions on right leaning pollsters. After all it’s a tax write-off isn’t it. And the left’s contribution to the pollster nonsense is a much tinier spend, relying more on the grapevine than upon branded marketing.
    People remember the left with favour, after all the only times the people of this country have faired marginally better is under a Labour Government. But fair minded kiwis give National the chance, repeatedly, to build the better future that is promised but never delivered. Then revert back to Labour when it all becomes way too hard because families need to be fed to survive.

    Branding, polling, sloganeering, badmouthing, and other marketing nonsense are the tools of slick used car salespersons, politicians, and advertising executives that in today’s age are trusted about as much as the nasty perverted relative that we all despise.

    Perhaps polling peoples dogs and cats by laying out different coloured bowls of food would provide more honest, and consistent results.

Comments are closed.