2017 General Election: The things I got wrong

I think it’s important, both as an academic and a politician, to look back at things and see where you went wrong in the hope you won’t make the same mistakes the next time around. It’s important to get rid of all those errors so you can make a whole set of brand new ones the next time around, rather than just repeating the same ones again and again. I’m lucky in that my chosen field within political science is parties and party systems which is related to and uses data from political behaviour and elections, but is much more about analysing things after the fact rather than trying to test theories by making predictions.

That’s why I wasn’t building a complex model to predict the election and didn’t really jump into making anything more than the vaguest predictions. However, that didn’t stop me being wrong about YouGov’s prediction which, along with the broadcasters’ exit poll, appears to have been the most accurate of all the models. I dismissed it because it didn’t match up with my expectations and perceptions, so I did the natural thing (as did so many other people) of sucking in through my teeth and muttering ‘dodgy methodology’ and ‘looking for headlines’, without thinking about why they might have come up with something that challenged my perceptions.

One importnt thing to learn is that big data crunching like this has a better perspective than you. From the bits of Colchester I’d seen and spoken to, I didn’t feel that Labour were in second place here, but until yesterday I’d never seen people queuing to vote in my local polling station either which was a clear sign of something unexpected going on. It does raise an issue I think we often elide in our discussions of voter behaviour where we assume that ‘the voters’ and ‘the non-voters’ are the same people at each election, and often neglect to consider movement between the two groups. We also – and this is something common to politicians and academics – forget that people don’t exist solely in terms of our labels. Just because we have someone down as a Tory, Labour or Lib Dem voter doesn’t mean that they consider themselves that in the same way and in some conditions – especially when the links between parties and voters are weak – they’re not going to behave in the way we expect.

I also missed the relative popularity of Jeremy Corbyn, though in my defence his election did seem to be following the same pattern as Miliband’s two years ago: the crazed revolutionary depicted in the right-wing media turning out to not be much like that when the public saw them, getting more confident as the election went on, but then a final onslaught of negative press burying them. Except this time Corbyn managed to keep that momentum up, and even if he didn’t shake it off to the extent that Blair did, he did achieve it better than Miliband.

I’ve only had a couple of hours sleep in the last thirty-six, so those are the errors that come to mind right now, but do feel free to go through my election posts in painstaking detail and point out anything else I got wrong in the comments. I’m still mulling over the questions of where we are and what comes next, but things aren’t unfolding with the same sense of post-election urgency that they did in 2010 and 2015 – possibly because everyone’s still shell-shocked from a bizarre night – so writing about that can wait until tomorrow when my brain’s capable of thinking in a bit more depth.

Tempted to call that next post Day 1 of the 2017 General Election v2, but I’ll probably resist that temptation when I’m better rested.

2017 General Election Diary Day 43: And now it gets interesting

You know when you’re trying to think of a way to describe how things are going in the election as things turn a little weird and then someone hits the perfect metaphor? That:

I know I promised back at the start of this diary, all those weeks ago, that I wouldn’t spend it following 2015 into the dark corners of polling obsessions but YouGov threw out a little hand grenade of a projection last night, suggesting that things might be about to get weird on us. Rather than putting the Tory majority in the ‘how far back do we have to go to find a comparison?’ range, it instead suggested they might lose seats (and their majority) while Labour would gain to put us firmly in hung Parliament territory. The interesting thing about this was that it wasn’t based on applying a uniform national swing across constituencies but instead looking at how different demographics have said they would vote and then working that out constituency by constituency. It’s a controversial method, that didn’t come up with the right projection for the US Presidential election in the elecoral college last year, but it would be something that produced contrary results to other pollsters if this is a realigning election where there’s mass movement of voters between parties. If that happens, then it will make election night very interesting as results won’t be easily predictable by extrapolating from the first few.

It also offers up the joyous prospect of the Tories gaining votes while losing seats. If any of them were to then complain about this as being an injustice and the voters not being properly represented, I may well die laughing.

Of course, this is the point in election campaigns where people can get over-excited and all sorts of wild speculation can break out. It’s where people spend time debating whether the Edstone will need planning permission to be erected in the Number 10 garden, where we wonder which Liberal Democrat candidates might be able to be appointed straight to ministerial office in the Clegg government or any other number of scenarios that seem likely in the heated air of an election campaign, then afterwards are forgotten about as everyone remembers that the result was the one they predicted and expected all along. It’s a national outbreak of candidatitis, sweeping out from party activists to infect the whole country, then disappearing some time around 10pm next Thursday.

And if a wild projection wasn’t enough to excite you, the country – or that bit of it that obsesses over politics on social media, at least – has got debate fever. Yes, tonight is the BBC election debate, which has been suddenly made an event of interest by Jeremy Corbyn today announcing that he would appear in it having previously said he wouldn’t. This means the Conservatives will now be the only party there without a leader representing them as Amber Rudd will be standing in for Theresa May while the Prime Minister goes off to speak to a small rally of Tory activists in a carefully sanitised warehouse somewhere off the M4. Sorry, I meant campaign and ‘meet the people’ because luckily, she’s not campaigning for a job that occasionally requires you to meet in public and debate with other people.

It’s a clever move by Corbyn, as he does have the momentum in the head to head battle and unless he breaks down and declares ‘all power to the Soviets!’ in the middle of the debate (not that quoting Lenin is necessarily harmful nowadays) he can continue to disarm the Tory strategy against him. They’ve been painting him as a crazed Marxist revolutionary wanting to bring down the system, but his recent appearances (especially against Paxman) have been more sardonic history teacher who the students love because he keeps going off on tangents in lessons and never sets any homework. Everyone’s now frantically re-preparing their tactics and points for tonight, which might even make it interesting. That’s why I’m writing this beforehand, when it might still be interesting, rather than afterwards when the reality sets in and commentators intone ‘we are all Ruddites now’.

As ever, we shall conclude with Election Leaflet Of The Day which today comes from an interesting independent – Tim Lord, standing in Cities of London and Westminster. Like many independents he has one big issue he’s standing on but his is an interesting case of the national becoming local in a distinct constituency. ‘Voted Remain? Vote for Tim.’ is his message, pointing out that the Cities’ current MP, Mark Field, is signed up to May’s Brexit strategy, and as it’s a place with lots of interests in maintaining close ties with the EU, he’s hoping that will motivate them to switch to him. (This article spells it out in more depth) It could be an interesting tactic that delivers a shock, it could be yet another damp squib, but it makes a usually safe seat somewhat interesting.

Eight days left until activists who’ve been up since the crack of dawn gird themselves for a push at reminding people getting home from work that it’s time to go vote.

2015 General Election Day 24: Breaking the silence

yougovpredictionsI’m going to break one of my rules for these election blog posts today to talk about a poll. However, this one isn’t a voting intention poll, so I won’t be having to write ‘variations within the margin of error’ too many times. No, what I want to talk about is this YouGov poll, asking voters to predict who they think will win the election. This wasn’t asking who they wanted to win the election – one expects that would be similar but not identical to voting intention – but rather who they think will win the election. The same questions had been asked in February and found that when asked which party would win 42% thought the Conservatives would win, compared to 30% Labour and in terms of Prime Minister, 44% thought Cameron compared to 24% Miliband.

Now, though, the results are a lot closer: 37% Conservatives 36% Labour on the party question and 37% Cameron 34% Miliband on the leader question. Yes, there’s a difference between the two, but I think that just shows gaps in political knowledge rather than anything else. What’s more interesting is the way people’s expectations have changed in the last two months, and what this might tell us about the election result. There’s a famous book and theory in public opinion research called The Spiral Of Silence, written by Elizabeth Noelle-Neumann. The theory covers a lot of culture and mass communication, but one of the starting points for it was a German election in the 1960s (I can’t remember the exact one, and someone has the library’s only copy, so can’t check). There’d been regular polls throughout the election showing the election was close, but the final result – and the prediction of Noelle-Neumann and her team – was a clear CDU victory. They’d spotted this by not just asking people how they were going to vote, but who they thought would win and noticed that while the voting intention question remained tight, the CDU opened up a lead in the perception question which widened as the campaign went on.

The theory is that because people thought the CDU would win, they were more likely to decide later on that it was better to join the majority and back the winner and this accounted for the late change that wasn’t captured by the polls. In terms of this election, the February result backs up the initial theory of this election – that as it got closer, more and more voters would break for the Tories (Lynton Crosby’s proposed ‘crossover moment’), following the expected winner. Unfortunately for that theory, the latest figures show that something quite different has happened. A Tory victory – and Cameron’s lead over Miliband – is no longer widely expected and if anything, the momentum is with Labour. We don’t know what’s driven this change – it could be people reassessing Labour and Miliband after seeing him, it could be bad reaction to the Tory campaign, it could be the effect of local campaigning – but it’s a much more interesting change than anything we’ve seen in the voting intention polls so far, and it’s an indication that this is turning into a much tougher fight than the Tories expected.

Another indication of that is Boris Johnson joining David Cameron on the campaign trail today. He’s been kept mostly within the M25 for the first few weeks, and I think that’s entirely deliberate, as Cameron and Osborne wanted to be seen winning the election without Johnson’s help to boost Osborne’s chances in the next Tory leadership fight. I also think bringing him in is a risk as I think the point will soon come when there’s a public tipping point and people become much more aware of the real politician hiding behind the ‘What larks! Ho ho, here comes Boris the bumbling fool!’ facade.

Not worrying about image or minor things like being expected to win, today’s minor party is the Socialist Party of Great Britain (SPGB), standing ten candidates in the election, mostly in the south east. The SPGB is one of Britain’s oldest parties, formed back in 1904 as a split from the social democratic wing of the labour movement that would eventually form the Labour Party. The SPGB took an ‘impossibilist’ stance as opposed to Labour’s reformist one, holding that capitalism needed to be overthrown rather than reformed from within. It’s important not to confuse them with the Socialist Party (the former Militant, now part of TUSC) as relations between the two are about as good as you might expect between two splinters of the Left with easily-confused names. The chance of a peaceful solution to that dispute in the near future is about as likely as either side being represented in Parliament, but the SPGB has managed to continue for over 100 years without that so they’re definitely in it for the long haul.

They have been out leafletting, and Election Leaflets today gives us a good example of their work. It’s not a bad leaflet, actually, especially compared to the sort of information you normally get from left-wing parties and candidates. It doesn’t assume that the recipient is well-versed in Marx, it doesn’t hector them or shout slogans at them and it doesn’t spend half its time indulging in internecine warfare against other factions and fractions. If they carry this on, the five hundred years they’d need to get majority support (as calculated by Ken Macleod in The Star Fraction) might be shortened by a decade or two.

On political stereotypes and Doctor Who

YouGov have done a survey asking people their opinions about Doctor Who and what characteristics they want to see in the next Doctor. As politics and Doctor Who are two of this blog’s continuing obsessions, I couldn’t resist writing about it – and this post becomes even more ‘my entire blogging history in one post’ if I tell you I’m doing it while I wait for the highlights of the Criterium du Dauphine cycling to come on TV.

(Insert your standard disclaimer here about polling not necessarily being accurate, margins of error, just a bit of fun etc)

It’s perhaps not surprising that Lib Dem voters are more likely to be Who fans than supporters of other parties (see Alex Wilcock’s ‘How Doctor Who Made Me A Liberal‘ or my take on it here) but it’s nice to see it statistically confirmed – 41% of Lib Dem supporters are interested in the series, compared to 34% of Labour, 29% of Tories and just 26% of UKIP supporters.

I’m actually surprised to see David Tennant topping the ‘favourite Doctor’ part of the survey by quite a convincing margin – 43% to Tom Baker’s 16% and Matt Smith’s 14%. He won a similar DWM poll while he was the Doctor, but he’s now three years out of the role, which does indicate that he may well have replaced Tom Baker as the public’s image of the Doctor. (He is one of my favourites, but if I’d have been polled, I’d have doubled Patrick Troughton’s support amongst Lib Dems.) However, fun confirmation of stereotypes comes with Jon Pertwee getting his highest ratings from UKIP and Tory voters, but absolutely no support from Lib Dems. It’s possibly because he’s the most ‘establishment’ of all the Doctors – no other Doctor spent so much time hanging around the military – though one could also argue that the Pertwee era was full of images of a proudly independent Britain with its own space programme and big energy projects. As soon as he went, Tom Baker’s first story saw international sovereignty being pooled to protect nuclear codes in ‘Robot’ and the English countryside, if it was real at all, was depicted as being full of androids.

There’s also interest in the questions about what characteristics the new Doctor should have. Even without the breakdown by party, I’m surprised to see that the population of Britain are relatively open to the idea of a different Doctor. The only characteristics that get bare majority support are British (54%) and male (52%) – and male only gets about 40% support from Labour and Lib Dem voters. That gives me hope that when – and I believe it is a question of ‘when’, not ‘if’, even if it’s not this time – we get a female Doctor, the general populace will be much more inclined to accept it and see how it goes than certain Who fans believe they will be.

Other figures almost look as though they were created by the stereotype-o-matic such as 50% of UKIP voters thinking it’s important the Doctor is white, compared to 5% of Lib Dems, though I’m confused by a couple of spikes (which might just be statistical noise because of small sample size) – Tories are more likely to want the Doctor to be attractive, while Labour voters are more likely to want the actor to already be a household name.

My general position is that I want the next Doctor to be played by someone interesting – I’ve not been the biggest fan of the last three years of the series, but I think Matt Smith’s done a good job with some weak material and has been very good when he gets a good script – and most of the actors who I’ve thought could be interesting Doctors have been different from the norm. (That said, I do edge towards the ‘I’d like a woman Doctor, but not one written by Steven Moffat‘ position) If it was up to me, I’d be trying to persuade one of Adrian Lester, Maxine Peake, Chiwetel Ejiofor, Naomie Harris or Ben Whishaw to take the role – but it’s not up to me, so I just get to wait, watch and see what comes next. Hopefully, I’ll still be around for the 100th anniversary, when all this speculation will seem as quaint and irrelevant as ‘can you really get another completely different actor to play the Doctor?’ was in 1966.

If I can’t change your mind

Now here’s something interesting in the data for the latest YouGov poll. Not the headline figures – Conservatives 33%, Labour 30%, Liberal Democrats 29% – but a question asking people whether they might change their vote.

63% said they’d pretty much made up their mind, while 31% said they might change their mind. Those 31% were then asked which party they might vote for if they changed their mind, and responded:

Conservative: 10%
Labour: 13%
Liberal Democrat: 31%
Respect: 0%
Scottish National Party / Plaid Cymru: 2%
United Kingdom Independence Party (UKIP): 9%
British National Party (BNP): 4%
Green: 6%
Some other party: 2%
Would not vote: 9%
Don’t know: 15%

What that says to me is that not only have a lot of people have switched to the Liberal Democrats in the past few days, but that there are many other people (around 9% of the electorate, if I’m reading it right) who might come the same way in the coming weeks. Meanwhile, there’s a much smaller number of people who might switch to one of the other parties, and that suggests to me that the potential Liberal Democrat vote in this election is much higher than the potential Labour or Conservative vote – there’s a hell of a lot still to play for!

(original link via John Nor on Twitter)

No, I don’t know what the hourly rate is

A few months after Matt managed it, I’ve now managed to get my YouGov account over £50, which means I should be getting my first cheque from them soon. If I rememer when I started doing surveys there correctly, expect the next post of this sort smetime around 2010. You too can share in this abundance by signing up with them if you want to (ooh, that’s a slick way of getting you to click on a link that might make me a few pennies more, isnt it?).