So the UK General Election results are now known except for 1 out of 650 seats, Tories have 318 seats but needed 326 for a majority of 1. The Tory eventual vote share in 2017 was 43%, Labour was 40%, a 7% difference an puts the Tories 8 seats below. In 2015 it was Tories 36.9%, Labour 30%, a 23% difference.Our algorithms had it spot on
, predicting a hung Parliament, lower and median were hung parliaments with the upper bound a slightly smaller Tory majority. So we got it, right?
Well, yes, the algorithms behaved very well, but the humans (mainly me) didn’t.
The reason was the worry over social media biassing too liberal/labour, as it did in 2010 and 2105. Now we knew it wasn’t as biassed as 2015, given the larger demographic now on it, but we (I) thought there would be a bias, which we put at somewhere between 3 and 8%, so our adjusted eventual range meant the median was a small Tory win, the high end was a bit better than they did, the lower was a hung parliament. The predictions were very tight - we were talking c 20 - 30 seat spread over 650 seats, but the Tory win was on the bottom limit, and we actually called the median as a small Tory win
So - algorithms got it (there was nearly no social media bias in 2017) and the humans over-compensated for the bias.
(To be fair, this was still a range from hung to “slightly better than last time”, with a tiny solution range and it was a damn sight better than nearly all the polls were able to do, and way better than the pundits)
Another thing we'd note is that the narrative played out in the polls, punditry and press about the election bore little relationship to what we were seeing, and as our systems were nearly spot on in their prediction and most of the above were way off, I am inclined to believe our system a lot more (It also makes me very cynical about the ability and motives of said polls, pundits and press). To summarise, we saw the following:
- There was no "huge Tory lead", it was a chimera, they were fooling themselves.
- In week 1 the LibDems made most of the running but quickly fell away as the main parties got going
- After a week or so Labour started to close the gap on the Tories.
- Labour's Manifesto was a step change. We saw no sharp impact from the Tory Manifesto, except from about then on Labour started to gain on them faster. They were quite close to each other so may all be wrapped up together..
- The only topics to "break the surface" as very influential were Brexit and the NHS, Scottish referendum a distant 3rd. All others were in the noise.
- From about 2 weeks before election day the end outcome had emerged on our system, if you go back to our post 1 week before the election
you can see the result graphically and it hardly changed.
In short – fire the human, promote the algorithms!
We have turned our analytic engine onto the UK election, looking at social media to predict it. It has worked for Brexit and Trump, got both French phases right (but underestimated Macron). It says that the Tories will win, with about the same majority as now, with a margin of error somewhere between a small loss (largest party but not enough to form a majority) and a c 20 - 25 seat lead).
(Update - the results are now known, and it was the lower bound of our prediction. In fact, the algorithms had it spot on, the error was me adjusting for possible social media bias - see below - which was non-existent in fact.)
You can see more detail of the prediction here on the DataSwarm site
However, the hard bit is predicting the bias of Social Media - in 2015 Social Media thought Labour would win, but the Tories squeaked in, there was quite a strong bias towards Labour on social media . We don't know where social media is now, we believe there is less of a bias as there are more people on it ,so that means it is more representative of the demos. In our view it is worth several additional % points which gives an outcome somewhere between a hung vote and several tens of seats lead, on average about the same.
Tomorrow, all will be revealed and we will no doubt be tweaking the predictive algorithms again.
This is a summary of the more detailed post on what we are seeing on the DataSwarm Analytics blog
. In essence we are seeing Labour's rise continue, and they are now very close to the Tories (or more accurately, the "Theresa Party" as that's what their strategy has been till very recently. Here's what we've been seeing over the election weeks:
The Liberal Democrats came out the blocks early, with a clear story, and garnered a disproportionately large social media share initially. As expected, the other opposition parties took a while to get going, but we were surprised that the Tories – who called the election so one would expect them to hit the ground running, did not.
In fact, we have seen the opposite for the Tories. They seemed slow to get going, and never really took off. We also never saw this “huge lead” they were supposed to have that convinced them to call the election (we have some views on this, based on bubblevision but will wait till after the election to talk about that), and support has ebbed away from them from the outset.
Labour have been doing well, picking up support. We see two major drivers of this:
- The Labour Manifesto was very popular with a lot of people, not just traditional Labour voters
- Jeremy Corbyn is simply not as bad as the British media and his opponents have repeatedly painted him, so – as with Trump – he wins merely by being better than the very low expectations people had. In this way I think the anti-Corbyn brigade have shot themselves in the foot.
The other parties are quite small or regional, they won’t shift much so we are not really watching them.
Some caveats though - we have been here before, Social Media always underestimates Tory support as its key demographics are largely missing, and the "Shy Tory" (talk liberal in public, vote conservative in private ) effect.
So, who do we think will win? Well, there was a debate last night which is showing signs of being a "tipping point" for Labour but its early days and the last weekend will be a major influence we think. We will give our final predictions early next week - stay tuned.
Le Pen couldn't win now, right? (Above - 538's prediction of Trump's chances of success the night before the US election)
Well, using our analytical systems we predicted that it would be Macron & Le Pen last Friday
, and this Monday it is clear that this has come to pass.
But it has been an interesting experience, for 2 main reasons:
1. Our system is based on social media data analysis. But France has a lower social media penetration than the US and UK so what happens on social media is not as good a representation of the overall picture as the US and UK. Thus, our results needed quite a lot of interpetation and normalisation. One day there will be algorithms that can do this easily, but today it's still largely about pattern spotting, and knowing what's happened elsewhere and/or before (eg UK Liberal Democrats were going to win "bigly" according to Twitter in 2010, but they came distant third - because early social media adopters trend strongly liberal. We saw this in France too with Hamon & Melenchon)
2. Macron is a "Mule"
in the Asimovian
sense - he came out of nowhere (he had no presence or even party a year ago) and he broke any models one can build from looking at long term patterns (Trump and the Brexit leavers are similar forces breaking today's politics, and one ignoree their impact at great risk). We built a "Euromodel" for elections to test against based on recemnt trends in UK, Holland, France & Germany, and the main hypotheses were:
- The centre-right absorbs more of the far-right thinking, squeezing them, while the more centre-oriented followers stay with the rightward moving centre-right due to the of lack of a better home to go to.
- The center left collapses, its voters move both right and left
The latter has happened again (as it did in Holland), but the Macron-mule effect meant that the former didn't - Macron has managed to replace the Centre-Right and take some of the rightwards moving Centre-Left vote without shifting that far Right (yet).
The big picture hasn't changed - our model said the 2nd round would go to the Far Right and Centre-Right candidates, and the Centre candidate should still win eventually, in theory. Also mapping a two horse race is simpler than a multiple horse one, even if it is very close. But "Macron the Mule" is not the original Centre-Right and still "anything could happen", so 538's predictions today that Le Pen is over
are surprising given their experience in the US (the graphic at the top of the page is their prediction the day before the US election voting). While they are making a sensible prediction sensible based on the data so far, they are in our view premature as the 2 horse race has hardly started and the data has not really started to tell us what will really happen - we will need to take all the lessons of the first round into the second.
Obviously a very serious entry and in no way, an excuse for the headline.....
This is Kasper, our Portuguese Water Dog, who is now on the Internet, thanks to his GPS/GSM tracker. Kasper has a fascination with the local muntjacs. Fortunately, they run much faster than him, but he does sometimes end up lost after a chase. So we have fitted him out with a tracker and it works quite well, allowing us to track him in real time via a mobile app. The battery lasts over 24 hours, so if he was seriously lost we would have a good chance to locate him. He has his own location systems and usually finds us first!
The point of this (really!) is the growing ubiquity of the IoT (Internet of Things) and in this case the IoD (Internet of Dogs).
More Recent Articles