Archive for Nate Silver

Polls Apart

Posted in Bad Statistics, Politics with tags , , , , , , , on May 9, 2017 by telescoper

Time for some random thoughts about political opinion polls, the light of Sunday’s French Presidential Election result.

We all know that Emmanuel Macron beat Marine Le Pen in the second round ballot: he won 66.1% of the votes cast to Le Pen’s 33.9%. That doesn’t count the very large number of spoilt ballots or abstentions (25.8% in total). The turnout was down on previous elections, but at 74.2% it’s still a lot higher than we can expect in the UK at the forthcoming General Election.

The French opinion polls were very accurate in predicting the first round results, getting the percentage results for the four top candidates within a percentage or two which is as good as it gets for typical survey sizes.

Nate Silver Harry Enten has written a post on Nate Silver’s FiveThirtyEight site claiming that the French opinion polls for the second round “runoff” were inaccurate. He bases this on the observation that the “average poll” in between the two rounds of voting gave Macron a lead of about 22% (61%-39%). That’s true, but it assumes that opinions did not shift in the latter stages of the campaign. In particular it ignores Marine Le Pen’s terrible performance in the one-on-one TV debate against Macron on 4th May. Polls conducted after that show (especially a big one with a sample of 5331 by IPSOS) gave a figure more like 63-37, i.e. a 26 point lead.

In any case it can be a bit misleading to focus on the difference between the two vote shares. In a two-horse race, if you’re off by +3 for one candidate you will be off by -3 for the other. In other words, underestimating Macron’s vote automatically means over-estimating Le Pen’s. A ‘normal’ sampling error looks twice as bad if you frame it in terms of differences like this.  The last polls giving Macron at 63% are only off by 3%, which is a normal sampling error…

The polls were off by more than they have been in previous years (where they have typically predicted the spread within 4%. There’s also the question of how the big gap between the two candidates may have influenced voter behaviour,  increasing the number of no-shows.

So I don’t think the French opinion polls did as badly as all that. What still worries me, though, is the different polls consistently gave results that agreed with the others to within 1% or so, when there really should be sampling fluctuations. Fishy.

By way of a contrast, consider a couple of recent opinion polls conducted by YouGov in Wales. The first, conducted in April, gave the following breakdown of likely votes:


The apparent ten-point lead for the Conservatives over Labour (which is traditionally dominant in Wales) created a lot of noise in the media as it showed the Tories up 12% on the previous such poll taken in January (and Labour down 3%); much of the Conservative increase was due to a collapse in the UKIP share. Here’s the long-term picture from YouGov:


As an aside I’ll mention that ‘barometer’ surveys like this are sometimes influenced by changes in weightings and other methodological factors that can artificially produce different outcomes. I don’t know if anything changed in this regard between January 2017 and May 2017 that might have contributed to the large swing to the Tories, so let’s just assume that it’s “real”.

This “sensational” result gave various  pundits (e.g. Cardiff’s own Roger Scully) the opportunity to construct various narratives about the various implications for the forthcoming General Election.

Note, however, the sample sample size (1029), which implies an uncertainty of ±3% or so in the result. It came as no surprise to me, then, to see that the next poll by YouGov was a bit different: Conservatives on 41% (+1), but Labour on 35% (+5). That’s still grim for Labour, of course, but not quite as grim as being 10 points behind.

So what happened in the two weeks between these two polls? Well, one thing is that many places had local elections which resulted in lots of campaigning. In my ward, at least, that made a big difference: Labour increased its share of the vote compared to the 2012 elections (on a 45% turnout, which is high for local elections). Maybe then it’s true that Labour has been “fighting back” since the end of April.

Alternatively, and to my mind more probably, what we’re seeing is just the consequence of very large sampling errors. I think it’s likely that the Conservatives are in the lead, but by an extremely uncertain margin.

But why didn’t we see fluctuations of this magnitude in the French opinion polls of similar size?

Answers on a postcard, or through the comments box, please.


American Psycho

Posted in Politics with tags , , , on November 8, 2016 by telescoper


Well today’s the date of the election of the next President of the United States of America. Will it be Hillary Clinton? Or will it be an unstable racist misogynist, a pathological liar, and a man who has boasted of a string of sexual assaults? It remains to be seen. The polls are alarming close. Hillary Clinton is ahead by just 3 or 4 percent nationally but only a handful of states really matter and some of those are too close to call. I’ve been following Nate Silver’s 538 election forecast for a while now. It seems to me his methodology more accurately estimates the uncertainty in the opinion polls. After narrowing considerably when the FBI decided to throw a spanner into the works last week, the probability of a Clinton win is now over a little over 70%. Uncomfortable, but the odds have been below 2-1 very recently.

At the weekend I decided that I would follow my usual betting practice and place a wager on the outcome that I don’t want to happen. Hunting around, the best odds I could find were 18-5 against Donald Trump. I put a monkey on, so will walk away with £2300 if Trump wins. I plan to use the proceeds to begin work on the construction of a fallout shelter in my garden. If an unstable psychopath like Donald Trump gets his hands on the American nuclear codes I don’t hold out much hope for the future of civilization.

I followed the same strategy on Referendum Day as I felt it in my bones that Vote Leave was going to win. I ended up depressed but compensated to the tune of £1000. I’m afraid to say I feel the same way now about the likelihood of a Trump victory. Not very scientific, I know, but there you go.

I have never paid much attention to American politics in the past. It is as incomprehensible to me as British politics must be to them. Gore Vidal summed it up for me:

There is only one party in the United States, the Property Party … and it has two right wings: Republican and Democrat. Republicans are a bit stupider, more rigid, more doctrinaire in their laissez-faire capitalism than the Democrats, who are cuter, prettier, a bit more corrupt — until recently … and more willing than the Republicans to make small adjustments when the poor, the black, the anti-imperialists get out of hand. But, essentially, there is no difference between the two parties.

Things have changed this time. Although both parties still represent the moneyed classes more than anyone else, but this time the Republican contender has overtly fascist tendencies. No wonder Nigel Farage admires him so much. Americans are free to vote for whomever they wish, of course. I don’t have a say, as I’m a foreigner. All I can say is that you should be very careful what you wish for.

Although I find it deeply depressing that this race is even close, I won’t lose any sleep over the election night. I don’t have a television, and I’ll do what I did on the day of the EU referendum. Drink some wine, listen to music and then go to sleep. There’s no point in worrying about things that are out of your hands. And if Trump does win, at least it shortens the list of countries I will have to consider emigrating to if and when the UK does leave the European Union..