Higgs-mania Day
I woke up this morning to the BBC Radio News at 7am announcing that scientists at CERN were going to report “hints” of the discovery of the Higgs Boson at the Large Hadron Collider; you can find a longer discussion by the BBC here. This was later accompanied by articles tackling the important questions of the day such as whether the discovery of the Higgs would justify the enormous expense of Brian Cox the LHC.
Prize for the most inaccurate science report goes to the Daily Fail:
‘God’ particle found:
Atom smasher reveals Higgs boson, the key to the universe
Evidence soon emerged however that this particular squib might be of the damp variety. Consistent with previous blogospheric pronouncements, a paper on the arXiv this morning suggested no convincing detection of the Higgs had actually been made by the ATLAS experiment.
I then had to make an important choice between watching the live webcast of the CERN seminar at which detailed information on the Higgs searches was to be presented or to accept a free lunch with the examiners of a PhD candidate. I chose the latter.
Catching up on events after lunch confirmed the underwhelming nature of the Higgs “detection”, but with some intriguing evidence an excess signal at around 126 GeV at the 2.3 sigma level, in the frequentist parlance favoured by particle physicists and others who don’t know how to do statistics properly. In the words of the late John Bahcall: “half of all three-sigma detections are false“. Of course if they used proper Bayesian language, scientists wouldn’t make so many nonsensical statements. Personally, I just don’t do sigmas.
My attention then switched to the CMS experiment. As a point of information you should be aware that CMS stands for Compact Muon Solenoid, where “compact” is a word used by particle physicists to mean “fucking enormous”. CMS makes pictures like this:
Anyway, it seems from the CMS part of the presentation that they find a bit of a peak at a similar mass ~ 125 GeV but spread out over a larger range, this time at a level of – sigh – 2.6 sigma.
All in all, it’s a definite maybe. Putting the results together in the way only a frequentist can the result is a 2.4 sigma detection. In other words, nothing any serious scientist would call convincing.
It’s interesting how certain these particle physicists are that the Higgs actually exists. It might, of course, and I think these results may be pointing the way to more convincing evidence based on more data. However, I still think we should bear in mind the words of Alfred North Whitehead:
There is no more common error than to assume that, because prolonged and accurate mathematical calculations have been made, the application of the result to some fact of nature is absolutely certain.
If there is a Higgs boson with a mass of 125 GeV then that would of course be an exciting discovery, but if there isn’t one at all wouldn’t that be even more exciting?
Final word from the Director of CERN:
We have not found it yet, we have not excluded it yet, stay tuned for next year.
Thunder and hail descended on Cardiff just as the webcast finished, which is clearly not a coincidence although I couldn’t say how many sigmas were involved.
And a final, final word from the Chief Executive of the Science & Technology Facilities Council, John Womersley:
There is still some way to go before the existence of the Higgs boson can be confirmed or not, but excitement is mounting. UK physicists and engineers have played a significant role in securing today’s results, and will continue to be at the forefront of exploring the new frontiers of knowledge opened by the results coming from the LHC. This is an incredibly exciting time to be involved in physics!
Brian Cox is 43.
Follow @telescoper
In the Dark 
December 13, 2011 at 3:04 pm
I must say, although I’m not qualified to comment on the scientific content, I found the analysis tremendously entertaining.
December 13, 2011 at 3:10 pm
I agree with you, of course, about the unseemliness of talking about sigmas. Even if one must be frequentist, it’s better just to talk about p-values or significances, not translate them into sigmas.
Having gotten that out of the way, let me ask this: what’s your source for the 1.9 sigma number? I don’t see it in the link you provided. I don’t think it’s possible that the combined result could be less significant than each of the two individual ones.
December 13, 2011 at 3:13 pm
Actually it was my mistake. The 1.9 sigma is the CMS excess below 130 GeV. The number that should have gone in that place is 2.4.
December 13, 2011 at 3:23 pm
ps. The 1.9 comes from the famous “look-elsewhere effect”
http://physics.aps.org/synopsis-for/10.1103/PhysRevLett.107.101801
I.e. f you want a meaningful statistical analysis, look elsewhere.
December 13, 2011 at 4:51 pm
I know it exists, I found it behind my fridge when I was cleaning just the other day. They won’t find it in Geneva though, coz I’ve got it. Maybe I should put it on eBay.
December 13, 2011 at 5:04 pm
Did you find any sigmas with it?
December 13, 2011 at 5:12 pm
The “look-elsewhere” effect is fine – it’s just counting the effective number of independent channels they search over. The evidence looks pretty good to me (nice plots at http://resonaances.blogspot.com/). Both experiments see excess events in the 125-GeV channel; ATLAS sees excesses in two distinct signatures, which makes the whole thing look especially convincing and consistent.
Overall, ATLAS quotes 3.6sigma (2.5 after LE correction), and CMS has 2.6 (1.9 after LE). Taking a naive approach (and why not…), one would add in quadrature to get 4.4 or 3.1. But the latter figure is too pessimistic, as it adds the LE correction twice. So something in the region of 3.7sigma must be about right. You can worry about non-Gaussian errors, but to most astronomers, that’s a definite detection. PP types insist on 5sigma, but that’s mainly to allow for counting issues like LE. So barring an unknown systematic (which is always a worry, however many sigma you have), this looks like a definite detection to me. It might be fun to try a Bayesian approach to this, but I doubt the formal significance will alter much.
In a way, this result is a bad thing for physics. If we ruled out the standard-model Higgs, that would be a clear statement. As it is, we’ve seen something that’s consistent with the SM Higgs – but giving out prizes will have to wait quite a while for it to be characterised. Most theorists seem to think a low-mass Higgs needs supersymmetry anyway, so presumably this could be some piece of non-SM physics that’s completely different from the vanilla Higgs.
December 13, 2011 at 8:47 pm
maybe astronomy should move to a 5-sigma limit? we would then be able to justify extending the life of all our experiments/telescopes/etc by ~3x.
December 13, 2011 at 5:14 pm
Five of them.
December 13, 2011 at 5:25 pm
Other nice plots at http://blog.vixra.org/category/large-hadron-collider/ especially the summary at the end which combines ATLAS, CMS, and Tevatron.
December 13, 2011 at 5:36 pm
Crystal clear where many have struggled. Thanks. Might be worth explaining your distaste for sigmas for the non-statos. (Spent some time today playing with trivial data using SDs and frequency representations. SDs much better at making things look smoother…Hmmm)
December 13, 2011 at 7:31 pm
To open up a different can of worms: am I the only person to be annoyed that the BBC describe Peter Higgs as an English scientist when he has lived and worked in Scotland for fifty years?
I presume that if the flukey coincident reported today turns out to be just the BBC will describe him as Scottish?
December 13, 2011 at 7:52 pm
Without question they got it from the Wikipedia entry for Higgs, which calls him that. It reminds me of how Andy Murray is British every year until he gets knocked out of Wimbledon, at which point he suddenly becomes Scottish.
December 13, 2011 at 9:52 pm
At least they didn’t say Higgs was a Belgian…
December 13, 2011 at 11:40 pm
‘It’s interesting how certain these particle physicists are that the Higgs actually exists.’ …
What in anything you heard today gives you the right to say that? It’s about as accurate as if I were to say ‘it’s interesting how certain these cosmologists are that dark energy actually exists’. I must have missed all those quotes to the effect of ‘We are certain the Higgs exists.’ Or did you mind-read?
Actually, the main advance of the last year has been the enormous range of things which particle physicists have shown *not* to exist. Experimentalists are very fond of a good exclusion result.
Now maybe there were a few people saying there are good theoretical reasons why something like the Higgs ought to exist. And that’s a reasonable thing to believe – though it shouldn’t make you particularly ‘interested’, in the sense of interested in a curious disease or psychological disorder. The motivation for a Higgs-like-object is several decades old and thoroughly mundane in particle physics terms compared to more recent theories – or, heaven forbid, compared to a lot of cosmological models.
December 14, 2011 at 9:23 am
I’m not certain dark energy exists, and have posted to that effect several times.
If you read the article you’ll see the point I was trying to make was that not finding the Higgs would in some ways be more interesting than finding it.
December 14, 2011 at 9:46 am
As Oscar Wilde said: “There is only one thing worse than not finding the Higgs, and that is…”
December 14, 2011 at 1:52 am
Yeah, I’m also surprised that Telescoper says we think we found the Higgs. We don’t! – despite headlines in newspapers.
The put-downs over frequentist statistical techniques are clever and amusing, but people should realize that there are many Bayesians among the particle types. CMS compared the results of frequentist and Bayesian calculations, for the sake of it, and found hardly any numerical difference. And as for p-values, the presentations today were full of them.
This post is very amusing, for sure, but also inaccurate. 😦
December 14, 2011 at 8:39 am
I can’t comment on the observations, and I’m not surprised that Bayesian and frequentist answers coincide *in this case*, but Peter’s put-downs of frequentist stats have the further merit of being correct.
December 14, 2011 at 9:22 am
Nowhere in this article do I say that scientists think they’ve found the Higgs. In fact there are several places where I say specifically that they do not and link to several articles containing accurate reporting of the real situation.
.
December 14, 2011 at 9:44 am
Peter! There’s a tiny dot just under the text of your last post immediately above. Is that the Higgs?
December 14, 2011 at 11:03 am
I knew it was here somewhere!
December 14, 2011 at 9:51 am
If it’s at 125GeV then it weighs about the same as an iodine atom.
December 14, 2011 at 2:21 pm
[…] here’s some more analysis. This article on “Higgs mania” from the excellent In The Dark blog starts with him being woken at 7am with hints on the radio that the Higgs would be announced […]
December 19, 2011 at 9:52 am
I’m normally a fan of this blog, but I have to say I don’t agree with the tone or content of this piece.
Of course I agree that the Bayesian approach is a mathematically rigourous way of quantifying statistical inferences. I also agree that, like it or not, ultimately this is the way that most scientists actually proceed – for instance, despite the impressive 6 sigma (that word again) result from OPERA, most physicists do not actually believe that the neutrinos were travelling faster than light. Effectively, they have a very strong theoretical prior against such a possibility.
The problem is – as you’ve pointed out on this blog yourself – that with Bayes’ theorem if you put garbage in you get garbage out. And quite often it is really hard to tell which theoretical priors are garbage and which are not. In such situation – and especially when you have a repeatable particle physics experiment – the frequentist approach to reporting experimental results seems to me to be the most honest approach. You put your data out there without making any judgement about the believability of any particular theoretical interpretation, and you let each theorist interpret the meaning of your results based on his own personal set of priors.
In most cases, as far as I can see, the frequentist approach and the Bayesian approach lead to the same conclusions. There are two important cases where they may differ. One example is an OPERA-like result: here the clear implication is our priors are wrong. Either neutrinos can travel faster than light, or there is some subtle physical effect in the experiment we do not yet understand. The frequentist style of reporting starkly highlights this.
The second case is more subtle and more disturbing. This occurs when marginal data (for example the BAO, or the ISW effect) are analysed in the context of a strong prior towards a particular model (in this case LCDM). If the strong prior leads to a strong claim in favour of the model, and this strong claim informs the priors used by the next group of scientists, you are in serious danger of confirmation bias. (And in fact http://arxiv.org/abs/1112.3108 seems to show that exactly such a confirmation bias might well exist in cosmology.)
Sometimes the Bayesian approach also seems to me to vastly inflate the importance of the data. Hence the – to my eyes astonishing – claim that, for instance, ISW data provide 4.5 sigma evidence for LCDM, while simultaneously still being consistent with no signal at all. Perhaps a frequentist approach to reporting results would better reflect the real quality of the data.
PS: I am a cosmologist, not a particle physicist.
December 19, 2011 at 9:57 am
Incidentally, I also think the snarky comment about the quote from John Bahcall is a bit out of place. Clearly it was a slightly flippant comment anyway, but if I translate
“half of all three-sigma detections are false“
as
“half of all detections that have a less than 1% likelihood of being due to random chance do not require a modification of fundamental theory after all”
then it makes perfect sense, even though we’re talking in terms of probabilities, not sigmas. The point being that by the time you’ve got enough data to show a 5 sigma result, you probably understand the systematics of your detector a whole lot better.
December 19, 2011 at 10:35 am
You’re entitled to your opinion about the piece of course. All I can say is that I’m allergic to hype which, notwithstanding the clear hint that a detection might be on the way, is what all this press furore was. I don’t think scientists should play this kind of game.
The main problem with frequentist methods is that they are fundamentally ill-conceived. In particular they do not actually answer the question a physicist wants to ask, e.g. how strong is the evidence of the existence of a new particle in these data? The frequentist answer is to a quite different question: how likely are these data to have arisen by chance in a model without a new particle?
John Bahcall’s comment is not as flippant as you suggest, because there’s no certain connection between the frequentist p-value (or its correspondong sigma) and the probability that a particular alternative model is actually correct. As I’ve said many times before on here P(M|D) is not the same as P(D|M)!
The bottom line is that the Bayesian framework is the only one that provides a guarantee of consistent reasoning (as long as it is done correctly). I think some people dislike it precisely for that reason,because it sometimes gives answers they don’t like, perhaps because they are less easy to hype?
December 19, 2011 at 10:57 am
ps. The additional problem with using sigmas in public is that 2.5 sounds like it’s close to 3.
December 19, 2011 at 11:29 am
I completely agree with you about hype. I rather thought my point was that in a Bayesian framework, the (perhaps unjustified) strength of the prior you apply may lead you to a conclusion which is over-hyped.
As far as I can tell, asking the question in the frequentist way (“how likely is it that I should see a signal when there actually isn’t a signal to be seen”) is precisely the best way to avoid hype.
Once we are sure a signal has in fact been seen, then a Bayesian analysis can tell us which model it favours (among those that produce signals). Until then, as I thought the scientists at CERN were rather keen to stress, don’t jump the gun.
December 19, 2011 at 5:00 pm
There are enough data to wash out any prior that is not unreasonably narrow; the issue is merely how to phrase the findings.
As for the superluminal neutrinos, the prior – and therefore the posterior – does not and indeed cannot into account an unknown but systematic source of error, as distinct from variables that are uncontrolled and differ slightly from run to run (‘random error’). That is what sceptics of superluminality believe is going on. Going frequentist cannot help here (or anywhere).
December 20, 2011 at 10:05 am
With reference to the Higgs, there clearly isn’t enough data yet to wash out any prior. That’s precisely the point.
Equally, with the BAO and ISW observations there isn’t enough data to wash out the effect of the prior. A “frequentist phrasing” would (and does, where you can find it) make this obvious, a “Bayesian phrasing” does not.
Hence my point that the frequentist phrasing is a more honest reflection of the actual state of the data. More honesty = less hype.
December 20, 2011 at 10:11 am
I find this comment entirely incomprehensible.
December 20, 2011 at 11:36 am
Sesh: How many actual collisions go to make up the observed discrepancy at 125GeV? If it’s a large number – which I don’t know, as I’m not a particle physicist either – then I’d stand by my comment about the prior being washed out.
We simply disagree about whether Bayesian or frequentist phrasing is clearer. I don’t particularly want to go further into that in a post about the Higgs.
December 20, 2011 at 11:55 am
Anton,
The numbers are different in the ATLAS and CMS experiments but we’re talking about tens, rather than hundreds, of events…
Peter
December 20, 2011 at 10:40 am
Sorry you find the comment incomprehensible Peter. I find several things incomprehensible too. Perhaps you could explain to me how http://arxiv.org/abs/arXiv:0801.4380 finds 4.5 sigma evidence for LCDM in the ISW data (Table 1), when the data is not even good enough to reject the null hypothesis (no ISW signal – Table 2)?
If you asked the question of whether ISW observations had found evidence for dark energy and you phrased it in the same manner as a particle physicist would, you would undoubtedly conclude “we have not found it yet, we have not excluded it yet”. I don’t see how anyone could possibly claim that this approach results in “hype” – it seems precisely the opposite to me.
Your evidence for “hype” in this post appears to be entirely based on a headline in the Daily Mail. Everything actual particle physicists are saying is explicitly designed to avoid hype.
December 20, 2011 at 11:11 am
I’m not an author on that paper, so don’t see why you’re holding me to account for the claims within it. It’s not actually relevant to this post either.
I included sensible statements from particle physicists precisely in order to make it clear that the media hype was at odds with what the scientists are saying.
December 20, 2011 at 11:17 am
Well, my point was simply that I thought you’d have more trouble with the way cosmologists treat data than with the way particle physicists do. But clearly there’s not much point continuing such a discussion over the internet, so I’ll just go back to enjoying the other parts of your blog!
December 20, 2011 at 1:51 pm
[…] the media frenzy abates after the latest experimental results from the Large Hadron Collider show tantalising but […]
December 20, 2011 at 8:50 pm
Here’s the Swansea Evening Post on the Higgs hype:
http://www.thisissouthwales.co.uk/Huge-buzz-work-hadron-collider/story-14132892-detail/story.html
“They have narrowed down the window of where the Hicks could be.”
July 3, 2012 at 8:45 am
[…] wrote a rather facetious post about the last episode of Higgs-mania way back in December because I found the actual announcement […]
July 8, 2012 at 8:39 am
[…] here’s some more analysis. This article on “Higgs mania” from the excellent In The Dark blog starts with him being woken at 7am with hints on the radio that the Higgs would be announced […]
October 23, 2012 at 5:45 pm
[…] Higgs-mania Day: I woke up this morning to the BBC Radio News at 7am announcing that scientists at CERN were going to report “hints” of the discovery of the Higgs Boson at the Large Hadron Collider (link) […]
November 12, 2012 at 2:37 pm
[…] else is this a big deal? Take a look at the discussions about whether the God particle was indeed detected at the Large Hadron Collider […]