Six Years In The Dark

Posted in Biographical with tags on September 16, 2014 by telescoper

When I logged onto WordPress to write yesterday’s post I received a message that it was the 6th anniversary of my registration with them as a blogger and thus took my first step into the blogosphere; that was way back on 15th September 2008. I actually wrote my first post that day too. Here it is, in all its glory:

So here we are then. I’ve finally decided to start writing a blog. I’ve been reading quite a few of them recently and most appeared to consist of a load of ill-informed opinionated drivel. So I thought “I can do that!”. And here we are.

I don’t know who (if anyone) will be reading this or even what I’m going to write, but let’s see how it goes until everyone concerned gets bored with it.

And before I start, I’d like to thank Phil Brown from the British Association for the Advancement of Science for inviting me to set this up. I never would have got around to it if he hadn’t done so.

So blame him!

Unfortunately I didn’t really know what I was doing on my first day at blogging – no change there then –  and I didn’t actually manage to figure out how to publish this earth-shattering piece. It was only after I’d written my second post that I realized that the first one wasn’t actually live, so the two appear in the wrong order in my archive.

I’d like to take this opportunity to send my best wishes, and to thank, everyone who reads this blog, however occasionally. According to the WordPress stats, I’ve got readers from all round the world, including one in the Vatican! If you’re interested in statistics then, as of 9.15 this morning, I have published 2537 blog posts in all, and have received 2,032,090 hits altogether; I get an average of about 1300 per day, but this varies in a very erratic fashion. There have been 20,201 comments published on here and 857,904 rejected as spam or abuse; a lot goes on behind the scenes that you don’t want to know about!

Anyway, the numbers don’t really matter but it does mean a lot to know that there are people who find my ramblings interesting enough to look at, and sometimes even to come back for more! This blog is read by a number of powerful and influential people too, as well as John Womersley….

 

 

 

Frequentism: the art of probably answering the wrong question

Posted in Bad Statistics with tags , , , , , , on September 15, 2014 by telescoper

Popped into the office for a spot of lunch in between induction events and discovered that Jon Butterworth has posted an item on his Grauniad blog about how particle physicists use statistics, and the ‘5σ rule’ that is usually employed as a criterion for the detection of, e.g. a new particle. I couldn’t resist bashing out a quick reply, because I believe that actually the fundamental issue is not whether you choose 3σ or 5σ or 27σ but what these statistics mean or don’t mean.

As was the case with a Nature piece I blogged about some time ago, Jon’s article focuses on the p-value, a frequentist concept that corresponds to the probability of obtaining a value at least as large as that obtained for a test statistic under a particular null hypothesis. To give an example, the null hypothesis might be that two variates are uncorrelated; the test statistic might be the sample correlation coefficient r obtained from a set of bivariate data. If the data were uncorrelated then r would have a known probability distribution, and if the value measured from the sample were such that its numerical value would be exceeded with a probability of 0.05 then the p-value (or significance level) is 0.05. This is usually called a ‘2σ’ result because for Gaussian statistics a variable has a probability of 95% of lying within 2σ of the mean value.

Anyway, whatever the null hypothesis happens to be, you can see that the way a frequentist would proceed would be to calculate what the distribution of measurements would be if it were true. If the actual measurement is deemed to be unlikely (say that it is so high that only 1% of measurements would turn out that big under the null hypothesis) then you reject the null, in this case with a “level of significance” of 1%. If you don’t reject it then you tacitly accept it unless and until another experiment does persuade you to shift your allegiance.

But the p-value merely specifies the probability that you would reject the null-hypothesis if it were correct. This is what you would call making a Type I error. It says nothing at all about the probability that the null hypothesis is actually a correct description of the data. To make that sort of statement you would need to specify an alternative distribution, calculate the distribution based on it, and hence determine the statistical power of the test, i.e. the probability that you would actually reject the null hypothesis when it is correct. To fail to reject the null hypothesis when it’s actually incorrect is to make a Type II error.

If all this stuff about p-values, significance, power and Type I and Type II errors seems a bit bizarre, I think that’s because it is. It’s so bizarre, in fact, that I think most people who quote p-values have absolutely no idea what they really mean. Jon’s piece demonstrates that he does, so this is not meant as a personal criticism, but it is a pervasive problem that results quoted in such a way are intrinsically confusing.

The Nature story mentioned above argues that in fact that results quoted with a p-value of 0.05 turn out to be wrong about 25% of the time. There are a number of reasons why this could be the case, including that the p-value is being calculated incorrectly, perhaps because some assumption or other turns out not to be true; a widespread example is assuming that the variates concerned are normally distributed. Unquestioning application of off-the-shelf statistical methods in inappropriate situations is a serious problem in many disciplines, but is particularly prevalent in the social sciences when samples are typically rather small.

While I agree with the Nature piece that there’s a problem, I don’t agree with the suggestion that it can be solved simply by choosing stricter criteria, i.e. a p-value of 0.005 rather than 0.05 or, in the case of particle physics, a 5σ standard (which translates to about 0.000001!  While it is true that this would throw out a lot of flaky ‘two-sigma’ results, it doesn’t alter the basic problem which is that the frequentist approach to hypothesis testing is intrinsically confusing compared to the logically clearer Bayesian approach. In particular, most of the time the p-value is an answer to a question which is quite different from that which a scientist would actually want to ask, which is what the data have to say about the probability of a specific hypothesis being true or sometimes whether the data imply one hypothesis more strongly than another. I’ve banged on about Bayesian methods quite enough on this blog so I won’t repeat the arguments here, except that such approaches focus on the probability of a hypothesis being right given the data, rather than on properties that the data might have given the hypothesis.

I feel so strongly about this that if I had my way I’d ban p-values altogether…

Not that it’s always easy to implement a Bayesian approach. It’s especially difficult when the data are affected by complicated noise statistics and selection effects, and/or when it is difficult to formulate a hypothesis test rigorously because one does not have a clear alternative hypothesis in mind. Experimentalists (including experimental particle physicists) seem to prefer to accept the limitations of the frequentist approach than tackle the admittedly very challenging problems of going Bayesian. In fact in my experience it seems that those scientists who approach data from a theoretical perspective are almost exclusively Baysian, while those of an experimental or observational bent stick to their frequentist guns.

Coincidentally a paper on the arXiv not long ago discussed an interesting apparent paradox in hypothesis testing that arises in the context of high energy physics, which I thought I’d share here. Here is the abstract:

The Jeffreys-Lindley paradox displays how the use of a p-value (or number of standard deviations z) in a frequentist hypothesis test can lead to inferences that are radically different from those of a Bayesian hypothesis test in the form advocated by Harold Jeffreys in the 1930’s and common today. The setting is the test of a point null (such as the Standard Model of elementary particle physics) versus a composite alternative (such as the Standard Model plus a new force of nature with unknown strength). The p-value, as well as the ratio of the likelihood under the null to the maximized likelihood under the alternative, can both strongly disfavor the null, while the Bayesian posterior probability for the null can be arbitrarily large. The professional statistics literature has many impassioned comments on the paradox, yet there is no consensus either on its relevance to scientific communication or on the correct resolution. I believe that the paradox is quite relevant to frontier research in high energy physics, where the model assumptions can evidently be quite different from those in other sciences. This paper is an attempt to explain the situation to both physicists and statisticians, in hopes that further progress can be made.

This paradox isn’t a paradox at all; the different approaches give different answers because they ask different questions. Both could be right, but I firmly believe that one of them answers the wrong question.

Parliamo Glasgow

Posted in Politics, Television with tags , , , on September 15, 2014 by telescoper

Whatever the outcome of Thursday’s referendum on independence, it’s clear that we who live South of the Border need to try harder to understand the Scots much better than we have so far. Because Glasgow and its environs appear to be hotbeds of the pro-secession vote, I can think of no better way to begin this process than by learning more about Glaswegian and the way what she is spoke, in order to improve mutual respect and foster dialogue.

Here are some useful lessons based on everyday situations and characters, as portrayed by the inimitable Stanley Baxter…

 

 

The Arrival of the Queen of Sheba

Posted in Biographical, Music with tags , , , on September 14, 2014 by telescoper

It’s been such a hectic week getting ready for our new arrivals at the University of Sussex that I’ve been largely limited to posting short items and recycled material. Today is no exception either, as I have been on campus again for another Freshers’ induction week event and now have to prepare a talk for new students in the School of Mathematical and Physical Sciences for tomorrow morning. Anyway, all these busy preparations made me think today of the famous instrumental passage from Act III of Handel’s Oratorio Solomon which depict in wonderfully lively fashion  similar preparations preceding the Arrival of the Queen of Sheba. I know that’s a feeble pretext for posting a bit of music, but I thought I’d at least put a little twist on it by including a performance rather different from what you might be expecting.

This version of The Arrival of the Queen of Sheba is by wonderful Welsh harpist Glenda Clwyd whom I remember hearing play at a couple of events when I was living in Cardiff. This clip is of an impromptu performance and there are a couple of small mistakes, but I think it’s a lovely rendition, the uniquely gentle sound of the harp making it less frantic than most versions.

 

 

Freshers’ Week Reminiscences

Posted in Biographical, Education with tags , , , , on September 13, 2014 by telescoper

So here I am again, on campus, on a Saturday, this time to attend some receptions for new students (“Freshers”) who have just arrived at the University of Sussex to start their courses. I always enjoy meeting the new intake at this time of year; we sometimes call them “The Autumn Collection”, although it’s only mid-September and definitely not autumn yet. In fact it’s very warm and sunny and summery on Falmer campus today. The  downside of these annual events is that the students look much younger every year, so every one makes me feel a lot older than the one before!

Looking through my back catalogue of blog posts I realize that this blog is six years old next week. One of my first blog posts was about  memories of my own first day at University and it seems appropriate to repeat some of it here. I notice actually that virtually all Freshers’ weeks I’ve written about over the past six years have been accompanied by fine weather. I find this kind of weather a bit spooky because it always takes me back to the time when I left home to go to University, as thousands of fledgling students are about to do this year in their turn. I did it 32 years ago, getting on a train at Newcastle Central station with my bags of books and clothes. I said goodbye to my parents there. There was never any question of them taking me in the car all the way to Cambridge. It wasn’t practical and I wouldn’t have wanted them to do it anyway. After changing from the Inter City at Peterborough onto a local train, we trundled through the flatness of East Anglia until it reached Cambridge. The weather, at least in my memory, was exactly like today.

I don’t remember much about the actual journey, but I must have felt a mixture of fear and excitement. Nobody in my family had ever been to University before, let alone to Cambridge. Come to think of it, nobody from my family has done so since either. I was a bit worried about whether the course I would take in Natural Sciences would turn out to be difficult, but I think my main concern was how I would fit in generally.

I had been working between leaving school and starting my undergraduate course, so I had some money in the bank and I was also to receive a full grant. I wasn’t really worried about cash. But I hadn’t come from a posh family and didn’t really know the form. I didn’t have much experience of life outside the North East either. I’d been to London only once before going to Cambridge, and had never been abroad.

I didn’t have any posh clothes, a deficiency I thought would mark me as an outsider. I had always been grateful for having to wear a school uniform (which was bought with vouchers from the Council) because it meant that I dressed the same as the other kids at School, most of whom came from much wealthier families. But this turned out not to matter at all. Regardless of their family background, students were generally a mixture of shabby and fashionable, like they are today. Physics students in particular didn’t even bother with the fashionable bit. Although I didn’t have a proper dinner jacket for the Matriculation Dinner, held for all the new undergraduates, nobody said anything about my dark suit which I was told would be acceptable as long as it was a “lounge suit” (whatever that is).

Taking a taxi from the station, I finally arrived at Magdalene College. I waited outside, a bundle of nerves, for some time before entering the Porter’s Lodge and starting my life as a student. My name was found and ticked off and a key issued for my room in the Lutyens building. It turned out to be a large room, with a kind of screen that could be pulled across to divide the room into two, although I never actually used this contraption. There was a single bed and a kind of cupboard containing a sink and a mirror in the bit that could be hidden by the screen. The rest of the room contained a sofa, a table, a desk, and various chairs, all of them quite old but solidly made. Outside my  room, on the landing, was the gyp room, a kind of small kitchen, where I was to make countless cups of tea over the following months, although I never actually cooked anything there.

I struggled in with my bags and sat on the bed. It wasn’t at all like I had imagined. I realized that no amount of imagining would ever really have prepared me for what was going to happen at University.

I  stared at my luggage. I suddenly felt like I had landed on a strange island where I didn’t know anyone, and couldn’t remember why I had gone there or what I was supposed to be doing. I’ve had that feeling ever since, but after 32 years I think I’m used to it.

Say no to the commercialization of education!

Posted in The Universe and Stuff with tags , on September 12, 2014 by telescoper

There is much complaint these days about the alleged  commercialization of UK Higher Education, so I  wanted to take this opportunity to state Virgin Airlines that I will not be taking this as a Carling cue to introduce any form of commercial Coca Cola sponsorship of any Corby Trouser Press form into the School of Mathematical Macdonalds and Panasonic Physical Sciences, and certainly not into this Burger King blog.

This week I’ve been working hard preparing for the new Marks and Spencer term  and especially for the arrival of our new  Samsung students who will be starting their  Dixons degrees next week.  The Nokia preparations have gone pretty well  although I have had Betfair trouble cramming all the Sainsbury things I’ve had to do this BMW week, so I’ll be in Tesco tomorrow and Wonga Sunday to finish off a few Pizza Express jobs, but at least I’ll be able to attend the Vodafone Vice-Chancellor’s receptions for new students on the Carlsberg campus this Waitrose weekend.

In between these Ericsson events I hope to find some time to write a little more Morrisons of the second edition of my book on cosmology, including stuff about the Carphone Warhouse cosmic microwave background (CMB) which produces some of the noise on a Sony television screen, some of which  Classic FM signal came from the edge of the Next Universe.  The CMB plays an Emirates important role in TK Maxx cosmology as it is the Marlboro smoking gun of the Sainsbury Big Bang and established our Standard Life model of the L’Oreal Universe. The old British Airways edition is a bit out of Aviva date so I will be updating it with Starbucks references to  the Planck First Direct results, although I obviously haven’t decided yet what to say about Barclays BICEP2.  I think I’ll be adding a Goodfella’s Pizza paragraph or two referring to the House of Fraser Hubble Crown Paints  Ultra Deep Kentucky Fried Chicken Field as well.

Anyway, for now its  Thank God It’s Friday time to go HSBC home and drink several Dorothy Perkins glasses of Amazon wine.

Comet Sale Now On!

 

September Song – the Django Reinhardt version

Posted in Jazz with tags , , on September 11, 2014 by telescoper

Summer’s drawing to a close and preparing for the imminent arrival of new intake of students is taking up a lot of my time this week, so I thought I’d just put up something I’ve posted before, in the form of a piece of music that celebrates the genius of Django Reinhardt, the great Belgian-born gypsy guitarist who overcame the terrible  injuries he suffered as a child (in a fire in his caravan) to become one of the greatest jazz musicians of all time.  He had a unique style of playing the guitar he invented himself to get around the fact that the third and fourth digits on his left hand were so badly burned he could effectively only use two fingers. He also had an unparalleled gift for melodic improvisation that won him admirers all around the world and across all styles of music. Add him to your list of famous Belgians right away, for he was most certainly a musical genius.

Here he’s playing the beautifully poignant September Song, by Kurt Weill:

Oh, it’s a long, long while from May to December
But the days grow short when you reach September

 

Follow

Get every new post delivered to your Inbox.

Join 3,588 other followers