Archive for the Education Category

The Worthless University Rankings

Posted in Bad Statistics, Education with tags , , , on September 23, 2016 by telescoper

The Times Higher World University Rankings, which were released this weekk. The main table can be found here and the methodology used to concoct them here.

Here I wish to reiterate the objection I made last year to the way these tables are manipulated year on year to create an artificial “churn” that renders them unreliable and impossible to interpret in an objective way. In other words, they’re worthless. This year, editor Phil Baty has written an article entitled Standing still is not an option in which he makes a statement that “the overall rankings methodology is the same as last year”. Actually it isn’t. In the page on methodology you will find this:

In 2015-16, we excluded papers with more than 1,000 authors because they were having a disproportionate impact on the citation scores of a small number of universities. This year, we have designed a method for reincorporating these papers. Working with Elsevier, we have developed a new fractional counting approach that ensures that all universities where academics are authors of these papers will receive at least 5 per cent of the value of the paper, and where those that provide the most contributors to the paper receive a proportionately larger contribution.

So the methodology just isn’t “the same as last year”. In fact every year that I’ve seen these rankings there’s been some change in methodology. The change above at least attempts to improve on the absurd decision taken last year to eliminate from the citation count any papers arising from large collaborations. In my view, membership of large world-wide collaborations is in itself an indicator of international research excellence, and such papers should if anything be given greater not lesser weight. But whether you agree with the motivation for the change or not is beside the point.

The real question is how can we be sure that any change in league table position for an institution from year to year are is caused by methodological tweaks rather than changes in “performance”, i.e. not by changes in the metrics but by changes in the way they are combined? Would you trust the outcome of a medical trial in which the response of two groups of patients (e.g. one given medication and the other placebo) were assessed with two different measurement techniques?

There is an obvious and easy way to test for the size of this effect, which is to construct a parallel set of league tables, with this year’s input data but last year’s methodology, which would make it easy to isolate changes in methodology from changes in the performance indicators. The Times Higher – along with other purveyors of similar statistical twaddle – refuses to do this. No scientifically literate person would accept the result of this kind of study unless the systematic effects can be shown to be under control. There is a very easy way for the Times Higher to address this question: all they need to do is publish a set of league tables using, say, the 2015/16 methodology and the 2016/17 data, for comparison with those constructed using this year’s methodology on the 2016/17 data. Any differences between these two tables will give a clear indication of the reliability (or otherwise) of the rankings.

I challenged the Times Higher to do this last year, and they refused. You can draw your own conclusions about why.

Rank Nonsense

Posted in Bad Statistics, Education, Politics with tags , , , , , on September 8, 2016 by telescoper

It’s that time of year when international league tables (also known as “World Rankings”)  appear. We’ve already had the QS World University Rankings and the Shanghai (ARWU) World University Rankings. These will soon be joined by the Times Higher World Rankings, due out on 21st September.

A lot of people who should know a lot better give these league tables far too much attention. As far as I’m concerned they are all constructed using extremely suspect methodologies whose main function is to amplify small statistical variations into something that looks significant enough to justify constructing  a narrative about it. The resulting press coverage usually better reflects a preconceived idea in a journalist’s head than any sensible reading of the tables themselves.

A particularly egregious example of this kind of nonsense can be found in this week’s Guardian. The offending article is entitled “UK universities tumble in world rankings amid Brexit concerns”. Now I make no secret of the fact that I voted “Remain” and that I do think BrExit (if it actually happens) will damage UK universities (as well as everything else in the UK). However, linking the changes in the QS rankings to BrExit is evidently ridiculous: all the data were collected before the referendum on 23rd June anyway! In my opinion there are enough good arguments against BrExit without trying to concoct daft ones.

In any case these tables do not come with any estimate of the likely statistical variation from year to year in the metrics used to construct them, which makes changes impossible to interpret. If only the compilers of these tables would put error bars on the results! Interestingly, my former employer, the University of Sussex, has held its place exactly in the QS rankings between 2015 and 2016: it was ranked 187th in the world in both years. However, the actual score corresponding to these two years was 55.6 in 2015 and 48.4 in 2016. Moreover, Cambridge University fell from 3rd to 4th place this year but its score only changed from 98.6 to 97.2. I very much doubt that is significant at all, but it’s mentioned prominently in the subheading of the Guardian piece:

Uncertainty over research funding and immigration rules blamed for decline, as Cambridge slips out of top three for first time.

Actually, looking closer, I find that Cambridge was joint 3rd in 2015 and is 4th this year. Over-interpretation, or what?

To end with, I can’t resist mentioning that the University of Sussex is in the top 150 in the Shanghai Rankings for Natural and Mathematical Sciences this year, having not been in the top 200 last year. This stunning improvement happened while I was Head of School for Mathematical and Physical Sciences so it clearly can not be any kind of statistical fluke but is entirely attributable to excellent leadership. Thank you for your applause.

 

 

Worrying Times for UK Physics

Posted in Education, The Universe and Stuff with tags , on August 19, 2016 by telescoper

As I’m more-or-less in between jobs at the moment, this is the first August in many that I haven’t been involved the clearing and confirmation process that helps students find places at university after the A-level results are released. I know how stressful it is for admissions staff and prospective students alike, so I’m not sorry to be out of it for once!

On the other hand I did notice something worrying that seems to be the continuation of a trend I noticed last year.  I quote from a piece issued by the Institute of Physics about the number of students taking A-level physics last year:

Although there was an overall rise of 2% in the number of A-level entries, the number taking physics fell to 36,287 compared with 36,701 last year – the first time numbers have fallen since 2006. The number of girls taking physics rose by 0.5%, however.

That decline is slight, of course, and it was  obviously too early to decide whether it indicated whether or not the UK has reached “Peak Physics”. Well, this year has confirmed that trend. According to a piece by the Wellcome Trust the number of entrants for physics A-level has fallen further this year, from 36,287 in 2015 to 35,344 in 2016. The Institute of Physics has also commented.

Virtually all students who get a Physics A-level do go to university, but by no means all do physics. It is also a qualifying subject for engineering and technology programmes, as well as medicine. It’s not clear yet whether the decline in A-level entry reflects a decline in the number of students going to start physics degrees at University this year, but this seems probable. This is good news if you’re an applicant with a Physics A-level, of course, because it increases the chances of you getting a place, but it’s no so good for physics as academic discipline.

Physics departments in UK universities are already competing for a very small pool of students with a Physics A-level.  The removal of student number controls allows  large universities to recruit as many students as they like, so the competition between universities for such a small number of applicants is extremely intense. Moreover, some universities, e.g. Newcastle and Hull, have opened up physics courses that they had previously closed, and others have started  new programmes based on what was anticipated to be an overall increase in demand. To support this expansion, many institutions have recruited extra numbers of teaching faculty assuming the salary costs would be covered from tuition fees. If the decline in overall student numbers continues then the budgets of many physics departments are going to look pretty grim, with potentially serious  consequences for the long-term sustainability of physics in many institutions.

I have to confess I’m worried. The physics community urgently needs to find out what is behind this fall. It’s not restricted to physics, in fact. Both biology and chemistry have also experienced a decline in the number of A-level entrants (from 44,864 to 43,242 and from 52,644 to 51,811 respectively), but the effect on physics is likely to be greater for the reasons I discussed above.

Mathematics numbers have also fallen, but by a much smaller percentage and from a much higher level: from 92,711 to 92,163.  I‘ve argued before that there’s a case on a number of grounds for scrapping the physics A-level as a requirement for entry to university as long as the student has mathematics. That may be a step too far for some, but it’s clear that if physics is to prosper we all have to think more creatively about how to increase participation. But how? Answers on a postcard – or through the comments box – please!

 

 

Clearing Advice for Physics and Astronomy Applicants!

Posted in Education with tags , , , , on August 18, 2016 by telescoper

Today’s the day! This year’s A-level results are out today, Thursday 18th August, with the consequent scramble as students across the country to confirm places at university. Good luck to all students everywhere waiting for your results. I hope they are what you expected!

For those of you who didn’t get the grades they needed, I have one piece of very clear advice:

1-dont-panic

The clearing system is very efficient and effective, as well as being quite straightforward to use, and there’s still every chance that you will find a place somewhere good. So keep a cool head and follow the instructions. You won’t have to make a decision straight away, and there’s plenty of time to explore all the options.

As a matter of fact there are a few places still left for various courses in the School of Physics & Astronomy at Cardiff University. Why should you choose Cardiff? Well, obviously I have a vested interest since I’m rejoining the University this September so I’m biased. However you could take into account that Physics & Astronomy at Cardiff is top of the Russell Group in the latest National Student Survey and that there are wonderful newly expanded and refurbished teaching spaces on site.

For further information check here!

Universities must do more to stop violence

Posted in Education, Uncategorized with tags , on August 15, 2016 by telescoper

I’ve thought very hard over the last couple of days about whether to comment on the shocking case reported by the Independent last week of a (male) senior lecturer (Dr Lee Salter) at Sussex University who beat up a (female) student with whom he had been having an affair. In the end I decided that I had to comment, as the case raises some very important questions.

I didn’t know anything about this until last week so I have nothing to add to the account of the events and subsequent criminal conviction given in the newspaper and suggest you read the details there. I will restrict my comments to the wider issues.

On Friday 12th August, shortly after the news broke of Lee Salter’s conviction, the University released a statement which I thought raised more questions than it answered. It  subsequently updated the statement to say that Dr Salter was no longer an employee of the University. Whether that means he was dismissed or that he resigned is not clear.

Among the statements made by the University in its press release is the following:

The University does not tolerate violence of any kind. However, in cases involving criminal charges, it is important that such matters are dealt with by the police and the courts, which take precedence over employment procedures. Pending the outcome of the criminal proceedings, the University kept the situation under review and monitored and assessed any risk to its students.

In my role as a Head of School at Sussex (a job I left just a couple of weeks ago), I had to deal with some disciplinary matters  so I’m very familiar with the content of the relevant procedures. In fact I did more of these than you’d probably imagine, though I can’t write about the details because they are bound by confidentiality.

It is indeed the case that if a disciplinary case involves criminal elements then the established practice is to let the courts decide first before continuing with the disciplinary investigation. For one thing, a conviction in a criminal case usually makes the subsequent internal investigation simpler.

Acquittal in a criminal case does not mean dropping the disciplinary, however, as the standard of proof in a criminal case (“beyond reasonable doubt”) is stronger than that of an internal investigation which is that of a civil court (“on the balance of the evidence”). It is quite possible for the latter standard to be met when the former is not. So it was reasonable for the University to wait for the outcome of the criminal trial before proceeding.

However, the University of Sussex’s own disciplinary procedure also states:

“The University will take disciplinary action in accordance with its procedures against anyone who behaves in a violent manner including, should it be necessary, the immediate exclusion of the perpetrator from the campus.

Based on the account given in the Independent I find it difficult to understand why the University did not take this course of action in this case.

Of course a suspect is innocent until proven guilty, but suspension (paid) and exclusion from campus would not, in my view, have been unnecessarily prejudicial given the seriousness of the charges. Salter would not have been able to do teaching, but could have carried on research from home. The University’s failure to take this step is extremely worrying as in my view it gives inadequate consideration to the effect on the victim of the continued presence of the perpetrator.

For the record I should state that I have very good reasons for having zero tolerance to any form of violence, whether committed by staff or students or political protestors or security guards. You can read why here.

I’ve blogged before about the difficulties surrounding confidentiality and other issues disciplinary procedures in the context of sexual harassment. In that piece – which was actually about science departments – I tried to stress the importance of sticking to proper procedure, but I also explained that dealing with such matters after the fact is never going to provide a fully satisfactory remedy. What is needed is to change campus culture to ensure that abusive harassing and violent behaviour doesn’t happen in the first place. But applying procedures properly would at least be a start…

 

 

 

 

The Rising Stars of Sussex Physics

Posted in Bad Statistics, Biographical, Education with tags , , , , on July 28, 2016 by telescoper

This is my penultimate day in the office in the School of Mathematical and Physical Sciences at the University of Sussex, and a bit of news has arrived that seems a nice way to round off my stint as Head of School.

It seems that Physics & Astronomy research at the University of Sussex has been ranked as 13th in western Europe and 7th in the UK by leading academic publishers, Nature Research, and has been profiled as one of its top-25 “rising stars” worldwide.

I was tempted to describe this rise as ‘meteoric’ but in my experience meteors generally fall down rather than rise up.

Anyway, as regular readers of this blog will know, I’m generally very sceptical of the value of league tables and there’s no reason to treat this one as qualitatively any different. Here is an explanation of the (rather curious) methodology from the University of Sussex news item:

The Nature Index 2016 Rising Stars supplement identifies the countries and institutions showing the most significant growth in high-quality research publications, using the Nature Index, which tracks the research of more than 8,000 global institutions – described as “players to watch”.

The top 100 most improved institutions in the index between 2012 and 2015 are ranked by the increase in their contribution to 68 high-quality journals. From this top 100, the supplement profiles 25 rising stars – one of which is Sussex – that are already making their mark, and have the potential to shine in coming decades.

The institutions and countries examined have increased their contribution to a selection of top natural science journals — a metric known as weighted fractional count (WFC) — from 2012 to 2015.

Mainly thanks to a quadrupling of its physical sciences score, Sussex reached 351 in the Global 500 in 2015. That represents an 83.9% rise in its contribution to index papers since 2012 — the biggest jump of any UK research organisation in the top 100 most improved institutions.

It’s certainly a strange choice of metric, as it only involves publications in “high quality” journals, presumably selected by Journal Impact Factor or some other arbitrary statistical abominatio,  then taking the difference in this measure between 2012 and 2015  and expressing the change as a percentage. I noticed one institution in the list has improved by over 4600%, which makes Sussex’s change of 83.9% seem rather insignificant…

But at least this table provides some sort of evidence that the investment made in Physics & Astronomy over the last few years has made a significant (and positive) difference. The number of research faculty in Physics & Astronomy has increased by more than 60%  since 2012 so one would have been surprised not to have seen an increase in publication output over the same period. On the other hand, it seems likely that many of the high-impact papers published since 2012 were written by researchers who arrived well before then because Physics research is often a slow burner. The full impact of the most recent investments has probably not yet been felt. I’m therefore confident that Physics at Sussex has a very exciting future in store as its rising stars look set to rise still further! It’s nice to be going out on a high note!

 

 

Private Eye on Physics Graduation

Posted in Education, The Universe and Stuff with tags , , on July 19, 2016 by telescoper

Given the occasion I thought I’d just post this rather excellent cartoon I saw last year  Private Eye

Physics Graduation