Archive for January 18, 2013

REF moves the goalposts (again)

Posted in Bad Statistics, Education, Science Politics with tags , , , on January 18, 2013 by telescoper

The topic of the dreaded 2014 Research Excellence Framework came up quite a few times in quite a few different contexts over the last few days, which reminded me that I should comment on a news item that appeared a week or so ago.

As you may or may not be aware, the REF is meant to assess the excellence of university departments in various disciplines and distribute its “QR” research funding accordingly.  Institutions complete submissions which include details of relevant publications etc and then a panel sits in judgement. I’ve already blogged of all this: the panels clearly won’t have time to read every paper submitted in any detail at all, so the outcome is likely to be highly subjective. Moreover, HEFCE’s insane policy to award the bulk of its research funds to only the very highest grade (4* – “internationally excellent”) means that small variations in judged quality will turn into enormous discrepancies in the level of research funding. The whole thing is madness, but there seems no way to inject sanity into the process as the deadline for submissions remorselessly approaches.

Now another wrinkle has appeared on the already furrowed brows of those preparing REF submissions. The system allows departments to select staff to be entered; it’s not necessary for everyone to go in. Indeed if only the very best researchers are entered then the typical score for the department will be high, so it will appear  higher up  in the league tables, and since the cash goes primarily to the top dogs then this might produce almost as much money as including a few less highly rated researchers.

On the other hand, this is a slightly dangerous strategy because it presupposes that one can predict which researchers and what research will be awarded the highest grade. A department will come a cropper if all its high fliers are deemed by the REF panels to be turkeys.

In Wales there’s something that makes this whole system even more absurd, which is that it’s almost certain that there will be no QR funding at all. Welsh universities are spending millions preparing for the REF despite the fact that they’ll get no money even if they do stunningly well. The incentive in Wales is therefore even stronger than it is in England to submit only the high-fliers, as it’s only the position in the league tables that will count.

The problem with a department adopting the strategy of being very selective is that it could have a very  negative effect on the career development of younger researchers if they are not included in their departments REF submission. As well as taking the risk that people who manage to convince their Head of School that they are bound to get four stars in the REF may not have the same success with the various grey eminences who make the decision that really matters.

Previous incarnations of the REF (namely the Research Assessment Exercises of 2008 and 2001) did not publish explicit information about exactly how many eligible staff were omitted from the submissions, largely because departments were extremely creative in finding ways of hiding staff they didn’t want to include.

Now however it appears there are plans that the Higher Education Statistics Agency (HESA) will publish its own figures on how many staff it thinks are eligible for inclusion in each department. I’m not sure how accurate these figures will be but they will change the game, in that they will allow compilers of league tables to draw up lists of the departments that prefer playing games to   just allowing the REF panels to  judge the quality of their research.

I wonder how many universities are hastily revising their submission plans in the light of this new twist?

Advertisements

Winter Weather Contrasts

Posted in Biographical with tags , , , on January 18, 2013 by telescoper

The world-famous British weather has been pulling out all the stops over the last few days. During a short break in the proceedings yesterday I did a bit of flat-hunting in Brighton. It was a lovely bright morning, with the winter sun low in the sky making the city look absolutely beautiful. Here’s a pic I took with my Blackberry of the Palace Pier from Marine Parade…
IMG-20130117-00042

I managed to get away at a reasonable hour after the end of the interviews and made it home to Cardiff before the snow arrived, which it eventually did around one o’clock in the morning. It’s still snowing a bit, actually, but it’s now mixed with drizzle. The slushy streets are unusually quiet. There’s not all that much in Cardiff itself, but the examinations due to start at 9am this morning were delayed until 9.30 to allow students and staff extra time to get to the various venues. The main thing is that it’s very dark, with grey clouds filling the sky. Here’s another Blackberry pic, taken on my walk into work this morning..

IMG-20130118-00043

The contrast with Brighton yesterday is considerable, except for the temperature. Bright and dry in Brighton, dark and damp in Cardiff and cold in both.

Still, at least the “red snow alert” broadcast by the BBC came to nothing. This lot is definitely white.