REF moves the goalposts (again)

The topic of the dreaded 2014 Research Excellence Framework came up quite a few times in quite a few different contexts over the last few days, which reminded me that I should comment on a news item that appeared a week or so ago.

As you may or may not be aware, the REF is meant to assess the excellence of university departments in various disciplines and distribute its “QR” research funding accordingly.  Institutions complete submissions which include details of relevant publications etc and then a panel sits in judgement. I’ve already blogged of all this: the panels clearly won’t have time to read every paper submitted in any detail at all, so the outcome is likely to be highly subjective. Moreover, HEFCE’s insane policy to award the bulk of its research funds to only the very highest grade (4* – “internationally excellent”) means that small variations in judged quality will turn into enormous discrepancies in the level of research funding. The whole thing is madness, but there seems no way to inject sanity into the process as the deadline for submissions remorselessly approaches.

Now another wrinkle has appeared on the already furrowed brows of those preparing REF submissions. The system allows departments to select staff to be entered; it’s not necessary for everyone to go in. Indeed if only the very best researchers are entered then the typical score for the department will be high, so it will appear  higher up  in the league tables, and since the cash goes primarily to the top dogs then this might produce almost as much money as including a few less highly rated researchers.

On the other hand, this is a slightly dangerous strategy because it presupposes that one can predict which researchers and what research will be awarded the highest grade. A department will come a cropper if all its high fliers are deemed by the REF panels to be turkeys.

In Wales there’s something that makes this whole system even more absurd, which is that it’s almost certain that there will be no QR funding at all. Welsh universities are spending millions preparing for the REF despite the fact that they’ll get no money even if they do stunningly well. The incentive in Wales is therefore even stronger than it is in England to submit only the high-fliers, as it’s only the position in the league tables that will count.

The problem with a department adopting the strategy of being very selective is that it could have a very  negative effect on the career development of younger researchers if they are not included in their departments REF submission. As well as taking the risk that people who manage to convince their Head of School that they are bound to get four stars in the REF may not have the same success with the various grey eminences who make the decision that really matters.

Previous incarnations of the REF (namely the Research Assessment Exercises of 2008 and 2001) did not publish explicit information about exactly how many eligible staff were omitted from the submissions, largely because departments were extremely creative in finding ways of hiding staff they didn’t want to include.

Now however it appears there are plans that the Higher Education Statistics Agency (HESA) will publish its own figures on how many staff it thinks are eligible for inclusion in each department. I’m not sure how accurate these figures will be but they will change the game, in that they will allow compilers of league tables to draw up lists of the departments that prefer playing games to   just allowing the REF panels to  judge the quality of their research.

I wonder how many universities are hastily revising their submission plans in the light of this new twist?

6 Responses to “REF moves the goalposts (again)”

  1. This just gets sillier and sillier.

    I am beginning to understand friends and colleagues who have left the country to avoid this kind of thing.

  2. To the left of centre Says:

    Reblogged this on To the left of centre and commented:
    I had considered writing about this myself. I noticed an article in the Times Higher Education a few days ago about HESA publishing contextual staff data. I’ve written a number of times about my concerns about REF2014 (see, for example, REF2014: Good or Bad and The negative impact of REF) and it is amazing – to me at least – that at this late stage they are deciding on procedures. Rather than writing my own post, I thought I would simply reblog this as it mostly says what I was going to say myself.

  3. Rob Ivison Says:

    I’m accustomed to hearing folk criticise STFC following the failure of their dept to win a suitable grant. Presumably what they really mean is “the university robber barons on AGP didn’t rate my science”.

    Criticisms of the REF seems similar. Who is making all these REF rules, if not the robber barons that run most of our elite universities?

    Most of the commentators on this blog are from those very universities – the winners, loosely speaking – and even they’re not happy, so God help the rest!

    • telescoper Says:

      The REF rules are made by the good folk at HEFCE not by universities, although as you say the VCs of some of the latter will be happy whatever the rules are, as long as their institutions get the dosh.

  4. Dave Carter Says:

    One thing which would help would be if HESA would tell us which staff it thinks are eligible before we have to make the submission.

    • Dave Carter Says:

      And in general, with the REF, I don’t mind them moving the goalposts, but it would be good if they let us know when they do, and they told us where the goalposts have gone.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: