Out of the REF

I was talking over Zoom with some former colleagues from the United Kingdom last week, and was surprised to learn that, despite the Covid-19 pandemic, the 2021 Research Excellence Framework is ploughing ahead next year, only slightly delayed. There’s no stopping bureaucratic juggernauts once they get going…

One of the major plusses of being in Ireland is that, outside the UK academic system, there is no REF. One can avoid the enormous workload and stress generated by this exercise in bean-counting My memories of the last REF in 2014 when I was Head of School at Sussex are quite painful, as it went badly for us then. I hope that the long-term investments we made then will pay off, though, and I hope things turn out better for Sussex this time especially for the Department of Physics & Astronomy for which the impact and environment components of the assessment dragged the overall score down.

The census period for the new REF is 1st August 2013 to 31st July 2020. Not being involved personally in the REF this time round I haven’t really paid much attention to the changes that have been adopted since 2014. One I knew about is that the rules make it harder for institutions to leave staff out of their REF return. Some universities played the system in 2014 by being very selective about whom they put in. Only staff with papers considered likely to be rated top-notch were submitted.

Having a quick glance at the documents I see two other significant differences.

One is that in 2014, with very few exceptions, all staff had to submit four research outputs (i.e. papers) to be graded. in 2021 the system is more flexible: the total number of outputs must equal 2.5 times the summed FTE (full-time equivalent) of the unit’s submitted staff, with no individual submitting more than 5 and none fewer than 1 (except in special cases related to Covid-19). Overall, then there will be fewer outputs than before, the multiplier of FTE being 2.5 (2021) instead of 4 (2014). There will still be a lot, of course, so the panels will have a great deal of reading to do. If that’s what they do with the papers. They’ll probably just look up citations…

The other difference relates to staff who have left an institution during the census period. In 2014 the institution to which a researcher moved got all the credit for the publications, while the institution they left got nothing. In 2021, institutions “may return the outputs of staff previously employed as eligible where the output was first made publicly available during the period of eligible employment, within the set number of outputs required.” I suppose this is to prevent the departure of a staff member causing too much damage to the institution they left.

I was wondering about this last point when chatting with friends the other day. I moved institutions twice during the relevant census period, from Sussex to Cardiff and then from Cardiff to Maynooth. In principle, therefore, both former employees could submit my outputs I published while I was there to the 2021 REF. I only published a dozen or so papers while I was at Sussex – the impact of being Head of School on my research productivity was considerable – and none of them are particularly highly cited so I don’t think that Sussex will want to submit any of them, but they could if they wanted to. They don’t have to ask my permission!

I doubt if Cardiff will be worried about my papers. Among other things they have a stack of gravitational wave papers that should all be 4*.

Anyway, thinking about the REF an amusing thought occurred to me about Research Assessment. My idea was to set up a sort of anti-REF (perhaps the Research Inferiority Framework) based not on the best outputs produced by an institutions researchers but on the worst. The institutions producing the highest number of inferior papers could receive financial penalties and get relegated in the league tables for encouraging staff to write too many papers that nobody ever reads or are just plain wrong. My guess is that papers published in Nature might figure even more prominently in this

6 Responses to “Out of the REF”

  1. If you think about it, if you are assessing a university’s UoA’s performance – rather than individuals – it makes more sense for outputs to be returned from the unit where the work was undertaken, rather than the unit where the staff member is at the time of the REF census date.

    I would imagine the change to the 2.5 x N outputs, where N is the number of research-active staff (defined as those with research as part of their contract), is meant to stop universities trying to maximise their REF scores by only submitting a percentage of their staff number.

    Another major change is that impact will be 25% of the overall score, as opposed to 20% for REF2014, making impact case studies even more important.

    • telescoper Says:

      That makes sense. At Sussex we couldn’t put everyone in because we didn’t have enough impact cases. There weren’t people who didn’t have enough decent publications.

  2. “My idea was to set up a sort of anti-REF (perhaps the Research Inferiority Framework) based not on the best outputs produced by an institutions researchers but on the worst.”

    Perhaps riff-raff?

  3. One could argue, and some have argued, that those who score the worst should get the most money, since they obviously need it to help them improve, for example by attracting high-profile researchers, while those who do extremely well would probably still do very good on substantially less.

    What does one want to do? Produce a vicious circle by rewarding the good with means to make them even better? Or improve the quality overall?

    • Except that high profile researchers generally want to join other high quality researchers, not poor quality ones. Or bring in their own high quality people.

      • Phillip Helbig Says:

        Not necessarily; some might welcome the chance to be able to do good work outside of the rat race. Of course, more money might be needed to bring in such people, and also some money to hire the people they want, which would address your second point.

        The alternative is to have a few very good places and let the rest go to waste. If you can’t make it in a very good place, you can’t make it at all. While that is OK if the reason is that you are just not good enough, it does give too much power to those in control of the good places.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: