The REF goes on

A few communications with former colleagues from the United Kingdom last week reminded me that, despite the Covid-19 pandemic, the deadline for submissions to the 2021 Research Excellence Framework is next week. It seems very strange to me to push ahead with this despite the Coronavirus disruption, but it’s yet another sign that academics have to serve the bureaucrats rather than the other way round.

I know quite a few people at quite a few institutions that are completely exhausted by the workload required to deal with the enormous exercise in paperwork that is intended to assess the quality and impact of research at UK universities.

With apologies for adding to the stack of memes based on recent events in the Suez Canal, it made me think of this:

One of the major plusses of being in Ireland is that there is no REF, so I’m able to avoid the enormous workload and stress generated by this exercise in bean-counting. That’s good because there are more than enough things on my plate right now, and more are being added every day.

My memories of the last REF in 2014 when I was Head of School at Sussex are quite painful, as it went badly for us then. I hope that the long-term investments we made then will pay off, though, and I hope things turn out better for Sussex this time especially for the Department of Physics & Astronomy for which the impact and environment components of the assessment dragged the overall score down.

Not being involved personally in the REF this time round I haven’t really paid much attention to the changes that have been adopted since 2014. One I knew about is that the rules make it harder for institutions to leave staff out of their REF return. Some universities played the system in 2014 by being very selective about whom they put in. Only staff with papers considered likely to be rated top-notch were submitted.

Having a quick glance at the documents I see two other significant differences.

One is that in 2014, with very few exceptions, all staff had to submit four research outputs (i.e. papers) to be graded. in 2021 the system is more flexible: the total number of outputs must equal 2.5 times the summed FTE (full-time equivalent) of the unit’s submitted staff, with no individual submitting more than 5 and none fewer than 1 (except in special cases related to Covid-19). Overall, then there will be fewer outputs than before, the multiplier of FTE being 2.5 (2021) instead of 4 (2014). There will still be a lot of papers, of course, not least because many Departments have grown since 2014, so the panels will have a great deal of reading to do. If that’s what they do with the papers. They’ll probably just look up citations…

The other difference relates to staff who have left an institution during the census period. In 2014 the institution to which a researcher moved got all the credit for the publications, while the institution they left got nothing. In 2021, institutions “may return the outputs of staff previously employed as eligible where the output was first made publicly available during the period of eligible employment, within the set number of outputs required.” I suppose this is to prevent the departure of a staff member causing too much damage to the institution they left and also to credit the institution where the work was done rather than specifically the individual who did it.

Thinking about the REF an amusing thought occurred to me about Research Assessment. My idea was to set up a sort of anti-REF (perhaps the Research Inferiority Framework) based not on the best outputs produced by an institutions researchers, but on the worst. The institutions producing the highest number of inferior papers could receive financial penalties and get relegated in the league tables for encouraging staff to write too many papers that nobody ever reads or are just plain wrong. My guess is that papers published in Nature might figure even more prominently in this…

Anyway, let me just take this opportunity to wish former colleagues at Cardiff and Sussex all the best for their REF submission on Wednesday 31st March. I hope it turns out well


12 Responses to “The REF goes on”

  1. A major change is that all staff that have research as part of their contract must be submitted, so that departments can no longer pick and choose which staff to submit.

    Also, the percentage of funding associated with impact has increased to 25%, although the number of impact case studies required has decreased somewhat.

    In the UK, the block grant for research to HEIs – the QR funding – is linked to REF performance. Do universities in Ireland get a research block grant, and if so how is it calculated? Purely on staff numbers? (As used to be the case in the UK pre-RAE, the precursor to REF).

    • telescoper Says:

      The Higher Education Authority includes an element of research funding for universities in its recurrent block grant. You can read about that here. I don’t know the details of the calculation though. About 20% of research funding to Irish universities comes from the HEA (kind of the equivalent of HEFCE).

      If I recall correctly the old UK system was based on student numbers rather than staff numbers – funding for research was tied to funding for teaching which is sensible if you believe in research-led teaching. That link was broken when the former polytechnics became universities in 1992, as many of them didn’t really do much research.

    • ‘Departments can no longer pick and choose’… not entirely. Someone who does not have ‘significant responsibility for research’ is not submitted, so various games can be and are played with the criteria for that to avoid submitting people who might pull the score down. These are submitted in the REF code of practice — itself a huge time sink. It’s got to the point where the selection of outputs seems like a minor part of the whole exercise.

      • I thought it came down to whether they had research included in their contract or terms of employment? In which case would be difficult to leave out. Looking at the guidelines, as well as the criterion you listed it also says that independent research ‘…is an expectation of their job role’. So games may be played, but cannot see research-active staff being happy to be left out as its then implied that there is no expectation of them to do research.

        Is there also a possibility that a panel member might say something like ‘I see no papers from XXXX in this submission, and I know that they are research-active. We need to check this out?’ Which could cause problems for the HEI if being seen to play games.

      • The exact criteria are in individual institutions’ code of practice (which have to be approved). My institution put in as the criterion a certain minimum level of time allocated to research in the workload model (which was approved), and then of course demanded that people not wanted in the REF should be allocated less than that… fortunately with no actual effect in my department.

  2. Looked at that website and the Irish funding appears to be mostly linked to postgraduate research student numbers, although there is an element based on research metrics.

    The UK funding based on numbers must have been broken pre-1992 because the RAE started in 1988, but back then it was mostly a metric based system – number of papers, PDRAs and PhD students, if memory serves. Also think that the old funding was based on staff numbers, but could be wrong.

    • telescoper Says:

      The earlier incarnations of the RAE – I remember the first one conducted by the UGC in 1986 – were much lighter touch and, I believe, just based on publications. The first RAE named as such was in 1992.

      • Yep – I was thinking of the 1989 one – which we probably worked on in 1988. However although – looking at wiki – it says that papers were graded, I do not recall ever submitting a paper. We just had a table with total number of papers, PDRAs and students. Perhaps it was different in NI. Was only an SERC Advanced Fellow at the time so not really involved until 1992. However even then don’t recall submitting outputs…

        Ah well, just be glad you are not involved.

        I read somewhere that it costs the equivalent of running 2 universities to run the exercise, when you factor in the amount of admin that university staff have to devote to it. Could well believe it.

      • telescoper Says:

        Yes. I seem to remember just having to produce a list of papers. I don’t recall getting any grades back.

  3. Fully agree on the costs front… Some years back I was on the STFC Education and Training Committee as an industrial rep…. One of our tasks was a Research fellowship scheme, and I remember pointing out that if the cost of the excercise was taken into account (which of course it was not, since panel time is lost in the murk of univerity costings), then at least 2 additional Fellowships could have been funded if the awards had been made by drawing lots… Clearly not realistic, but I do think the cost of such excercises should be addressed. For comparison, an industrial bid for an ESA contract is generally reckoned to cost ~10% of the contract value.

    • I did the calculation once for the consolidated grants. In terms of panel time the cost of the round was around 10% of the award. Of course, most research is done by PDRAs and not by academics so maybe this was a pretty good exchange. For the REF the cost is astronomical for no obvious benefit to anyone.

  4. […] You can find further discussion of the REF submission rules, especially concerning changes with respect to 2014, here. […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: