Archive for Teaching Excellence Framework

Do you have Confidence in the Teaching Excellence Framework?

Posted in Bad Statistics with tags , , , , on January 4, 2017 by telescoper

The  Teaching Excellence Framework (TEF) is, along with a number of other measures in the 2016 Higher Education and Research Bill, causing a lot of concern in academic circles (see, e.g., this piece by Stephen Curry). One of the intentions of the TEF is to use relatively simple metrics to gauge “teaching quality” in higher education institutions. On top of the fundamental questions of exactly what “teaching quality” means and how it might be measured in any reliable way, there is now another worry: the whole TEF system is to be run by people who are statistically illiterate.

To demonstrate this assertion I refer you to this excerpt from the official TEF documentation:

tef

The highlighted “explanation” of what a confidence interval means is false. It’s not slightly misleading. It’s not poorly worded. It’s just false.

I don’t know who from HEFCE wrote the piece above, but it’s clearly someone who does not understand the basic concepts of statistics.

I can’t imagine what kind of garbled nonsense will come out of the TEF if this is the level of understanding displayed by the people running it.  That garbage will also be fed into the university league tables with potentially devastating effects on individuals, departments and institutions, so my gripe is not just about semantics – this level of statistical illiteracy could have very serious consequences for Higher Education in the UK.

Perhaps HEFCE should call in some experts in statistics to help? Oh, no. I forgot. This country has had enough of experts…

 

 

 

 

Advertisements

The Culture of Over-Assessment in STEM

Posted in Education with tags , , , , on December 7, 2016 by telescoper

This afternoon I went to yet another meeting about assessment and feedback in University teaching involving members of staff and students from the School of Physics & Astronomy here at Cardiff University as well as some people from other schools and departments. Positive though this afternoon’s discussion was, it didn’t do anything to dissuade me from a long-held view that the entire education system holds back the students’ ability to learn by assessing them far too much. This is a topic that I’ve blogged about a few times before over the years (see, e.g., here) but given that the problem hasn’t gone away (and indeed is probably going to get worse as a result of the Teaching Excellence Framework which the Westminster government is trying to impose on universities),  I make no apologies for repeating the main points here.

One important point we need to resolve to pin down essentially what is meant by “Research-led Teaching”, which is what we’re supposed to be doing at universities. In my view too much teaching is not really led by research at all, but mainly driven by assessment. The combination of the introduction of modular programmes and the increase of continuously assessed coursework has led to a cycle of partial digestion and regurgitation that involves little in the way of real learning and certainly nothing like the way research is done. I don’t know why we’ve got into this situation but it can’t be allowed to continue.

I’m not going to argue for turning the clock back entirely but, for the record, my undergraduate degree involved no continuous assessment at all (apart from a theory project I opted for in my final year. Having my entire degree result based on the results of six three-hour unseen examinations in the space of three days is not an arrangement I can defend, but note that despite the lack of continuous assessment I still spent less time in the examination hall than present-day students.

That’s not to say I didn’t have coursework. I did, but it was formative rather than summative; in other words it was for the student to learn about the subject, rather for the staff to learn about the student. I handed in my stuff every week, it was marked and annotated by a supervisor, then returned and discussed at a supervision.

People often tell me that if a piece of coursework “doesn’t count” then the students won’t do it. There is an element of truth in that, of course. But I had it drummed into me that the only way really to learn my subject (Physics) was by doing it. I did all the coursework I was given because I wanted to learn and I knew that was the only way to do it. I think we need to establish that as a basic principle of education in physics (and similar subjects).

The very fact that coursework didn’t count for assessment made the feedback written on it all the more useful when it came back because if I’d done badly I could learn from my mistakes without losing marks. This also encouraged me to experiment a little, such as using a method different from that suggested in the question. That’s a dangerous strategy nowadays, as many seem to want to encourage students to behave like robots, but surely we should be encouraging students to exercise their creativity rather than simply follow the instructions? The other side of this is that more challenging assignments can be set, without worrying about what the average mark will be or what specific learning outcome they address.

I suppose what I’m saying is that the idea of Learning for Learning’s Sake, which is what in my view defines what a university should strive for, is getting lost in a wilderness of modules, metrics, percentages and degree classifications. We’re focussing too much on those few aspects of the educational experience that can be measured, ignoring the immeasurable benefit (and pleasure) that exists for all of us humans in exploring new ways to think about the world around us.