The Culture of Over-Assessment in STEM

This afternoon I went to yet another meeting about assessment and feedback in University teaching involving members of staff and students from the School of Physics & Astronomy here at Cardiff University as well as some people from other schools and departments. Positive though this afternoon’s discussion was, it didn’t do anything to dissuade me from a long-held view that the entire education system holds back the students’ ability to learn by assessing them far too much. This is a topic that I’ve blogged about a few times before over the years (see, e.g., here) but given that the problem hasn’t gone away (and indeed is probably going to get worse as a result of the Teaching Excellence Framework which the Westminster government is trying to impose on universities),  I make no apologies for repeating the main points here.

One important point we need to resolve to pin down essentially what is meant by “Research-led Teaching”, which is what we’re supposed to be doing at universities. In my view too much teaching is not really led by research at all, but mainly driven by assessment. The combination of the introduction of modular programmes and the increase of continuously assessed coursework has led to a cycle of partial digestion and regurgitation that involves little in the way of real learning and certainly nothing like the way research is done. I don’t know why we’ve got into this situation but it can’t be allowed to continue.

I’m not going to argue for turning the clock back entirely but, for the record, my undergraduate degree involved no continuous assessment at all (apart from a theory project I opted for in my final year. Having my entire degree result based on the results of six three-hour unseen examinations in the space of three days is not an arrangement I can defend, but note that despite the lack of continuous assessment I still spent less time in the examination hall than present-day students.

That’s not to say I didn’t have coursework. I did, but it was formative rather than summative; in other words it was for the student to learn about the subject, rather for the staff to learn about the student. I handed in my stuff every week, it was marked and annotated by a supervisor, then returned and discussed at a supervision.

People often tell me that if a piece of coursework “doesn’t count” then the students won’t do it. There is an element of truth in that, of course. But I had it drummed into me that the only way really to learn my subject (Physics) was by doing it. I did all the coursework I was given because I wanted to learn and I knew that was the only way to do it. I think we need to establish that as a basic principle of education in physics (and similar subjects).

The very fact that coursework didn’t count for assessment made the feedback written on it all the more useful when it came back because if I’d done badly I could learn from my mistakes without losing marks. This also encouraged me to experiment a little, such as using a method different from that suggested in the question. That’s a dangerous strategy nowadays, as many seem to want to encourage students to behave like robots, but surely we should be encouraging students to exercise their creativity rather than simply follow the instructions? The other side of this is that more challenging assignments can be set, without worrying about what the average mark will be or what specific learning outcome they address.

I suppose what I’m saying is that the idea of Learning for Learning’s Sake, which is what in my view defines what a university should strive for, is getting lost in a wilderness of modules, metrics, percentages and degree classifications. We’re focussing too much on those few aspects of the educational experience that can be measured, ignoring the immeasurable benefit (and pleasure) that exists for all of us humans in exploring new ways to think about the world around us.

10 Responses to “The Culture of Over-Assessment in STEM”

  1. As one sitting in the same meeting today I completely agree that we have far too much “summative” continual assessment in our modules. It’s part of the current insane preoccupation with measurement, questionnaires and league tables, which the Teaching so-called Excellence Framework will make even worse.

    But I’m not sure we can change things too much: we might easily get a lower score on the NSS “assessment and feedback” question if we have less of it. That shouldn’t really be a motivation, but it is. The very questions chosen in the NSS are designed to set the agenda – it’s all about control.

    True, in an idealised view of university education, every student would be self motivated to learn. Like you, I had only formative assessment when a student at university. It was also the norm when I joined Cardiff as a lecturer. Sadly, many students I talk to nowadays say that they will only work through problems if they get marks “which count”. (And they are very concerned about every fraction of a mark.) Their expectations have been distorted by the continual pressure to get marks. I hope we *can* somehow change this, and we should try, but the lack of real control by those doing the teaching and researching in universities doesn’t give me cause for optimism!

  2. John Peacock Says:

    Peter: I agree 100%. And to the charge sheet should be added the increasing tendency to examine courses instantly: in Edinburgh, we’re currently fighting a proposal from the centre that 50% of courses should be examined prior to Christmas, in every year. This is educational madness, as it leaves no time for understanding to mature, or for connections between courses to become apparent. Feeling such connections emerge during bulk revision for finals over the Easter holidays was and remains one of the greatest joys of my academic career.

    Now it will be argued that we are unusual, having ended up as academics, and that typical students even back whenever didn’t need this treatment, never mind now. It’s hard to rule this out, but I still end up feeling that more time to think has to be a good thing, and that marked coursework should be discouraged in favour of formative work – we’ve all seen the narrowness that comes from the focus on marks.

    So how to get students to graft, if not by offering marks? You paint a noble picture of being motivated by a desire for understanding, but I have to admit that this wasn’t why I worked as a student – it was more a question of not wanting to look dumb in supervisions. This is where the Oxbridge system really scores: the merciless concentration of a single expert tutor on just you and a partner for several hours a week. There’s no hiding place, and not doing the homework just isn’t an option. But at Edinburgh our groups are bigger, so students can coast along quietly or absent themselves without it being noticed.

    So this is the only true way to get high NSS scores: give the students more individualised attention from tenured academics or at least senior postdocs, and then will learn and be happy doing so. The trouble is that we spend less time with students than we might. You could say that this is laziness, or a selfish concentration on research. But the truth is that there are so many other things that steal our time, which are neither student contact nor thinking about research (from REF down to home-grown bureacratic idiocies) – so people are understandably reluctant to donate more contact hours lest research time shrink to zero. The only solution is to get the system off our backs: there are plenty of hours in the week to give students the contact they need and get research done – we just need to be able to concentrate on these priorities.

    • telescoper Says:

      Yes, one of the points I made at the meeting yesterday is that we also spend a huge amount of time marking and setting coursework that is of very little benefit to the students – and they spend a huge amount of time doing it too. I’m sure we could ditch a lot of this and concentrate more on the personal contact which is actually vital.

      Here at Cardiff we do small-group tutorials in Years 1 and 2 (usually 4 or 5 students), though many places have scrapped this I think they’re vital. On the other hand students also say it depends a lot on the tutor – some are better than others at getting students involved. Also we don’t have similar things in Years 3 and 4.

      I’m going to be teaching an MSc course next term on The Physics of the Early Universe. It will be a relatively small class (~20 students) so I might try adding some small-group activities in.

      P.S. I don’t think I was ever particularly scared of looking stupid in tutorials. I just assumed that I would!

  3. Brian Schmidt Says:

    Peter: Here Here!

    In my Cosmology class at ANU I made the assessment of class entirely from a set of research-styled problems to solve – where students have to work on substantive problems from beginning to end – and use tutorials to figure out how to do them over the course of 2-3 weeks. Most of the students never quite figured out that the problems sets were not to assess the lecture material , and many object – but I still reckon it is the best way for them to actually learn cosmology.

    Hard part is creating problem sets which they can solve that are interesting…

    This isn’t going to be so useful necessarily in first year – but it is one tool to limit the time wasted on assessment.

    With respect to John’s notes on small-group tutorials. I agree that the Oxbridge small-group tutorial provides an amazing incentive to work as one goes, but it is not scalable to 40% of the population – and still requires a crazy career-defining exam at the end. Although its is hard to argue that is is not effective at creating some of the greatest academics in the world.

    Can we figure out how to use the advantages of it, scale it up, and perhaps use some assessment along the way to remove the issues around the final exam. I think whatever we do, to get scalability, will need to use the students themselves to help each other learn, with a way of elevating the more sophisticated issues to the Prof.

    Motivating students to keep up remains a problem, peer-to-peer assessment is one way (e.g. students judge each others knowledge of material in each tutorial rank-order) and this can be augmented by input from Prof and quizzes/exams to normalise these marks, and this can form part of the student’s assessment. I personally think getting assessment in smaller bits along the way is better than a big exam at the end. But I have no data to support this view.

    • telescoper Says:

      I think it’s absolutely wonderful that you still find time to teach when you’re a VC!

      • Brian Schmidt Says:

        Sadly – I had to give up teaching this year as VC… But I am looking at how I might contribute next year at least a bit.

  4. I agree with the general point about the small group tutorials as a way of engaging with students vs. the testing/grades approach. But this style of teaching doesn’t work well for everyone either. I’m pretty sure I would struggle as I’m not very good at talking-while-learning and need a lot of time to digest things; and my experience with leading small group tutorials in Oxford was that some students thrive and others completely shut down.

    I would also worry that students from poorer backgrounds, women, minorities would feel less confident and find it hardest to adjust to this style of teaching – especially as our senior academics are still predominantly white & male. A tutor can be an excellent role model (my 2nd year tutor became my 4th yr project supervisor, then my PhD supervisor!), but can also demolish someone’s confidence and ambition with one careless comment. In that sense, more formal assessment in the form of tests & exams might be better at keeping a level playing field.

    • John Peacock Says:

      Some fair points here, and I wouldn’t suggest that there’s a universal best solution. Discussions with peers can certainly be instructive, and yes tutorials can be intimidating as much as illuminating for what the tutor says. I suppose my point was really that there’s no substitute for personal graft. You learn most when you meet a problem you can’t do at first, but keep battering away until you find a way round it. Fear of tutorial embarrassment was an effective way of making me put in the necessary hours.

      • There is no “right” way to learn: each student must find out what works for them. Likewise there is no “right” way to teach: we should be trying to offer as wide a range of activities as possible to allow students the best chance of finding what works for them.

        For this to work, though, students have to be sufficiently motivated to learn, and I don’t think giving marks is the right way of doing this.

  5. Penny A Gowland Says:

    Absolutely agree.. students are too stressed and cant see the big picture. It is no longer a case that ‘they wont work if we don’t make them’- they are working too hard and don’t spend any time assimilating things. Its like this from the age of 5 til the age of 22.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: