The Final Analysis

It’s that time of year again. The annual meeting of the Board of Examiners of the School of Physics & Astronomy at Cardiff University met this afternoon to consider the marks for students due to graduate this year and to draw up recommendations for the final degree classifications.

This year’s meeting was actually quite interesting and, at just over two hours, somewhat shorter than some we’ve had in previous years.  A group of students has already gathered in the foyer waiting for the dreaded list to be posted, and many will no doubt be celebrating or drowning their sorrows in local hostelries shortly after their fate is revealed.

Cardiff is a little old-fashioned in the way the final examiners’ meetings are conducted. For a start we still have a system of viva voce examinations for borderline candidates; they happened yesterday, in fact. Many universities have dispensed with this aspect of the process, but I still think they’re worthwhile. The Board of Examiners, including the two External Examiners,  also still has some discretion in how it arrives at the degree boundaries (which are nominally at 70% for a first, 60% for a 2.1, 50% for a 2.2, and 40% for a 3rd).

The tide is turning against this very traditional approach, however, and there are moves here to dispense with the viva examinations and with academic discretion. I’m not sure when this will happen, but it’s likely to be sooner rather than later. I think the main reason for this is to make the system more automatic so there’s less chance of legal challenge. In any case there doesn’t seem to me that there can be any educational reason for it.

The one thing that strikes me about the system we have is that it’s the whole business of classifying degrees into broad categories which is where the problem lies.  It was suggested some time ago that we should dispense with, e.g., the “Desmond” (2.2) and the “Thora Hird” (3rd) and instead simply give each student a transcript containing details of the entire spectrum of their academic performance. It’s been suggested again just recently too. That would seem to me to make much more sense than the current system of classifying degrees which involves (a) trying to condense a huge amout of information – examination marks, coursework, project assignments and the like – into a single number and then (b) drawing boundaries based on this number precisely where the distribution is most densely peaked. However, years have passed and nothing concrete has happened. The academic world is good at inertia.

Anyway, this isn’t the time or the place for a lengthy diatribe about the ins-and-outs of degree classifications. I’ve got to go back to marking my 1st year examination papers shortly in fact; these are considered by a separate meeting of the Board of Examiners.

It is time, however, for me to congratulate all our graduating students on their success. It’s the first group of Cardiff MPhys students that I’ve seen all the way through from entry  to graduation. I’ll be sad to see them go, but wish them all the best as they venture forth into the real world. At least I hope to see them back next month for graduation.  Until then, however, all I’ll  say is


20 Responses to “The Final Analysis”

  1. Thora’s new to me. Was always a Douglas here. Is it a regional thing, or has Hird left more trace in the language than a mere passing Hurd?

  2. The issue I have with producing a whole transcript in place of a summative number is that it makes the results much harder for a future employer to digest. This may be okay for someone staying in academia – other physicists should understand what ‘statistical mechanics’ or ‘non-linear optics’ means – but I’m not sure this would be true for most employers. This might unreasonably benefit those who do modules that sound relevant such as ‘business and finance’ over those that are more, well, physical.

    And are we going to expect future employers to plough through several pages of transcript?

  3. Peter – two things :

    (i) we still classify students, but we also issue a transcript and have done for many years. I had assumed that employers would get used to the transcript and making their own minds up, and so the classifications would fade away, but they don’t seem to have done so yet.

    (ii) w.r.t. academic discretion : we solve this by having two-stage boards. Each course (typically one sixth of the year’s diet) has a miniature course-board with setter and checker, and has the power to adjust marks before they go forward to the degree-programme exam board, which cannot alter marks. Any action taken by the course board is reviewed by the external, who confirms (or not) that the results compare to what would be seen at their own university. So discretion is shown by people who have looked seriously at the papers and the scripts, but the final exam board is fairly mechanical. Likewise there is a separate “special circumstances committee” which forwards recommendations to the exam board.

  4. John Peacock Says:

    For the majority of employers, the details of competencies in particular modules probably don’t matter – they just want some general impression of how well a student can cope with the whole process of critical thinking and problem solving. So I don’t think the move to transcripts will help anyone, except possibly agencies who will spring up to translate transcripts into overall grades for a fee…

    But I don’t think you’d come up with the current system of 4 pass grades if you were starting from scratch: our marks are more precise than that. When I did O levels, there were 6 pass grades; when I did A levels, there were 5 pass grades; today I grade Royal Society fellowship applicants on a scale of 1 to 7. So between 5 and 10 pass grades seems sensible.

    I’d also be tempted to abandon the pretence of uniformity. A 3/10 from the Uni of South Woldshire would just say you’re in the top 30% of your class there. Employers already believe that some universities matter more than others, so this matches reality. The advantage of this proposal is that it might prevent grade drift: especially once the fee cap is lifted in the next parilament, universities charging a gazillion pounds in fees are going to be awfully tempted to give out more Firsts so that the “customers” get value for money. Moving to a simple statement of ranking would remove this temptation. And we all know that relative judgements of ability are a lot more reliable than the pretence that we can set and mark exams to an invariant absolute standard.

    • telescoper Says:

      Having been an external examiner in several universities (and indeed an internal in quite a few too), I would say that at least in physics I think the degree classifications are broadly comparable across institutions. Certainly the first-class students from Cardiff are as good as you’ll find anywhere.

      I’m not sure about other courses.

    • This relative ranking is something that should be brought back to A-levels to avoid exactly the grade inflation you suggest for future universities. You could in fact have a dual classification system if politicians still want to keep the ‘more people getting As every year’ pronouncements.

      So you could get an A10 meaning A grade and in the top 10 percent, all the way down to an A30, judging from recent pass rates. This would certainly make our job on admission panels a lot easier!

      However, you’d have to cope with the fact that a lot of A level students are getting close to 100% marks in most modules…

      • John Peacock Says:

        Continuing the “back to the good old days” theme, I would certainly prefer to base classification (in School and in University) entirely on unseen exams, rather than coursework that amounts to giving high marks to anyone who is prepared to put in a reasonable effort. There is a correct argument that exams are not a fair reflection of student’s performance in the real world, but the way to cure that is to introduce more open-book exams. We need to be consistently testing students’ understanding of complex material, and too much of our assessment fails to do this.

      • telescoper Says:

        I continue to be dismayed how many examinations function mainly as memory tests rather than as tests of understanding. This is probably because of school examinations, but it has got so far into the system that students can get very high grades by simply memorizing and regurgitating material without much understanding. It’s not entirely the students’ fault, of course. Too many people set exams that can be approached this way, probably because too many would fail if they really did test understanding.

        For these reasons I’m all in favour of open-book exams and/or take-home tests. I’m not sure why we don’t have more of them other than because the university administrators don’t seem to like them….

    • Mark McCaughrean Says:

      On your point, John, about universities perhaps being tempted in the future to grant more firsts to satisfy the paying customers, I fear that that particular Rubicon was crossed years ago in some subjects.

      After all, some of the many invidious university league tables actually include “fraction of students gaining firsts and 2.1’s” as one of the elements. Hardly surprising then that some universities and/or departments have become rather more generous in handing out those particular sweeties over the past decade.

  5. Andy – that’s still 20 to 30 times as many numbers as the usual classification system. If you have an HR department that has got 2000 applications for 20 jobs that’s a lot of extra work to be done by people who won’t understand what the terms in the transcripts mean or how to compare one university’s course names to another’s. That’s why they like to filter in a simplistic metric like degree classification and I guess why it’s still being used, despite Edinburgh’s best efforts.

  6. Bryn Jones Says:

    Dispensing with discretion in exam boards could cause problems.

    Exam boards usually use discretion to push candidates who fall just below a classification boundary up to the degree class above. That way, issues of unfair treatment generally become unimportant: even if a student had been treated unfairly in one or two modules and had fallen short of the formal boundary of a degree class as a result, there is a tendency to push the student up by 1% or 2% in the final overall mark to push him/her to the class above. The student then has no reason to complain as there has been no adverse consequence. There is a generosity built into the system that can correct for unfairness.

    Scrapping this system to adopt an automatic allocation of the degree class would encourage formal challenges, even legal proceedings. Unfair treatment in individual lecture modules could affect the final degree classification in some cases. The generosity would have been removed.

  7. After being postgrad coordinator for 6 years, I can tell you that there is nothing more annoying than a transcript purely with marks on it, without an indication of how the rest of the class did. Currently, we do not accept students onto PhD without a 1st class degree, and when we have to decide if it is 1st class or not, it can be a real mare.

  8. telescoper Says:

    The UK is nominally signed up to the Bologna process but hasn’t taken any steps to ensure compliance with it. Our 4-year MPhys degrees lie outside the Bologna scheme, for example.

  9. telescoper Says:

    As for transcripts, I believe these are available at the moment in most universities- although some charge a fee. For PhD applications we request a full transcript in addition to the degree classification, but I suspect most employers won’t be interested in it.

    I think universities would help their students more by giving them specialist advice on how to construct a proper CV.

  10. telescoper Says:

    Among all the gripes about examinations I feel I should mention that I’ve just finished marking my first-year scripts. Among them was the first 100% perfect script I’ve seen in all my years as an academic, with one question tackled in a much better way than the model answer I prepared.

  11. I think it would do a massive disservice to students if viva voce examinations and teaching staff discretion were not used in determining a student’s final degree classification. That is one of the things that makes one’s degree classification so much fairer than the grades one gets in public exams like GCSEs and A-levels. Many times I have given my personal insight on a student’s abilities as I have perceived them in tutorials, class, etc, as their “raw” marks are often only part of the story.

    I have lectured at places where we got all final year students to give oral presentations on aspects of their final year courses, in addition to the exams they sat. At Imperial, it was only students who were on the borderline between degree classifications who got viva voce exams, but I think there is a lot to be said for all final year students having such an exam.

    Is the move by Cardiff to automate the process because lecturers feel too much of their (valuable) time is being taken up assessing students? Surely, as the University pays them primarily to teach/mentor/assess undergraduates and post-graduates, they should see it as one of their primary roles.

    • telescoper Says:

      It definitely doesn’t come from lecturers (or external examiners, for that matter). Like most things these days it’s decided by people in the university admin.

      My impression is, however, that most UK universities have already dispensed with vivas and Cardiff is one of the last to keep them.

      I should also say that I’m in favour of keeping the examination process as algorithmic as possible unless there are compelling reasons to make special allowances. My main concern with removing academic discretion is not with the degree boundaries, but in ensuring proper allowance is made for extenuating circumstances.

      • Peter,

        What do you think is the University admin’s motivation in wanting to streamline/automate the student assessment process? Is it one of “liability”, so that if a student complains about their degree classification the University can just point to the raw numbers and say that it is based on a strict algorithm, with little/no lecturer intervention, and therefore go away and stop complaining?

        Oral exams are really the best way to determine whether a student understands material. This is, of course, why an oral exam is given to pass one’s PhD. At other universities where I have lectured, we gave oral exams to all students in the third year (I think I said this above, apologies 🙂 ), and it really was good at seeing whether questions they appeared to have answered quite well in their exams really were understood. I really think this is what we should be aiming for in assessing our final year students, not using some streamlined/automated process.

        But then again, I’m used to thinking differently to most people 🙂

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: