Archive for UCAS Tariff

Value Added?

Posted in Bad Statistics, Education with tags , , , , , on October 22, 2012 by telescoper

Busy busy busy. Only a few minutes for a lunchtime post today. I’ve a feeling I’m going to be writing that rather a lot over the next few weeks. Anyway, I thought I’d use the opportunity to enlist the help of the blogosphere to try to solve a problem for me.

Yesterday I drew attention to the Guardian University league tables for Physics (purely for the purposes of pointing out that excellent departments exist outside the Russell Group). One thing I’ve never understood about these legal tables is the column marked “value added”. Here is the (brief) explanation offered:

The value-added score compares students’ individual degree results with their entry qualifications, to show how effective the teaching is. It is given as a rating out of 10.

If you look at the scores you will find the top department, Oxford, has a score of 6 for “value added”;  in deference to my alma matter, I’ll note that Cambridge doesn’t appear in these tables.  Sussex scores 9 on value-added, while  Cardiff only scores 2. What seems peculiar is that the “typical UCAS scores” for students in these departments are 621, 409 and 420 respectively. To convert these into A-level scores, see here. These should represent the typical entry qualifications of students at the respective institutions.

The point is that Oxford only takes students with very high A-level grades, yet still manages to score a creditable 6/10 on “value added”.  Sussex and Cardiff have very similar scores for entry tariff, significantly lower than Oxford, but differ enormously in “value added” (9 versus 2).

The only interpretation of the latter two points that makes sense to me would be if Sussex turned out many more first-class degrees given its entry qualifications than Cardiff (since their tariff levels are similar, 409 versus 420). But this doesn’t seem to be the case;  the fraction of first-class degrees awarded by Cardiff Physics & Astronomy is broadly in line with the rest of the sector and certainly doesn’t differ by a factor of several compared to Sussex!

These aren’t the only anomalous cases. Elsewhere in the table you can find Exeter and Leeds, which have identical UCAS tariffs (435) but value added scores that differ by a wide margin (9 versus 4, respectively).

And if Oxford only accepts students with the highest A-level scores, how can it score higher on “value added” than a department like Cardiff which takes in many students with lower A-levels and turns at least some of them into first-class graduates? Shouldn’t the Oxford “value added” score be very low indeed, if any Oxford students at all fail to get first class degrees?

I think there’s a rabbit off. Can anyone explain the paradox to me?

Answers on a postcard please. Or, better, through the comments box.

Advertisements