Archive for the Education Category

More Worthless University Rankings

Posted in Bad Statistics, Education with tags , , , on September 6, 2017 by telescoper

The Times Higher World University Rankings, which were released this week. The main table can be found here and the methodology used to concoct them here.

Here I wish to reiterate the objection I made last year and the year before that to the way these tables are manipulated year on year to create an artificial “churn” that renders them unreliable and impossible to interpret in any objective way. In other words, they’re worthless. This year the narrative text includes:

This year’s list of the best universities in the world is led by two UK universities for the first time. The University of Oxford has held on to the number one spot for the second year in a row, while the University of Cambridge has jumped from fourth to second place.

Overall, European institutions occupy half of the top 200 places, with the Netherlands and Germany joining the UK as the most-represented countries. Italy, Spain and the Netherlands each have new number ones.

Another notable trend is the continued rise of China. The Asian giant is now home to two universities in the top 30: Peking and Tsinghua. The Beijing duo now outrank several prestigious institutions in Europe and the US. Meanwhile, almost all Chinese universities have improved, signalling that the country’s commitments to investment has bolstered results year-on-year.

In contrast, two-fifths of the US institutions in the top 200 (29 out of 62) have dropped places. In total, 77 countries feature in the table.

These comments are all predicated on the assumption that any changes since the last tables represent changes in data (which in turn are assumed to be relevant to how good a university is) rather than changes in the methodology used to analyse that data. Unfortunately, every single year the Times Higher changes its methodology. This time we are told:

This year, we have made a slight improvement to how we handle our papers per academic staff calculation, and expanded the number of broad subject areas that we use.

What has been the effect of these changes? We are not told. The question that must be asked is how can we be sure that any change in league table position for an institution from year to year represents a change in “performance”,rather than a change in the way metrics are constructed and/or combined? Would you trust the outcome of a medical trial in which the response of two groups of patients (e.g. one given medication and the other placebo) were assessed with two different measurement techniques?

There is an obvious and easy way to test for the size of this effect, which is to construct a parallel set of league tables, with this year’s input data but last year’s methodology, which would make it easy to isolate changes in methodology from changes in the performance indicators. The Times Higher – along with other purveyors of similar statistical twaddle – refuses to do this. No scientifically literate person would accept the result of this kind of study unless the systematic effects can be shown to be under control. There is a very easy way for the Times Higher to address this question: all they need to do is publish a set of league tables using, say, the 2016/17 methodology and the 2017/18 data, for comparison with those constructed using this year’s methodology on the 2017/18 data. Any differences between these two tables will give a clear indication of the reliability (or otherwise) of the rankings.

I challenged the Times Higher to do this last year, and they refused. You can draw your own conclusions about why.

P.S. For the record, Cardiff University is 162nd in this year’s table, a rise of 20 places on last year. My former institution, the University of Sussex, is up two places to joint 147th. Whether these changes are anything other than artifacts of the data analysis I very much doubt.


Clearing Advice for Physics and Astronomy Applicants!

Posted in Education with tags , , , , on August 17, 2017 by telescoper

Today’s the day! This year’s A-level results are out today, Thursday 17th August, with the consequent scramble as students across the country to confirm places at university. Good luck to all students everywhere waiting for your results. I hope they are what you expected!

For those of you who didn’t get the grades they needed, I have one piece of very clear advice:


The clearing system is very efficient and effective, as well as being quite straightforward to use, and there’s still every chance that you will find a place somewhere good. So keep a cool head and follow the instructions. You won’t have to make a decision straight away, and there’s plenty of time to explore all the options.

As a matter of fact there are a few places still left for various courses in the School of Physics & Astronomy at Cardiff University. Why should you choose Cardiff? Well, obviously I have a vested interest since I work here, but here’s a video of some students talking about the School.

For further information check here!

Why not give back to students their marked examination scripts?

Posted in Education with tags , , , on July 6, 2017 by telescoper

Well, the examination period is over and we’re now in that curious interregnum in the academic year that lasts until graduation, when we get to congratulate students properly and send them on their way into the big wide world. I hope the weather is a bit cooler for that event. It’s no fun at all for either staff or students wearing a suit and tie with a heavy gown on top when the temperature is 30°!

Anyway, yesterday I had a meeting with a (Masters) student about one of his recent examinations, and it prompted me to write a short post about the reason for our discussion.

Here in the School of Physics & Astronomy at Cardiff University we have a system whereby students can get access to their marked examination scripts. By `script’ I mean what the student writes (usually in a special answer book), as opposed to the `paper’ which is the list of questions to be answered or problems to be solved in the script. This access is limited, and for the purpose of getting feedback on where they went wrong, not for trying to argue for extra marks. The students can’t take the scripts away, nor can they make a copy, but the can take notes which will hopefully help them in future assessments.

When I was Head of the School of Mathematical and Physical Sciences at Sussex University I tried to introduce such a system there, but it was met with some resistance from staff who thought this would cause a big increase in workload and lead to  difficulties with students demanding their marks be increased. That has never been the experience here at Cardiff: only a handful take up the opportunity and those that do are told quite clearly that the mark cannot be changed. This year I had only one student who asked to go through their script. I was happy to oblige and we had a friendly and (I think) productive meeting.

If I had my way we would actually give all students their marked examination scripts back as a matter of routine. The fact that we don’t is no doubt one reason for relatively poor performance in student satisfaction surveys about assessment and feedback. Obviously examination scripts have to go through a pretty strict quality assurance process involving the whole paraphernalia of examination boards (including external examiners), so the scripts can’t be given back immediately but once that process is complete there doesn’t seem to me any reason why we shouldn’t give their work, together with any feedback written on it,  back to the students in its entirety.

I have heard some people argue that under the provisions of the Data Protection Act students have a legal right to see what’s written on the scripts – as that constitutes part of their student record – but that’s not my point here. My point is purely educational, based on the benefit to the student’s learning experience.

Anyway, I don’t know how widespread the practice is of giving examination scripts back to students so let me conduct a totally unscientific poll. Obviously most of my readers are in physics and astronomy, but I invite anyone in any academic discipline to vote:



And, of course, if you have any further comments to make please feel free to make them through the box below!




MSc Opportunities in Data-Intensive Physics and Astrophysics

Posted in Biographical, Education with tags , , , , on June 22, 2017 by telescoper

Back to the office after external examining duties, I received an email this morning to say that the results have now been posted in Cambridge. I also had an email from Miss Lemon at Sussex that told me that their finalists’ results went up last Friday. We did ours in Cardiff last week. This provides me with a timely opportunity to congratulate all students at all three of these institutions – and indeed everywhere else – on their success!

It also occurred to me tha,t now that most students know how well they’ve done in their undergraduate degree, some may be thinking about further study, at postgraduate level. It seems a good opportunity to remind potential applicants about our two brand new MSc courses at Masters (MSc) level, called Data-Intensive Physics and Data-Intensive Astrophysics and they are both taught jointly by staff in the School of Physics and Astronomy and the School of Computer Science and Informatics in a kind of major/minor combination.

The aim of these courses is twofold.

One is to provide specialist postgraduate training for students wishing to go into academic research in a ‘data-intensive’ area of physics or astrophysics, by which I mean a field which involves the analysis and manipulation of very large or complex data sets and/or the use of high-performance computing for, e.g., simulation work. There is a shortage of postgraduates with the necessary combination of skills to undertake academic research in such areas, and we plan to try to fill the gap with these courses.

The other aim is to cater for students who may not have made up their mind whether to go into academic research, but wish to keep their options open while pursuing a postgraduate course. The unique combination of physics/astrophysics and computer science will give those with these qualifications the option of either continuing or going into another sphere of data-intensive research in the wider world of Big Data.

The motivation for these courses has been further strengthened recently by the announcement earlier this year of extra funding for PhD research in Data-Intensive Physics. We’ve been selecting students for this programme and making other preparations for the arrival of the first cohort in September. We’ve had many more applicants than we can accommodate this time, but this looks set to be a growth area for the future so anyone thinking of putting themselves in a good position for a PhD in Data-Intensive Physics or Astrophysics in the future might think about preparing by taking a Masters in Data-Intensive Physics or Astrophysics now!

I just checked on our admissions system and saw, as expected, conditional offers turning into firm acceptances now that the finals exam results are being published across the country but we have still got plenty of room on these courses so if you’re thinking about applying, please be assured that we’re still accepting new applications!


Module Evaluation

Posted in Biographical, Education with tags , , on May 31, 2017 by telescoper

It’s always with a measure of trepidation that I look at the feedback that students give on a module that I’ve been teaching, and this nervousness is considerably enhanced when it’s the first time I’ve lectured that material. This morning I grasped the nettle and clicked on the link to take me to my questionnaire results for my module Physics of the Early Universe. I was relieved that it was all fairly positive.

In the old days these things were done on paper, which meant quite a big job collecting and collating the results. Nowadays it’s all done online, which means not receiving any drawings or other artistic contributions that some students were wont to scribble on the questionnaires. Past experience has been that the response rate is lower for on-line surveys, but the response rate I got this time was pretty high – over 80%. Perhaps students are getting more accustomed to doing everything on line?

I never find the numerical scores particularly useful as one has no idea how to calibrate them, but the textual comments made by students are often interesting and helpful. They’re all anonymous, of course, to encourage students to be frank.

One thing that clearly went down very well was the use of Cardiff’s new lecture-capture system (called Panopto), which allows the lecturer to record everything – powerpoint, data visualizer, whiteboard and live action – for posterity. I recorded all my lectures and exercise classes in toto and put them up on our Virtual Learning Environment (called Learning Central) for the students to view at their leisure. It’s a significantly more sophisticated and flexible lecture capture system than the one we used when I was at Sussex, and the questionnaire responses showed that the students really appreciated the availability of the recordings; a representative comment can be found below.

Not all my colleagues are keen on the idea of lecture capture, but I like it a lot and am very happy to do it with my own lectures. It does seem that some university staff are wary of this innovation, but opinion may be changing. Please let me know what you think via the poll thatr I’ve been running on this for a few years:

It’s always difficult when you give a new set of lectures judging the pace appropriately. I spent more time on introductory material than I should perhaps have done, and also – as a number of students made clear in the module evaluation – should have done some more worked examples. I’ll try do better next time, and I am very grateful to those who took the time to complete the survey pointing out how I might improve. I always take constructive criticism very seriously.

It is of course the negative comments that are the most helpful in a practical sense, but it is always nice to find comments like these:

The lecturer is very passionate about the subject and that really helps as you can ask any question and he’ll be able to answer it. Furthermore, his enthusiasm helps to keep you engaged. I also found it helpful that the lectures were recorded, so I could look over them while working on coursework.

Before you accuse me of doing so, I admit that I have cherry-picked one of the good ones to show myself in a good light.

I’m less sure how to interpret this one:

The lectures were incredible.

Anyway, the students on this module have now finished the exam and will be waiting for the results which come out in a couple of weeks. If any happen to be reading this blog then thanks for your comments and

Widening Participation Matters

Posted in Education with tags , , , , , on May 3, 2017 by telescoper

Time for a mini-rant about the failure of many universities to make any real attempt to make higher education more accessible to students from diverse backgrounds, especially those from underrepresented social groups.

I found this item on Twitter the other day. It’s from a local newspaper in York, and it was accompanied by an article that applauded the University of York (rightly) for being in the top three Russell Group universities for widening participation.

The list shows all 24 universities in the Russell Group, along with the fraction of their students that come from state schools and the fraction that come from geographical areas where participation in higher education is low; ‘POLAR3’ is the latest iteration of the Participation of Local Areas survey carried out by the Higher Education Funding Council for England (HEFCE).

I’m very pleased that my current employer, Cardiff University, leads the Russell Group by this measure, followed by the Universities of Liverpool and York, respectively.

What doesn’t please me is that so many of these institutions have such low participation rates from this group, and also such a small fraction of students who were educated at state schools. Over 90% of the total number of students at UK universities were educated at state schools, but the only Russell Group member to exceed 90% is Queens University, Belfast. However, the school system in Northern Ireland is very different from the rest of the UK, with relatively few private schools, so the situation there is not really comparable.

When tuition fees were increased in 2012, Universities were only allowed the charge the maximum (£9K per annum) if they produced an `Access Agreement‘  outlining measures to be introduced that would `improve access, student success and progression among people from under-represented and disadvantaged groups‘. The evidence of the last five years is that participation rates at many of the Russell Group institutions listed above have not changed at all. The reason for this is simple: the members of senior management at these institutions simply do not careabout widening participation.

I emphasize that it’s the members of senior management who don’t care because I honestly believe that the majority of academic staff in these institutions (and indeed the rest of the higher education sector) do care a very great deal. Nowadays however the gulf between academics and managers is greater than ever

Some time ago I was interviewed for a job in senior management at one of the institutions in the table above. During the course of the interview I was asked, among many other things, what I thought the University needed to do better. Without hesitation I said `widening participation’. The members of the panel stared at me as if I’d taken leave of my senses. The institution concerned was doing in well in league tables and recruiting students and saw no reason to try to make itself more open. When asked why I thought it was so important, I said I thought it was a moral responsibility. What I meant was that I think universities should be run for the public good, not just for the good of people who went to a posh school. That received even more uncomprehending stares than my original statement.

I didn’t get that job. I’m not saying it was because of the way I answered that question. I’m sure there were plenty of other reasons not to employ me, but that is the part of the interview I remember most vividly. I had prepared a list of ideas (including foundation programmes, measures to boost graduate employability, work placements, schools liaison, etc), some of which I’d borrowed from my (then) employer, the University of Sussex (which was – and is – very good at widening participation), but I had wasted my time. They weren’t interested.

The current system of ‘Access Agreements’ clearly isn’t working for these institutions and there is no effective sanction to force them even to try to broaden participation. Until there is, they will continue not to care.

Parliament has recently enacted the Higher Education and Research Act (2017). This presented a great chance to tackle the failures described above but, as far as I can see, none of the new arrangements is likely to do anything to widen participation in the so-called `elite’ universities, so it’s been a wasted opportunity.




Building Momentum Towards Inclusive Teaching and Learning

Posted in Education with tags , , on May 2, 2017 by telescoper

I’ve had a very full day back after the Bank Holiday (Long) Weekend so I only have time for a brief post today.

I was giving a revision lecture this morning so I wasn’t able to attend an event in London organized by the Institute of Physics to launch a new report with the title Building Momentum Towards Inclusive Teaching and Learning. It was a shame I couldn’t go, as I’m a member of the IOP Diversity and Inclusion Committee which oversaw this report, but the exam period is coming up and I couldn’t reschedule the lecture.

Anyway, to quote the IOP web page:

There are particular challenges in providing an inclusive learning environment in all the physical sciences and especially in physics, due to the wide range of activities involved, such as lab sessions, problem classes and fieldwork, and the use of mathematical and scientific notation. General good practice guidance on inclusive curricula do not normally contain specialist information on the particular accessibility challenges of courses with substantial mathematical content given its non-linear nature (ie the relative positioning of letters, symbols and numbers and their relative sizes) and the limitations of assistive technology in manipulating this content.

The Equalities Act (2010) requires HEIs to make `reasonable adjustments’ to make their courses accessible to disabled students, but there’s often no reason why these `adjustments’ should not simply be standard provision for all students. That’s what `inclusive’ means. If, for example, lecture recordings and/or printed notes are made available for students who have difficulty taking notes, then why not make them available for everyone? That’s what `inclusive’ actually means.

To quote again:

By moving towards a more inclusive learning environment many organisational, structural and cultural barriers to disabled students can be removed. The focus on inclusivity means that “individual interventions is the exception, not the rule” as set out in the Department for Education’s report Inclusive Teaching and Learning in Higher Education as a Route to Excellence. This requires all staff in higher education – academics, support staff and senior institutional managers – to consider the needs of disabled students in all that they do – including the design, delivery and assessment of all academic teaching and learning.

I therefore encourage anyone who’s involved in teaching physics to read this report, which you can download as a PDF file here, and think about its recommendations when you start to plan teaching activities, whatever form they take.