Archive for the Education Category

The Culture of Over-Assessment in STEM

Posted in Education with tags , , , , on December 7, 2016 by telescoper

This afternoon I went to yet another meeting about assessment and feedback in University teaching involving members of staff and students from the School of Physics & Astronomy here at Cardiff University as well as some people from other schools and departments. Positive though this afternoon’s discussion was, it didn’t do anything to dissuade me from a long-held view that the entire education system holds back the students’ ability to learn by assessing them far too much. This is a topic that I’ve blogged about a few times before over the years (see, e.g., here) but given that the problem hasn’t gone away (and indeed is probably going to get worse as a result of the Teaching Excellence Framework which the Westminster government is trying to impose on universities),  I make no apologies for repeating the main points here.

One important point we need to resolve to pin down essentially what is meant by “Research-led Teaching”, which is what we’re supposed to be doing at universities. In my view too much teaching is not really led by research at all, but mainly driven by assessment. The combination of the introduction of modular programmes and the increase of continuously assessed coursework has led to a cycle of partial digestion and regurgitation that involves little in the way of real learning and certainly nothing like the way research is done. I don’t know why we’ve got into this situation but it can’t be allowed to continue.

I’m not going to argue for turning the clock back entirely but, for the record, my undergraduate degree involved no continuous assessment at all (apart from a theory project I opted for in my final year. Having my entire degree result based on the results of six three-hour unseen examinations in the space of three days is not an arrangement I can defend, but note that despite the lack of continuous assessment I still spent less time in the examination hall than present-day students.

That’s not to say I didn’t have coursework. I did, but it was formative rather than summative; in other words it was for the student to learn about the subject, rather for the staff to learn about the student. I handed in my stuff every week, it was marked and annotated by a supervisor, then returned and discussed at a supervision.

People often tell me that if a piece of coursework “doesn’t count” then the students won’t do it. There is an element of truth in that, of course. But I had it drummed into me that the only way really to learn my subject (Physics) was by doing it. I did all the coursework I was given because I wanted to learn and I knew that was the only way to do it. I think we need to establish that as a basic principle of education in physics (and similar subjects).

The very fact that coursework didn’t count for assessment made the feedback written on it all the more useful when it came back because if I’d done badly I could learn from my mistakes without losing marks. This also encouraged me to experiment a little, such as using a method different from that suggested in the question. That’s a dangerous strategy nowadays, as many seem to want to encourage students to behave like robots, but surely we should be encouraging students to exercise their creativity rather than simply follow the instructions? The other side of this is that more challenging assignments can be set, without worrying about what the average mark will be or what specific learning outcome they address.

I suppose what I’m saying is that the idea of Learning for Learning’s Sake, which is what in my view defines what a university should strive for, is getting lost in a wilderness of modules, metrics, percentages and degree classifications. We’re focussing too much on those few aspects of the educational experience that can be measured, ignoring the immeasurable benefit (and pleasure) that exists for all of us humans in exploring new ways to think about the world around us.

R.I.P. John M Stewart (1943-2016)

Posted in Biographical, Education, The Universe and Stuff with tags , , , on November 23, 2016 by telescoper

john-stewartI was very sad this morning to hear of the death of distinguished mathematical physicist Dr John M. Stewart (left). Apart from a few years in Munich in the 1970s John Stewart spent most of his working life in Cambridge, having studied there as an undergraduate and postgraduate and then returning from his spell at the Max Planck Institute to the Department of Applied Mathematics and Theoretical Physics for forty years.

John’s research mostly concerned relativistic fluid dynamics. Indeed, he was one of the pioneers of numerical relativity in the United Kingdom, and he applied his knowledge to a number of problems in early Universe cosmology and structure formation. I think it is fair to say that he wasn’t the most prolific researcher in terms of publications, which is perhaps why he only got promoted to Senior Lecturer in 2000 and never made it to a Chair, retiring as Reader in Gravitational Physics in 2010. However, his work was always of a very high technical standard and presented with great clarity and he was held in a very high regard by those who knew him and worked with him.

The tributes paid to John Stewart by King’s College (of which he was a Life Fellow) here and his colleagues in the Centre for Theoretical Cosmology here give a detailed account of his research achievements, so I refer you to them for more information about that aspect of his career.

I just wanted to add a personal note not about John Stewart’s research, but about something else mentioned in the obituaries linked to above: his teaching. I was fortunate enough to have him as a lecturer when I was studying Natural Sciences at Cambridge during the early 1980s. In the second year (Part IB) I specialised in Physics and Mathematics, and John taught part of the Mathematics syllabus. He was an absolutely superb teacher. For a start he was superbly well organized and had clearly thought very deeply about how best to present some quite difficult material. But it wasn’t just that. He projected a very engaging personality, with nice touches of humour, that made him easy to listen. His lectures were also very well paced for taking notes. In fact he was one of the few lecturers I had whose material I didn’t have to transcribe into a neat form from rough notes.

I have kept all the notes from that course for over thirty years. Here are a couple of pages as an example:

wp-1479909754508.jpeg

Anyone who has ever seen my handwriting will know that this is about as neat as I ever get!

When I was called upon to teach similar material at Cardiff and Sussex I drew on them heavily, so anyone who has learned anything from me about complex analysis, contour integration, Green’s functions and a host of other things actually owes a huge debt to John Stewart. Anything they didn’t understand was of course my fault, not his..

I also remember that John came to Queen Mary to give a seminar when I worked there in the early 90s as a postdoc. I was still a bit in awe of him because of my experience of him in Cambridge. His talk was about a method for handling the evolution of cosmological matter perturbations based on an approach based on the Hamilton-Jacobi formalism. His visit was timely, as I’d been struggling to understand the papers that had been coming out at the time on this topic. In the bar after his talk I plucked up the courage to explain to him what it was that I was struggling to understand. He saw immediately where I was going wrong and put me right on my misconceptions straight away, plucking a simple illustrative example apparently out of thin air. I was deeply impressed, not only by his ability to identify the issue but also with his friendly and helpful demeanour.

Rest in Peace, Dr John M. Stewart (1943-2016).

50 Years of the Astronomy Centre at the University of Sussex

Posted in Biographical, Education, The Universe and Stuff with tags , , , , , on October 18, 2016 by telescoper

On Saturday (15th October) I was back in Brighton for the first time since I left my job there at the end of July. The occasion was a very nice lunch party to celebrate 50 years of the Astronomy Centre at the University of Sussex, which started properly in 1966. It was a pleasant occasion, and great to have the chance to catch up with some people I haven’t seen for far too long. I had two stints in the Astronomy Centre: once as a student then postdoc from 1985 to 1990, and the other from 2013 to 2016 when I was Head of the School of which the Astronomy Centre is part. I had a lot more time to do research in the first incarnation than in the second!

Quite a few people present hadn’t realised I was no longer working at Sussex, which led to one or two slightly awkward conversations, but I was thankfully very far from being the centre of attention.

After the lunch itself we had short speeches from various alumni of the Astronomy Centre: esteemed science writer John Gribbbin (who was one of its first MSc students in 1966); Lord Martin Rees (who was briefly a Professor at Sussex, before he returned to Cambridge to take up the Plumian Professorship); John Barrow (who was my supervisor while I was there); Carlos Frenk (who was a postdoctoral researcher when I arrived in September 1985, but who left to take up a lectureship in Durham at the end of that year so we overlapped only for a short time); Andrew Liddle (who arrived near the end of my stay and was there for 22 years altogether, leaving at the end of 2012 to take up a post in Edinburgh); and Peter Thomas (current Director of the Astronomy Centre).

When I arrived in 1985 there were only four permanent faculty in the Astronomy Centre itself – Roger Tayler, Leon Mestel, John Barrow and Robert Smith – but research there was thriving and it was a great environment to work in. I count myself very lucky at having made such a good choice of a place to do my PhD DPhil. Leon and Robert both worked on stellar astrophysics, but after Leon’s retirement the centre increasingly focussed on cosmology and extragalactic astrophysics, which remains the case today. Roger Tayler sadly passed away in 1997, but Leon is still around: he is 89 years old and now lives in Cambridge.

Those present at the lunch were given a booklet featuring around 50 academic papers or other research “highlights”(e.g. the launch of Planck), approximately one for each year of the Astronomy Centre, chosen to be the “best” of that year. Each page was also shown as a slide during the lunch. I was thrilled to see that two of my papers (from 1987 and 1991 respectively) made it into the collection. The second one was published after I’d left Sussex, but I definitely did the work on it and submitted it while an employee of the Astronomy Centre. Andrew Liddle and John Barrow have the largest number of “greatest hits”, but the most famous paper is probably the classic “DEFW” which won Carlos Frenk and his collaborators the Gruber Prize about five years ago.

The book also contains various bits of interesting bibliometric information, such as this, which shows that the variation in the productivity of the Astronomy Centre over time.

us-astronomy-50-powerpoint

Anyway, for those who are interested, the whole collection of slides can be viewed here:

Thanks to Seb Oliver and the rest of the Astronomy Centre for organizing this very enjoyable event – and for sending me the slides! Here’s to the next 50 years of Astronomy at the University of Sussex!

 

Thinking of Applying for a PhD Place in Physics or Astronomy?

Posted in Education with tags , , , , , , on October 12, 2016 by telescoper

This morning I am to give a short talk to interested students within the School of Physics & Astronomy here at Cardiff University about postgraduate research in which I aim to pass on some, hopefully useful,  information about how to go about applying for PhDs  in Physics  and Astronomy. Since I’ve finished writing the talk more than the usual few minutes before I have to deliver it, I thought I’d jot down here a few general remarks that might be useful to people elsewhere who are thinking of taking the plunge when they graduate. I’m aiming this primarily at UK students applying for places in the UK; special considerations apply for students wanting to do graduate research abroad.

What is a PhD? The answer to that is relatively easy; it’s a postgraduate research degree. In order to obtain a PhD you have to present a thesis like that shown on the left (which happens to be mine, vintage 1988), typically in the range 100-250  pages long. A thesis has to satisfy two conditions for the award of the degree: it should contain original research, which is publishable in an academic journal; and it should present a coherent discussion of that original work within the context of ongoing work in the area of study. In Physics & Astronomy, the PhD is pretty much a prerequisite for any career in academic research, and it usually takes between 3 and 4 years to complete. After submission of the thesis you will have to undergo a viva voce examination conducted by two examiners, one internal and one external. This is quite a tough test, which  can last anywhere between about 2 and about 6 hours, during which you can be asked  detailed questions about your research and wide-ranging questions about the general area.

The Money Side. In the UK most PhDs are supported financially by the research councils, either EPSRC (most physics) or STFC (nuclear & particle physics, astronomy). These generally award quotas of studentships to departments who distribute them to students they admit. A studentship will cover your fees and pay a stipend, currently £14296 pa. That doesn’t sound like a lot, but you should at least remember that it is a stipend rather than a wage; it is therefore not taxed and there is no national insurance payable. There is a fee (currently £4121) payable for a PhD course, but that only comes into play if you are planning to fund yourself. If you receive a studentship it will normally cover the fee as an additional component. What I mean by that is you don’t need to pay it out of the stipend, it is separate. In top of that, research council funding also supplies a Research Training Grant which covers, e.g., travel and small items of equipment so you don’t need to pay for those out of your stipend either.

How do I choose a PhD? During the course of a postgraduate degree you are expected to become an expert in the area in which you specialize. In particular you should reach the point where you know more about that specific topic than your supervisor does. You will therefore have to work quite a lot on your own, which means you need determination, stamina and enthusiasm. In my view the most important criterion in your choice of PhD is not the institution where you might study but the project. You need to be genuinely excited by the topic in order to drive yourself to keep through the frustrations (of which there will be many). So, find an area that interests you and find the departments that do active research in that area by looking on the web. Check out the recent publications by staff in each department, to ensure that they are active and to have something to talk about at interview!

Qualifications. Most universities have a formal requirement that candidates for admission to the PhD should have a “good honours degree”, which basically means at least an Upper Second Class Honours degree. Some areas are more competitive than others, however, and in many disciplines you will find you are competing with a great many applicants with First Class degrees.

How to apply successfully. The application procedure at most universities is quite simple and can be done online. You will need to say something about the area in which you wish to do research (e.g. experiment/theory, and particular field, e.g. cosmology or star formation). You’ll also need a CV and a couple of references. Given the competition, it’s essential that you prepare. Give your curriculum vitae some attention, and get other people (e.g. your personal tutor) to help you improve it. It’s worth emphasizing particular skills (e.g. computing). If you get the chance, make use of your summer vacations by taking on an internship or other opportunity to get a taste of research; things like that will undoubtedly give your CV an edge.

The Interview. Good applicants will be invited for an interview, which is primarily to assess whether you have the necessary skills and determination, but also to match applicants to projects and supervisors. Prepare for your interview! You will almost certainly be asked to talk about your final-year project, so it will come across very badly if you’re not ready when they ask you. Most importantly, mug up about your chosen field. You will look really silly if you haven’t the vaguest idea of what’s going on in the area you claimed to be interested in when you wrote your  application!

Don’t be shy! There’s nothing at all wrong with being pro-active about this process. Contact academic staff at other universities by email and ask them about research, PhD opportunities. That will make a good impression. Also, don’t be afraid to ask for advice. Although we’re all keen to recruit good PhD students for our own departments, we academics are  conscious that it is also our job to give impartial advice. Ask your tutor’s opinion.

How many places should I apply for? Some research areas are more fashionable than others so the level of competition varies with field. As a general rule I would advise applying for about half-a-dozen places, chosen because they offer research in the right area. Apply to fewer than that and you might lose out to the competition. Apply to many more and you might not have time to attend the interviews.

What’s the timetable?  Most applications come in early in the new year for entry to the PhD in the following September/October. The Christmas break is therefore a pretty good time to get your applications sorted out. Interviews are normally held in February or March, and decisions made by late March. STFC runs a deadline system whereby departments can not force students to accept or decline offers before the end of March, so there should be ample time to visit all your prospective departments before having to make any decisions.

That’s all I can think of for now. I hope at least some of these comments are useful to undergraduates anywhere in the UK thinking of applying for a PhD. If there are any further questions, please feel free to ask through the comments box. Likewise if I’ve missed anything important, please feel free to suggest additions in the same manner…

The Worthless University Rankings

Posted in Bad Statistics, Education with tags , , , on September 23, 2016 by telescoper

The Times Higher World University Rankings, which were released this weekk. The main table can be found here and the methodology used to concoct them here.

Here I wish to reiterate the objection I made last year to the way these tables are manipulated year on year to create an artificial “churn” that renders them unreliable and impossible to interpret in an objective way. In other words, they’re worthless. This year, editor Phil Baty has written an article entitled Standing still is not an option in which he makes a statement that “the overall rankings methodology is the same as last year”. Actually it isn’t. In the page on methodology you will find this:

In 2015-16, we excluded papers with more than 1,000 authors because they were having a disproportionate impact on the citation scores of a small number of universities. This year, we have designed a method for reincorporating these papers. Working with Elsevier, we have developed a new fractional counting approach that ensures that all universities where academics are authors of these papers will receive at least 5 per cent of the value of the paper, and where those that provide the most contributors to the paper receive a proportionately larger contribution.

So the methodology just isn’t “the same as last year”. In fact every year that I’ve seen these rankings there’s been some change in methodology. The change above at least attempts to improve on the absurd decision taken last year to eliminate from the citation count any papers arising from large collaborations. In my view, membership of large world-wide collaborations is in itself an indicator of international research excellence, and such papers should if anything be given greater not lesser weight. But whether you agree with the motivation for the change or not is beside the point.

The real question is how can we be sure that any change in league table position for an institution from year to year are is caused by methodological tweaks rather than changes in “performance”, i.e. not by changes in the metrics but by changes in the way they are combined? Would you trust the outcome of a medical trial in which the response of two groups of patients (e.g. one given medication and the other placebo) were assessed with two different measurement techniques?

There is an obvious and easy way to test for the size of this effect, which is to construct a parallel set of league tables, with this year’s input data but last year’s methodology, which would make it easy to isolate changes in methodology from changes in the performance indicators. The Times Higher – along with other purveyors of similar statistical twaddle – refuses to do this. No scientifically literate person would accept the result of this kind of study unless the systematic effects can be shown to be under control. There is a very easy way for the Times Higher to address this question: all they need to do is publish a set of league tables using, say, the 2015/16 methodology and the 2016/17 data, for comparison with those constructed using this year’s methodology on the 2016/17 data. Any differences between these two tables will give a clear indication of the reliability (or otherwise) of the rankings.

I challenged the Times Higher to do this last year, and they refused. You can draw your own conclusions about why.

Rank Nonsense

Posted in Bad Statistics, Education, Politics with tags , , , , , on September 8, 2016 by telescoper

It’s that time of year when international league tables (also known as “World Rankings”)  appear. We’ve already had the QS World University Rankings and the Shanghai (ARWU) World University Rankings. These will soon be joined by the Times Higher World Rankings, due out on 21st September.

A lot of people who should know a lot better give these league tables far too much attention. As far as I’m concerned they are all constructed using extremely suspect methodologies whose main function is to amplify small statistical variations into something that looks significant enough to justify constructing  a narrative about it. The resulting press coverage usually better reflects a preconceived idea in a journalist’s head than any sensible reading of the tables themselves.

A particularly egregious example of this kind of nonsense can be found in this week’s Guardian. The offending article is entitled “UK universities tumble in world rankings amid Brexit concerns”. Now I make no secret of the fact that I voted “Remain” and that I do think BrExit (if it actually happens) will damage UK universities (as well as everything else in the UK). However, linking the changes in the QS rankings to BrExit is evidently ridiculous: all the data were collected before the referendum on 23rd June anyway! In my opinion there are enough good arguments against BrExit without trying to concoct daft ones.

In any case these tables do not come with any estimate of the likely statistical variation from year to year in the metrics used to construct them, which makes changes impossible to interpret. If only the compilers of these tables would put error bars on the results! Interestingly, my former employer, the University of Sussex, has held its place exactly in the QS rankings between 2015 and 2016: it was ranked 187th in the world in both years. However, the actual score corresponding to these two years was 55.6 in 2015 and 48.4 in 2016. Moreover, Cambridge University fell from 3rd to 4th place this year but its score only changed from 98.6 to 97.2. I very much doubt that is significant at all, but it’s mentioned prominently in the subheading of the Guardian piece:

Uncertainty over research funding and immigration rules blamed for decline, as Cambridge slips out of top three for first time.

Actually, looking closer, I find that Cambridge was joint 3rd in 2015 and is 4th this year. Over-interpretation, or what?

To end with, I can’t resist mentioning that the University of Sussex is in the top 150 in the Shanghai Rankings for Natural and Mathematical Sciences this year, having not been in the top 200 last year. This stunning improvement happened while I was Head of School for Mathematical and Physical Sciences so it clearly can not be any kind of statistical fluke but is entirely attributable to excellent leadership. Thank you for your applause.

 

 

Worrying Times for UK Physics

Posted in Education, The Universe and Stuff with tags , on August 19, 2016 by telescoper

As I’m more-or-less in between jobs at the moment, this is the first August in many that I haven’t been involved the clearing and confirmation process that helps students find places at university after the A-level results are released. I know how stressful it is for admissions staff and prospective students alike, so I’m not sorry to be out of it for once!

On the other hand I did notice something worrying that seems to be the continuation of a trend I noticed last year.  I quote from a piece issued by the Institute of Physics about the number of students taking A-level physics last year:

Although there was an overall rise of 2% in the number of A-level entries, the number taking physics fell to 36,287 compared with 36,701 last year – the first time numbers have fallen since 2006. The number of girls taking physics rose by 0.5%, however.

That decline is slight, of course, and it was  obviously too early to decide whether it indicated whether or not the UK has reached “Peak Physics”. Well, this year has confirmed that trend. According to a piece by the Wellcome Trust the number of entrants for physics A-level has fallen further this year, from 36,287 in 2015 to 35,344 in 2016. The Institute of Physics has also commented.

Virtually all students who get a Physics A-level do go to university, but by no means all do physics. It is also a qualifying subject for engineering and technology programmes, as well as medicine. It’s not clear yet whether the decline in A-level entry reflects a decline in the number of students going to start physics degrees at University this year, but this seems probable. This is good news if you’re an applicant with a Physics A-level, of course, because it increases the chances of you getting a place, but it’s no so good for physics as academic discipline.

Physics departments in UK universities are already competing for a very small pool of students with a Physics A-level.  The removal of student number controls allows  large universities to recruit as many students as they like, so the competition between universities for such a small number of applicants is extremely intense. Moreover, some universities, e.g. Newcastle and Hull, have opened up physics courses that they had previously closed, and others have started  new programmes based on what was anticipated to be an overall increase in demand. To support this expansion, many institutions have recruited extra numbers of teaching faculty assuming the salary costs would be covered from tuition fees. If the decline in overall student numbers continues then the budgets of many physics departments are going to look pretty grim, with potentially serious  consequences for the long-term sustainability of physics in many institutions.

I have to confess I’m worried. The physics community urgently needs to find out what is behind this fall. It’s not restricted to physics, in fact. Both biology and chemistry have also experienced a decline in the number of A-level entrants (from 44,864 to 43,242 and from 52,644 to 51,811 respectively), but the effect on physics is likely to be greater for the reasons I discussed above.

Mathematics numbers have also fallen, but by a much smaller percentage and from a much higher level: from 92,711 to 92,163.  I‘ve argued before that there’s a case on a number of grounds for scrapping the physics A-level as a requirement for entry to university as long as the student has mathematics. That may be a step too far for some, but it’s clear that if physics is to prosper we all have to think more creatively about how to increase participation. But how? Answers on a postcard – or through the comments box – please!