Archive for the Science Politics Category

A scientific paper with 5000 authors is absurd, but does science need “papers” at all?

Posted in History, Open Access, Science Politics, The Universe and Stuff with tags , , , , , , , , , on May 17, 2015 by telescoper

Nature News has reported on what appears to be the paper with the longest author list on record. This article has so many authors – 5,154 altogether – that 24 pages (out of a total of 33 in the paper) are devoted just to listing them, and only 9 to the actual science. Not, surprisingly the field concerned is experimental particle physics and the paper emanates from the Large Hadron Collider; it involves combining data from the CMS and ATLAS detectors to estimate the mass of the Higgs Boson. In my own fields of astronomy and cosmology, large consortia such as the Planck collaboration are becoming the exception rather than the rule for observational work. Large ollaborations  have achieved great things not only in physics and astronomy but also in other fields. A for  paper in genomics with over a thousand authors has recently been published and the trend for ever-increasing size of collaboration seems set to continue.

I’ve got nothing at all against large collaborative projects. Quite the opposite, in fact. They’re enormously valuable not only because frontier research can often only be done that way, but also because of the wider message they send out about the benefits of international cooperation.

Having said that, one thing these large collaborations do is expose the absurdity of the current system of scientific publishing. The existence of a paper with 5000 authors is a reductio ad absurdum proof  that the system is broken. Papers simply do not have 5000  “authors”. In fact, I would bet that no more than a handful of the “authors” listed on the record-breaking paper have even read the article, never mind written any of it. Despite this, scientists continue insisting that constributions to scientific research can only be measured by co-authorship of  a paper. The LHC collaboration that kicked off this piece includes all kinds of scientists: technicians, engineers, physicists, programmers at all kinds of levels, from PhD students to full Professors. Why should we insist that the huge range of contributions can only be recognized by shoe-horning the individuals concerned into the author list? The idea of a 100-author paper is palpably absurd, never mind one with fifty times that number.

So how can we assign credit to individuals who belong to large teams of researchers working in collaboration?

For the time being let us assume that we are stuck with authorship as the means of indicating a contribution to the project. Significant issues then arise about how to apportion credit in bibliometric analyses, e.g. through citations. Here is an example of one of the difficulties: (i) if paper A is cited 100 times and has 100 authors should each author get the same credit? and (ii) if paper B is also cited 100 times but only has one author, should this author get the same credit as each of the authors of paper A?

An interesting suggestion over on the e-astronomer a while ago addressed the first question by suggesting that authors be assigned weights depending on their position in the author list. If there are N authors the lead author gets weight N, the next N-1, and so on to the last author who gets a weight 1. If there are 4 authors, the lead gets 4 times as much weight as the last one.

This proposal has some merit but it does not take account of the possibility that the author list is merely alphabetical which actually was the case in all the Planck publications, for example. Still, it’s less draconian than another suggestion I have heard which is that the first author gets all the credit and the rest get nothing. At the other extreme there’s the suggestion of using normalized citations, i.e. just dividing the citations equally among the authors and giving them a fraction 1/N each. I think I prefer this last one, in fact, as it seems more democratic and also more rational. I don’t have many publications with large numbers of authors so it doesn’t make that much difference to me which you measure happen to pick. I come out as mediocre on all of them.

No suggestion is ever going to be perfect, however, because the attempt to compress all information about the different contributions and roles within a large collaboration into a single number, which clearly can’t be done algorithmically. For example, the way things work in astronomy is that instrument builders – essential to all observational work and all work based on analysing observations – usually get appended onto the author lists even if they play no role in analysing the final data. This is one of the reasons the resulting papers have such long author lists and why the bibliometric issues are so complex in the first place.

Having thousands of authors who didn’t write a single word of the paper seems absurd, but it’s the only way our current system can acknowledge the contributions made by instrumentalists, technical assistants and all the rest. Without doing this, what can such people have on their CV that shows the value of the work they have done?

What is really needed is a system of credits more like that used in the television or film. Writer credits are assigned quite separately from those given to the “director” (of the project, who may or may not have written the final papers), as are those to the people who got the funding together and helped with the logistics (production credits). Sundry smaller but still vital technical roles could also be credited, such as special effects (i.e. simulations) or lighting (photometic calibration). There might even be a best boy. Many theoretical papers would be classified as “shorts” so they would often be written and directed by one person and with no technical credits.

The point I’m trying to make is that we seem to want to use citations to measure everything all at once but often we want different things. If you want to use citations to judge the suitability of an applicant for a position as a research leader you want someone with lots of directorial credits. If you want a good postdoc you want someone with a proven track-record of technical credits. But I don’t think it makes sense to appoint a research leader on the grounds that they reduced the data for umpteen large surveys. Imagine what would happen if you made someone director of a Hollywood blockbuster on the grounds that they had made the crew’s tea for over a hundred other films.

Another question I’d like to raise is one that has been bothering me for some time. When did it happen that everyone participating in an observational programme expected to be an author of a paper? It certainly hasn’t always been like that.

For example, go back about 90 years to one of the most famous astronomical studies of all time, Eddington‘s measurement of the bending of light by the gravitational field of the Sun. The paper that came out from this was this one

A Determination of the Deflection of Light by the Sun’s Gravitational Field, from Observations made at the Total Eclipse of May 29, 1919.

Sir F.W. Dyson, F.R.S, Astronomer Royal, Prof. A.S. Eddington, F.R.S., and Mr C. Davidson.

Philosophical Transactions of the Royal Society of London, Series A., Volume 220, pp. 291-333, 1920.

This particular result didn’t involve a collaboration on the same scale as many of today’s but it did entail two expeditions (one to Sobral, in Brazil, and another to the Island of Principe, off the West African coast). Over a dozen people took part in the planning,  in the preparation of of calibration plates, taking the eclipse measurements themselves, and so on.  And that’s not counting all the people who helped locally in Sobral and Principe.

But notice that the final paper – one of the most important scientific papers of all time – has only 3 authors: Dyson did a great deal of background work getting the funds and organizing the show, but didn’t go on either expedition; Eddington led the Principe expedition and was central to much of the analysis;  Davidson was one of the observers at Sobral. Andrew Crommelin, something of an eclipse expert who played a big part in the Sobral measurements received no credit and neither did Eddington’s main assistant at Principe.

I don’t know if there was a lot of conflict behind the scenes at arriving at this authorship policy but, as far as I know, it was normal policy at the time to do things this way. It’s an interesting socio-historical question why and when it changed.

I’ve rambled off a bit so I’ll return to the point that I was trying to get to, which is that in my view the real problem is not so much the question of authorship but the idea of the paper itself. It seems quite clear to me that the academic journal is an anachronism. Digital technology enables us to communicate ideas far more rapidly than in the past and allows much greater levels of interaction between researchers. I agree with Daniel Shanahan that the future for many fields will be defined not in terms of “papers” which purport to represent “final” research outcomes, but by living documents continuously updated in response to open scrutiny by the community of researchers. I’ve long argued that the modern academic publishing industry is not facilitating but hindering the communication of research. The arXiv has already made academic journals virtually redundant in many of branches of  physics and astronomy; other disciplines will inevitably follow. The age of the academic journal is drawing to a close. Now to rethink the concept of “the paper”…

Tell Them Science Is Vital

Posted in Politics, Science Politics with tags on March 15, 2015 by telescoper

telescoper:

To follow up on my previous post, here’s a a lot more about the Tell Them Science Is Vital initiative…

Originally posted on Purely a figment of your imagination:

There’s another election coming up and, whatever the outcome, we’ll all (hopefully) still be doing our jobs and waiting for the situation to improve.

Something the government could do to lay foundations for education, industries and economic growth in the UK is to fund science. Over the last 2 decades, they’ve really let this slide. Enter #TellThemSiV, the new campaign from Science is Vital, to do just that…

Tell Them Science is Vital

In just a few weeks, Britain goes to the polls to vote in a new government. This is obviously a crucial time for science funding and policy.

That is why Science Is Vital needs you to contact your MP or parliamentary candidate.

Since 2010, the science budget, despite having been protected from the worst of the austerity measures by the ring-fence we fought for, will nevertheless have shrunk in real terms by up to 20%.

In…

View original 811 more words

Science Is Vital, So Don’t Let It Be Strangled.

Posted in Politics, Science Politics with tags , , on March 15, 2015 by telescoper

The General Election looming on the horizon could prove to be a watershed for scientific research in the United Kingdom. In the period immediately following the 2010 Election there was a great deal of nervousness about the possibility of huge cuts to spending on research. One of the most effective campaigns to persuade the new government against slashing funding for science on the grounds that scientific research was likely to be the principal fuel for any economic recovery was led by Science is Vital. I have written a few posts about this organisation.

The scientific community breathed a collective sigh of relief in autumn 2010 when the UK Government announced that research funding would be “ring-fenced” and maintained in cash terms for the duration of the Parliament. Things could have been far worse, as they have been in other parts of the public sector, but over the years the effect of inflation has been that this “flat cash” settlement involves a slow strangulation as opposed to a quick fall of the axe.

A recent piece in the Guardian includes this picture, which speaks for itself:

Science_spendingThe United Kingdom now spends less than 0.5% of its GDP on research, and this fraction is falling rapidly. We are now ranked last in the G8 by this criterion, way behind the USA and Germany. Why are we in this country so unbelievably miserly abou funding research? Other countries seem to recognize its important, so why can’t our politicians see it? We should be increasing our investment in science, not letting it wither away like this.

It seems to me that much more of this squeeze and we’ll be needing to close down major facilities and start withdrawing from important international collaborations. The Science and Technology Facilities Council (STFC) is particularly vulnerable, as such a large fraction of its budget is committed to long-term projects. It’s already trimmed funding for other activities to the bone, with research grants under particularly intense pressure. Will the ongoing Nurse Review of the Research Councils spell doom for STFC, as many of my colleagues think? Will be research funding  be transferred rom universities into research institutes?

Anyway, it seems an appropriate time to advertise the latest campaign from Science is Vital, which involves writing to candidates (including incumbent MPs) in your constituency to Tell Them That Science Is Vital. You might consider including some of the following, or others suggested by the website. If you’re a scientist, describe why your research is important. Here are some suggestions. If there is a local research institute in your constituency, explain how important it is to your local economy (how many people it employs, for example). If you’re a patient, or someone who cares for a patient, say how important you think research into that disease. Ask your candidate or MP to endorse the Science is Vital campaign to increase public funding of science to 0.8% of GDP. And if you do write, remember that the economic argument for investing research isn’t the only one…

SKA Matters

Posted in Science Politics, The Universe and Stuff with tags , , , , on March 13, 2015 by telescoper

There’s been quite a lot going on recently to do with the Square Kilometre Array (SKA), some of it scientific and some of it political, some of it good and some of it bad. At least those seem to me to be the appropriate descriptions.

First the scientific good news is the the SKA Board has decided which of the planned components of SKA should be constructed during the first phase, which has a budget of around €650M. Details can be found here but, in a nutshell, it seems that the SKA Survey Telescope, which was to be built around the existing pathfinder project ASKAP located in Australia is not going to be built in the first phase. This implies the low-frequency bit of SKA will be in Australia while the higher-frequency activities will be concentrated in South Africa. That seems a pragmatic decision to me based on the budgetary constraints and should lead to a lot of good science being done. At the very least it’s a clear decision.

This positive news has however been overshadowed by an unseemly spat over the choice of headquarters over the location of the SKA Headquarters which has culminated in a rather unhelpful story in Nature. SKA HQ has been temporarily housed at Jodrell Bank Observatory (in the Midlands) since 2012 and there are clearly some who would like it to be located there permanently. There is however a rival bid, from the historic Italian city of Padova, at whose university Galileo Galilei once lectured, and which remains one of the top universities in Italy.

I should put my cards on the table and say that I’ve enjoyed many visits to Padova in my career, starting when I was a PhD student back in the 1980s and have many fond memories of the place. The late co-author of my cosmology textbook, Francesco Lucchin was Director of the Department of Astronomy in Padova at that time. For many years Padova has been home to a large concentration of astronomers and is undoubtedly a centre of excellence. Moreover it is a city that is very well served by transport links, just a short distance from Venice so easily reachable by air, and also on a major railway line offering fast national and international services. It’s also a considerably better place to dine out than Jodrell Bank!

Specola

Padova’s astronomers are housed in the Castello Carrese which adjoins the Specola (above), a tower which was once a working astronomical observatory but, being right in the city centre, has not been useful for such purposes for many moons. When I first started going to Padova the Department of Astronomy was located in the tower and in some nearby buildings but the rest of the Castello Carrese was used as a prison. Now it’s been renovated and all the astronomers have been located there. I remember the frequent walks across the little bridge over the canal to a coffee bar where we often did some of our best research!

Given its strategically important location, Padova was bombed by Allied planes on a number of occasions in 1944 and 1945. My Italian colleagues would regularly draw my attention to the plaque near the entrance to the Specola pointing out that it was hit and badly damaged by Allied bombs during one raid.

Anyway I can certainly see the merits of locating SKA HQ in Padova but it’s not my decision to make. Those responsible have not yet made a final decision, but what’s sad is that a number of stories have been flying around in the media that imply that the UK is trying to exert undue political interference to stop SKA HQ being moved to Italy. Whether this is true or not I don’t know. As far as I’m concerned the powers that be are following proper process and that process has not yet been brought to a conclusion. Whatever the outcome, though, there’s no question that the language being used in the press coverage is very damaging. Let’s hope it can all be resolved amicably.

Now for a spot of lunch and then up to the Royal Astronomical Society where the topic of the Discussion Meeting is, somewhat ironically, Building an Open UK SKA-Science Consortium…

Uncertainty, Risk and Probability

Posted in Bad Statistics, Science Politics with tags , , , , , , , , on March 2, 2015 by telescoper

Last week I attended a very interesting event on the Sussex University campus, the Annual Marie Jahoda Lecture which was given this year by Prof. Helga Nowotny a distinguished social scientist. The title of the talk was A social scientist in the land of scientific promise and the abstract was as follows:

Promises are a means of bringing the future into the present. Nowhere is this insight by Hannah Arendt more applicable than in science. Research is a long and inherently uncertain process. The question is open which of the multiple possible, probable or preferred futures will be actualized. Yet, scientific promises, vague as they may be, constitute a crucial link in the relationship between science and society. They form the core of the metaphorical ‘contract’ in which support for science is stipulated in exchange for the benefits that science will bring to the well-being and wealth of society. At present, the trend is to formalize scientific promises through impact assessment and measurement. Against this background, I will present three case studies from the life sciences: assisted reproductive technologies, stem cell research and the pending promise of personalized medicine. I will explore the uncertainty of promises as well as the cunning of uncertainty at work.

It was a fascinating and wide-ranging lecture that touched on many themes. I won’t try to comment on all of them, but just pick up on a couple that struck me from my own perspective as a physicist. One was the increasing aversion to risk demonstrated by research funding agencies, such as the European Research Council which she helped set up but described in the lecture as “a clash between a culture of trust and a culture of control”. This will ring true to any scientist applying for grants even in “blue skies” disciplines such as astronomy: we tend to trust our peers, who have some control over funding decisions, but the machinery of control from above gets stronger every day. Milestones and deliverables are everything. Sometimes I think in order to get funding you have to be so confident of the outcomes of your research to that you have to have already done it, in which case funding isn’t even necessary. The importance of extremely speculative research is rarely recognized, although that is where there is the greatest potential for truly revolutionary breakthroughs.

Another theme that struck me was the role of uncertainty and risk. This grabbed my attention because I’ve actually written a book about uncertainty in the physical sciences. In her lecture, Prof. Nowotny referred to the definition (which was quite new to me) of these two terms by Frank Hyneman Knight in a book on economics called Risk, Uncertainty and Profit. The distinction made there is that “risk” is “randomness” with “knowable probabilities”, whereas “uncertainty” involves “randomness” with “unknowable probabilities”. I don’t like these definitions at all. For one thing they both involve a reference to “randomness”, a word which I don’t know how to define anyway; I’d be much happier to use “unpredictability”. Even more importantly, perhaps, I find the distinction between “knowable” and “unknowable” probabilities very problematic. One always knows something about a probability distribution, even if that something means that the distribution has to be very broad. And in any case these definitions imply that the probabilities concerned are “out there”, rather being statements about a state of knowledge (or lack thereof). Sometimes we know what we know and sometimes we don’t, but there are more than two possibilities. As the great American philosopher and social scientist Donald Rumsfeld (Shurely Shome Mishtake? Ed) put it:

“…as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – the ones we don’t know we don’t know.”

There may be a proper Bayesian formulation of the distinction between “risk” and “uncertainty” that involves a transition between prior-dominated (uncertain) and posterior-dominated (risky), but basically I don’t see any qualititative difference between the two from such a perspective.

Anyway, it was a very interesting lecture that differed from many talks I’ve attended about the sociology of science in that the speaker clearly understood a lot about how science actually works. The Director of the Science Policy Research Unit invited the Heads of the Science Schools (including myself) to dinner with the speaker afterwards, and that led to the generation of many interesting ideas about how we (I mean scientists and social scientists) might work better together in the future, something we really need to do.

Yes, science produces too many PhDs

Posted in Science Politics with tags , , , , , on February 19, 2015 by telescoper

I came across a blog post this morning entitled Does Science Produce Too Many PhDs? I think the answer is an obvious “yes” but I’ll use the question as an excuse to rehash an argument I have presented before, which is that most analyses of the problems facing yearly career researchers in science are looking at the issue from the wrong end. I think the crisis is essentially caused by the overproduction of PhDs in this field. To understand the magnitude of the problem, consider the following.

Assume that the number of permanent academic positions in a given field (e.g. astronomy) remains constant over time. If that is the case, each retirement (or other form of departure) from a permanent position will be replaced by one, presumably junior, scientist.

This means that over an academic career, on average, each academic will produce just one PhD who will get a permanent job in academia. This of course doesn’t count students coming in from abroad, or those getting faculty positions abroad, but in the case of the UK these are probably relatively small corrections.

Under the present supply of PhD studentships an academic can expect to get a PhD student at least once every three years or so. At a minimum, therefore, over a 30 year career one can expect to have ten PhD students. A great many supervisors have more PhD students than this, but this just makes the odds worse. The expectation is that only one of these will get a permanent job in the UK. The others (nine out of ten, according to my conservative estimate) above must either leave the field or the country to find permanent employment.

The arithmetic of this situation is a simple fact of life, but I’m not sure how many prospective PhD students are aware of it. There is still a reasonable chance of getting a first postdoctoral position, but thereafter the odds are stacked against them.

The upshot of this is we have a field of understandably disgruntled young people with PhDs but no realistic prospect of ever earning a settled living working in the field they have prepared for. This problem has worsened considerably in recent  years as the number of postdoctoral positions has almost halved since 2006. New PhDs have to battle it out with existing postdoctoral researchers for the meagre supply of suitable jobs. It’s a terrible situation.

Now the powers that be – in this case the Science and Technology Facilities Council – have consistently argued that the excess PhDs go out into the wider world and contribute to the economy with the skills they have learned. That may be true in a few cases. However, my argument is that the PhD is not the right way to do this because it is ridiculously inefficient.

What we should have is a system wherein we produce more and better trained Masters level students  and fewer PhDs. This is the system that exists throughout most of Europe, in fact, and the UK is actually committed to adopt it through the Bologna process.  Not that this commitment seems to mean anything, as precisely nothing has been done to harmonize UK higher education with the 3+2+3 Bachelors+Masters+Doctorate system Bologna advocates.

The training provided in a proper two-year Masters programme will improve the skills pool for the world outside academia, and also better prepare the minority of students who go on to take a PhD. The quality of the  PhD will also improve, as only the very best and most highly motivated researchers will take that path. This used to be what happened, of course, but I don’t think it is any longer the case.

The main problem with this suggestion is that it requires big changes to the way both research and teaching are funded. The research councils turned away from funding Masters training many years ago, so I doubt if they can be persuaded to to a U-turn now. Moreover, the Research Excellence Framework provides a strong incentive for departments to produce as many PhDs as they possibly can, as these are included in an algorithmic way as part of the score for “Research Environment”. The more PhDs a department produces, the higher it will climb in the league tables. One of my targets in my current position is to double the number of PhDs produced by my School over the period 2013-18. What happens to the people concerned seems not to be a matter worthy of consideration. They’re only “outputs”…

The Impact of Impact

Posted in Science Politics with tags , on February 18, 2015 by telescoper

telescoper:

Interesting analysis of the 2014 REF results by my colleague Seb Oliver. Among other things, it shows that Physics was the subject in which “Impact had the greatest impact”..

Originally posted on Seb Boyd:

 The Impact of Impact

I wrote the following article to explore how Impact in the Research Excellence Framework 2014 (REF2014) affected the average scores of departments (and hence rankings). This produced a “league table” of how strongly impact affected different subjects. Some of the information in this article was used in a THE article by Paul Jump due to come out 00:00 on 19th Feb 2015.  I’ve now also produced ranking tables for each UoA using the standardised weighting I advocate below (see Standardised Rankings).

UoA Unit of Assessment Effective Weight of GPA

ranking in each sub-profile as %

Outputs Impact Envir.
9 Physics 37.9 38.6 23.5
23 Sociology 34.1 38.6 27.3
10 Mathematical Sciences 37.6 37.5 24.9
24 Anthropology and Development Studies 40.2 35.0 24.8
6 Agriculture, Veterinary and Food Science 42.0 33.0 25.0
31 Classics 43.3 32.6 24.0
16 Architecture, Built Environment and Planning 48.6 31.1 20.3

View original 1,558 more words

Follow

Get every new post delivered to your Inbox.

Join 4,139 other followers