Archive for the Open Access Category

A scientific paper with 5000 authors is absurd, but does science need “papers” at all?

Posted in History, Open Access, Science Politics, The Universe and Stuff with tags , , , , , , , , , on May 17, 2015 by telescoper

Nature News has reported on what appears to be the paper with the longest author list on record. This article has so many authors – 5,154 altogether – that 24 pages (out of a total of 33 in the paper) are devoted just to listing them, and only 9 to the actual science. Not, surprisingly the field concerned is experimental particle physics and the paper emanates from the Large Hadron Collider; it involves combining data from the CMS and ATLAS detectors to estimate the mass of the Higgs Boson. In my own fields of astronomy and cosmology, large consortia such as the Planck collaboration are becoming the exception rather than the rule for observational work. Large ollaborations  have achieved great things not only in physics and astronomy but also in other fields. A for  paper in genomics with over a thousand authors has recently been published and the trend for ever-increasing size of collaboration seems set to continue.

I’ve got nothing at all against large collaborative projects. Quite the opposite, in fact. They’re enormously valuable not only because frontier research can often only be done that way, but also because of the wider message they send out about the benefits of international cooperation.

Having said that, one thing these large collaborations do is expose the absurdity of the current system of scientific publishing. The existence of a paper with 5000 authors is a reductio ad absurdum proof  that the system is broken. Papers simply do not have 5000  “authors”. In fact, I would bet that no more than a handful of the “authors” listed on the record-breaking paper have even read the article, never mind written any of it. Despite this, scientists continue insisting that constributions to scientific research can only be measured by co-authorship of  a paper. The LHC collaboration that kicked off this piece includes all kinds of scientists: technicians, engineers, physicists, programmers at all kinds of levels, from PhD students to full Professors. Why should we insist that the huge range of contributions can only be recognized by shoe-horning the individuals concerned into the author list? The idea of a 100-author paper is palpably absurd, never mind one with fifty times that number.

So how can we assign credit to individuals who belong to large teams of researchers working in collaboration?

For the time being let us assume that we are stuck with authorship as the means of indicating a contribution to the project. Significant issues then arise about how to apportion credit in bibliometric analyses, e.g. through citations. Here is an example of one of the difficulties: (i) if paper A is cited 100 times and has 100 authors should each author get the same credit? and (ii) if paper B is also cited 100 times but only has one author, should this author get the same credit as each of the authors of paper A?

An interesting suggestion over on the e-astronomer a while ago addressed the first question by suggesting that authors be assigned weights depending on their position in the author list. If there are N authors the lead author gets weight N, the next N-1, and so on to the last author who gets a weight 1. If there are 4 authors, the lead gets 4 times as much weight as the last one.

This proposal has some merit but it does not take account of the possibility that the author list is merely alphabetical which actually was the case in all the Planck publications, for example. Still, it’s less draconian than another suggestion I have heard which is that the first author gets all the credit and the rest get nothing. At the other extreme there’s the suggestion of using normalized citations, i.e. just dividing the citations equally among the authors and giving them a fraction 1/N each. I think I prefer this last one, in fact, as it seems more democratic and also more rational. I don’t have many publications with large numbers of authors so it doesn’t make that much difference to me which you measure happen to pick. I come out as mediocre on all of them.

No suggestion is ever going to be perfect, however, because the attempt to compress all information about the different contributions and roles within a large collaboration into a single number, which clearly can’t be done algorithmically. For example, the way things work in astronomy is that instrument builders – essential to all observational work and all work based on analysing observations – usually get appended onto the author lists even if they play no role in analysing the final data. This is one of the reasons the resulting papers have such long author lists and why the bibliometric issues are so complex in the first place.

Having thousands of authors who didn’t write a single word of the paper seems absurd, but it’s the only way our current system can acknowledge the contributions made by instrumentalists, technical assistants and all the rest. Without doing this, what can such people have on their CV that shows the value of the work they have done?

What is really needed is a system of credits more like that used in the television or film. Writer credits are assigned quite separately from those given to the “director” (of the project, who may or may not have written the final papers), as are those to the people who got the funding together and helped with the logistics (production credits). Sundry smaller but still vital technical roles could also be credited, such as special effects (i.e. simulations) or lighting (photometic calibration). There might even be a best boy. Many theoretical papers would be classified as “shorts” so they would often be written and directed by one person and with no technical credits.

The point I’m trying to make is that we seem to want to use citations to measure everything all at once but often we want different things. If you want to use citations to judge the suitability of an applicant for a position as a research leader you want someone with lots of directorial credits. If you want a good postdoc you want someone with a proven track-record of technical credits. But I don’t think it makes sense to appoint a research leader on the grounds that they reduced the data for umpteen large surveys. Imagine what would happen if you made someone director of a Hollywood blockbuster on the grounds that they had made the crew’s tea for over a hundred other films.

Another question I’d like to raise is one that has been bothering me for some time. When did it happen that everyone participating in an observational programme expected to be an author of a paper? It certainly hasn’t always been like that.

For example, go back about 90 years to one of the most famous astronomical studies of all time, Eddington‘s measurement of the bending of light by the gravitational field of the Sun. The paper that came out from this was this one

A Determination of the Deflection of Light by the Sun’s Gravitational Field, from Observations made at the Total Eclipse of May 29, 1919.

Sir F.W. Dyson, F.R.S, Astronomer Royal, Prof. A.S. Eddington, F.R.S., and Mr C. Davidson.

Philosophical Transactions of the Royal Society of London, Series A., Volume 220, pp. 291-333, 1920.

This particular result didn’t involve a collaboration on the same scale as many of today’s but it did entail two expeditions (one to Sobral, in Brazil, and another to the Island of Principe, off the West African coast). Over a dozen people took part in the planning,  in the preparation of of calibration plates, taking the eclipse measurements themselves, and so on.  And that’s not counting all the people who helped locally in Sobral and Principe.

But notice that the final paper – one of the most important scientific papers of all time – has only 3 authors: Dyson did a great deal of background work getting the funds and organizing the show, but didn’t go on either expedition; Eddington led the Principe expedition and was central to much of the analysis;  Davidson was one of the observers at Sobral. Andrew Crommelin, something of an eclipse expert who played a big part in the Sobral measurements received no credit and neither did Eddington’s main assistant at Principe.

I don’t know if there was a lot of conflict behind the scenes at arriving at this authorship policy but, as far as I know, it was normal policy at the time to do things this way. It’s an interesting socio-historical question why and when it changed.

I’ve rambled off a bit so I’ll return to the point that I was trying to get to, which is that in my view the real problem is not so much the question of authorship but the idea of the paper itself. It seems quite clear to me that the academic journal is an anachronism. Digital technology enables us to communicate ideas far more rapidly than in the past and allows much greater levels of interaction between researchers. I agree with Daniel Shanahan that the future for many fields will be defined not in terms of “papers” which purport to represent “final” research outcomes, but by living documents continuously updated in response to open scrutiny by the community of researchers. I’ve long argued that the modern academic publishing industry is not facilitating but hindering the communication of research. The arXiv has already made academic journals virtually redundant in many of branches of  physics and astronomy; other disciplines will inevitably follow. The age of the academic journal is drawing to a close. Now to rethink the concept of “the paper”…

BICEP2 bites the dust.. or does it?

Posted in Bad Statistics, Open Access, Science Politics, The Universe and Stuff with tags , , , , , , , , on September 22, 2014 by telescoper

Well, it’s come about three weeks later than I suggested – you should know that you can never trust anything you read in a blog – but the long-awaited Planck analysis of polarized dust emission from our Galaxy has now hit the arXiv. Here is the abstract, which you can click on to make it larger:

PlanckvBICEP2

My twitter feed was already alive with reactions to the paper when I woke up at 6am, so I’m already a bit late on the story, but I couldn’t resist a quick comment or two.

The bottom line is of course that the polarized emission from Galactic dust is much larger in the BICEP2 field than had been anticipated in the BICEP2 analysis of their data (now published  in Physical Review Letters after being refereed). Indeed, as the abstract states, the actual dust contamination in the BICEP2 field is subject to considerable statistical and systematic uncertainties, but seems to be around the same level as BICEP2’s claimed detection. In other words the Planck analysis shows that the BICEP2 result is completely consistent with what is now known about polarized dust emission.  To put it bluntly, the Planck analysis shows that the claim that primordial gravitational waves had been detected was premature, to say the least. I remind you that the original  BICEP2 result was spun as a ‘7σ’ detection of a primordial polarization signal associated with gravitational waves. This level of confidence is now known to have been false.  I’m going to resist (for the time being) another rant about p-values

Although it is consistent with being entirely dust, the Planck analysis does not entirely kill off the idea that there might be a primordial contribution to the BICEP2 measurement, which could be of similar amplitude to the dust signal. However, identifying and extracting that signal will require the much more sophisticated joint analysis alluded to in the final sentence of the abstract above. Planck and BICEP2 have differing strengths and weaknesses and a joint analysis will benefit from considerable complementarity. Planck has wider spectral coverage, and has mapped the entire sky; BICEP2 is more sensitive, but works at only one frequency and covers only a relatively small field of view. Between them they may be able to identify an excess source of polarization over and above the foreground, so it is not impossible that there may a gravitational wave component may be isolated. That will be a tough job, however, and there’s by no means any guarantee that it will work. We will just have to wait and see.

In the mean time let’s see how big an effect this paper has on my poll:

 

 

Note also that the abstract states:

We show that even in the faintest dust-emitting regions there are no “clean” windows where primordial CMB B-mode polarization could be measured without subtraction of dust emission.

It is as I always thought. Our Galaxy is a rather grubby place to live. Even the windows are filthy. It’s far too dusty for fussy cosmologists, who need to have everything just so, but probably fine for astrophysicists who generally like mucking about and getting their hands dirty…

This discussion suggests that a confident detection of B-modes from primordial gravitational waves (if there is one to detect) may have to wait for a sensitive all-sky experiment, which would have to be done in space. On the other hand, Planck has identified some regions which appear to be significantly less contaminated than the BICEP2 field (which is outlined in black):

Quieter dust

Could it be possible to direct some of the ongoing ground- or balloon-based CMB polarization experiments towards the cleaner (dark blue area in the right-hand panel) just south of the BICEP2 field?

From a theorist’s perspective, I think this result means that all the models of the early Universe that we thought were dead because they couldn’t produce the high level of primordial gravitational waves detected by BICEP2 have no come back to life, and those that came to life to explain the BICEP2 result may soon be read the last rites if the signal turns out to be predominantly dust.

Another important thing that remains to be seen is the extent to which the extraordinary media hype surrounding the announcement back in March will affect the credibility of the BICEP2 team itself and indeed the cosmological community as a whole. On the one hand, there’s nothing wrong with what has happened from a scientific point of view: results get scrutinized, tested, and sometimes refuted.  To that extent all this episode demonstrates is that science works.  On the other hand most of this stuff usually goes on behind the scenes as far as the public are concerned. The BICEP2 team decided to announce their results by press conference before they had been subjected to proper peer review. I’m sure they made that decision because they were confident in their results, but it now looks like it may have backfired rather badly. I think the public needs to understand more about how science functions as a process, often very messily, but how much of this mess should be out in the open?

 

UPDATE: Here’s a piece by Jonathan Amos on the BBC Website about the story.

ANOTHER UPDATE: Here’s the Physics World take on the story.

ANOTHER OTHER UPDATE: A National Geographic story

Elsevier in Australia

Posted in Open Access with tags , on May 23, 2014 by telescoper

telescoper:

More on open access, this time from the perspective of an Australian Mathematician, pointing out that the idea of Gold Open Access Hybrid Journals touted by some publishers is nothing but a scam. I won’t mention any names of course but Elsevier springs to mind.

Originally posted on Secret Blogging Seminar:

I’ve just got back from talking to Roxanne Missingham, the University Librarian here at ANU, about Elsevier, and I want to quickly report on what I learnt.

I don’t yet have any of the juicy numbers revealing what libraries are paying for their Elsevier subscriptions (as Timothy Gowers has been doing in the UK; if you haven’t read his post do that first!). Nevertheless there are some interesting details.

Essentially all the Australian universities, excepting a few tiny private institutes, subscribe to the Freedom collection (this is the same bundle that nearly everyone is forced into subscribing to). The contracts are negotiated by CAUL (the Council of Australian University Librarians).

My librarian was very frank about Article Processing Charges (APCs) constituting double-dipping, whatever it is that Elsevier and the other publishers say. The pricing of journal bundles is so opaque, and to the extent we understand it primarily based…

View original 356 more words

Elsevier Journals – The QMUL Figure

Posted in Open Access with tags , on May 18, 2014 by telescoper

telescoper:

More information on the scandalous cost of academic journals.

How would you feel if your institution paid over half a million pounds a year to Elsevier?

Originally posted on Edward F Hughes:

A few weeks ago I reblogged Tim Gowers’ post about the cost of Elsevier journals. I noticed that my own institution (QMUL) had deflected his Freedom of Information request. Curious to learn more, I did some digging.

It turns out that QMUL paid a total of £545,306.93 to Elsevier for the academic year 2013/14. Interestingly this is more than other universities that joined the Russell Group recently. However it’s still much cheaper than the bill Cambridge, UCL, Imperial or Edinburgh face.

Unfortunately QMUL weren’t able to provide any further breakdown of the figures. Apparently this information isn’t available to the university, which seems like a very odd way of doing business. I think it likely that the vast majority of the cost is the subscription fee.

I should point out that QMUL and Cambridge certainly have differentiated access to Elsevier journals. For example QMUL Library does not have access to Science Direct…

View original 214 more words

The Open Journal for Astrophysics Project

Posted in Open Access with tags , , , , on May 10, 2014 by telescoper

I owe many people various apologies for not posting anything for a while about the Open Journal for Astrophysics. For a start I have to admit that the call for test submissions last year was a bit premature. I should have been more patient and ensured that the system was complete before going public. I hope nobody has been too seriously inconvenienced by the ongoing delay.

The project has got stalled a bit largely because I have just had too many things to do to devote enough time to complete the final stages needed to go fully live and also one of the people helping with the project Arfon Smith moved to a new job. Arfon and Chris Lintott have posted an account of the project so far which gives a bit more detail about how we wanted to realize the project (hosted by GitHub); the code development has involved major work by Robert Simpson and Stuart Lynn in addition to Arfon and Chris.  In essence they say that the job is now about 80% complete. I would have said it was more like 75%, so the OJFA is in some sense just the OJF at the moment! Much of what remains is not actual programming stuff but administrative stuff involved with, e.g., arranging the assignement of  digital object identifiers (DOIs) and so on, all of which has been on my to-do list for several months now.

Anywhere, just to show you that the whole project isn’t just hot air here is a demonstration of the snazzy user interface which we plan to use to facilitate the online refereeing process:

However, in the spirit not only of open access publishing but also of open source programming, Arfon has made available all the codes that have been developed so far. One intention of this is that  these can be adapted  for other OJFs hence the construction of a generic website (theoj.org) as well as the hope that some folks out there might help us bright the OJFA itself to completion. Anyone out there with the requisite skills is welcome to volunteer, either through the comments box here or through the OJ repository. If we can get enough volunteers we can meet and put together a plan to bring this idea to completion at last.

Despite being forced to accept that my own workload makes it difficult for me to be as involved as I’d like to be in this project I’d still really love to get this project off the ground. I hope I can use the time freed up by no longer being a member of RAS Council to work on the OJFA. I no longer have a conflict of interest in that regard either; like many other learned societies the RAS currently makes a large fraction of its income from academic publishing!

As Arfon mentions in his piece, the recent BICEP2 episode in particular provides pretty strong motivation that we need a new concept of academic publishing. Practical difficulties may have intervened for now but the motivation for the project itself is stronger now than it has ever been.

Elsevier journals — some facts

Posted in Open Access with tags , , on April 24, 2014 by telescoper

telescoper:

Read this, and weep as you learn that Elsevier’s ruthless profiteering continues unabated…

Originally posted on Gowers's Weblog:

A little over two years ago, the Cost of Knowledge boycott of Elsevier journals began. Initially, it seemed to be highly successful, with the number of signatories rapidly reaching 10,000 and including some very high-profile researchers, and Elsevier making a number of concessions, such as dropping support for the Research Works Act and making papers over four years old from several mathematics journals freely available online. It has also contributed to an increased awareness of the issues related to high journal prices and the locking up of articles behind paywalls.

However, it is possible to take a more pessimistic view. There were rumblings from the editorial boards of some Elsevier journals, but in the end, while a few individual members of those boards resigned, no board took the more radical step of resigning en masse and setting up with a different publisher under a new name (as some journals have…

View original 10,674 more words

Scholarly publishers and their high profits

Posted in Open Access on March 25, 2014 by telescoper

telescoper:

Lest we forget the Great Academic Publishing Ripoff…

 

 

Originally posted on Alex Holcombe's blog:

I recently published the below chart to document the outrageous profit margins of scholarly publishers in the sciences.

Screen Shot 2013-01-09 at 12.35.26 PM

This post is to provide the sources for the numbers in the chart.

The Woolworths number comes from their website, where they write “As a group, Woolworths Limited makes less than seven cents in the dollar before we then pay interest and tax”.

The Rio Tinto figure of 23% is based on the operating profit they report divided by the consolidated sales revenue in their 2011 financial summary.

Apple’s profit of 35% is based on these numbers, dividing their operating income for the year ending September 2012 of 55.2 billion by the revenue for the same period of 156.5 billion.

The 34% number for Springer comes from Heather Morrison’s PhD thesis, in which she writes that “Springer’s Science + Business Media (2010) reported a return on sales (operating profit)…

View original 66 more words

Follow

Get every new post delivered to your Inbox.

Join 4,141 other followers