A Pointless Imprimatur?

In numerous rants about Open Access on this blog I’ve made the point that because of the arXiv the field I work in is way ahead of the game. Most researchers in astronomy astrophysics and cosmology post their papers on the arXiv, and many do that before the work has been accepted for publication. Even before the arXiv we used to circulate preprints ahead of publication.

But it seems there are some astronomers who aren’t aware of the culture of openness. Here is an excerpt from a referee report on a paper submitted to Astronomy & Astrophysics which has been circulated on Twitter:

In summary the referee thinks the paper should be rejected because it has already appeared on the arXiv. That’s a pretty extraordinary recommendation when the authors were following standard practice for the field!

In a sense, though, the referee is right. Journals are no longer needed in order to publish papers. We can all do that ourselves on the arXiv for free. What we do need is to have some quality control via peer review. The imprimatur of a journal is not pointless because it indicates the paper has met a quality threshold. Indeed, in my opinion, it’s the only thing a journal does that’s not pointless..

The Open Journal Of Astrophysics provides peer review for papers on the arXiv, dispensing with the rest of the cumbersome paraphernalia of journals that digital publishing has made redundant.

So if your paper is rejected by a journal because you have put in on the arXiv then why not just submit it to OJA instead?

35 Responses to “A Pointless Imprimatur?”

  1. I prefer to keep my papers off ArXiv until they have been refereed. May be old fashioned, but it is important to keep some quality control of what is in the ‘literature’. Especially the data: interpretations will change, but errors in the data are very hard to eradicate once published. Referees also often ask for some extra work which makes the conclusions stronger. I would like to see a clear distinction between what has been reviewed, and what has not.

    • I agree. Also, note that some institutions have a policy that nothing should go to arXiv before acceptance. Some people are afraid of plagiarism, especially if they are not able to rush through the publication process. Some people cannot (yet) submit to arXiv. I believe that the OJA has an alternate publication mechanism, but it should be more visible, especially before signing up. And OJA needs to publicly commit to do everything possible to enable publication on arXiv if a paper is accepted but the author cannot (yet) submit to arXiv.

      Also, I think that some people might be hesitant in supporting the OJA because if it succeeds then much of the literature will be lost forever, because traditional journals will fold and there is neither the requirement nor the desire to to make stuff on which they hold the copyright public.

      I think that the OJA needs to address both of these topics convincingly, or else it will probably remain a niche market.

      • … if it succeeds then much of the literature will be lost forever, because traditional journals will fold and there is neither the requirement nor the desire to to make stuff on which they hold the copyright public.

        Nice papers you people have here. What a pity if something were to happen to them.

    • One problem with papers which are put on arxiv after they are accepted/published is that I have seen many times authors not providing proper attribution to previous publications. This cannot be fixed if the paper is already accepted. Certainly I don’t expect the authors or the referee/editor to be uptodate on all literature. But certainly if its posted on arxiv it provides an opportunity to be fixed. Also sometimes one can get a benign referee and even a not so good paper can get accepted without changes. But if it is put on arxiv, the whole community can review it and help make it better.

      • If the paper reproduces a result which is already in the literature, then one can debate whether it should be rejected (because the author wasn’t familiar with the literature) or be published as a particularly good example of confirming a previous result (much more valuable than knowing the answer in advance). In either case, a “note added in proof” is clearly enough.

      • Hi all,
        Just to give an example (today), have a look at this paper (which appeared on arxiv https://arxiv.org/abs/1909.05303) published in APSS . It makes no reference to https://arxiv.org/abs/1704.04659, which also used the same dataset and a common algorithm XGBoost. Now because it is already published, there is no chance for the authors to rectify the situation and all papers which cite will forget about the previous papers (Of course the refereeing standards have also dropped precipitously). That’s why I hope people submit their papers to arxiv before acceptance.

      • It’s the responsibility of the referees and the authors to make sure that prior work is acknowledged; we need something better than “hey, saw your paper on arXiv; please cite my paper on the same topic”.

        One could argue that a paper should not be cited if the authors (for whatever reason) made no use of it in writing their newer paper.

      • I’m not saying that it shouldn’t happen. People have written me and asked me to add a relevant reference; there is always time for this at the proof stage. I’ve written to people and asked them to cite a relevant paper, often someone else’s but usually one of mine; this was usually done. Yes, it is easier if the paper is not yet accepted, but still possible otherwise, and shouldn’t trump other reasons for not wanting to submit to arXiv first.

  2. To me the words of the peer-reviewer you cite come across as simple snobbery and gate-keeping on that basis. Material on ArXiv is not, of course, formally peer-reviewed, nor does reading it constitute such. However, it creates opportunity for comment which may (or may not) be useful, if received with suitable critical analysis. Wouldn’t it be better to view the situation as a case of ‘both’ rather than ‘one excluding the other’? I have to say that on my own experience (having spent some time, myself, as editor of a peer-reviewed journal in another field) that peer-reviewing is as often used as a device for exclusion and/or for pushing personal barrows, than for actually improving the intellectual rigour and quality of what is published. Despite the pretense of impartial scholarship, in practise it all comes down to individual attitudes; and humans, alas, seem to have an unerring ability to weaponise just about everything.

  3. John Peacock Says:

    My view on this is that peer review is something whose time of usefulness has passed. In general conscientious referees do their best, and sometimes papers are significantly improved as a result, but this is rare. Equally often, you get a referee who is simply biased and obstructive, leading to massive delays in a good paper (I suffered a particularly bad example of this myself recently; fortunately the paper went straight to arXiv and so had dozens of citations by the time it was finally “published”). But most of the time I think a submitted manuscript appears with marginal changes that are in the noise.

    Thus peer review is not providing a useful filter on the sheer amount of stuff that we have to wade through. The real problem is that the majority of the literature is “not even wrong” – just a pointless distraction that makes it harder to find the few genuinely new and interesting results. If we had a journal with rigorous critical peer review that published only the best 10% of what it was sent, that would be valuable. But this doesn’t exist, and no-one is proposing it.

    If peer review was abandoned, I don’t think anything much would change. Writing papers is hard work, and the community is already doing it flat out – so the total quantity of stuff going on arXiv would probably not alter. The quality might even improve, as people would realise the first posting is final – so you can’t be sloppy with details hoping to tidy them up after refereeing. Yes there would be some terribly written and incorrect papers out there, but this is already massively the case: the community learns quickly to ignore them or to write new papers that identify errors and propose correct approaches. This self-policing function will operate effectively whether or not there is formal peer review.

    • Some of the motivation to ArXiV Before Review is to claim ownership of a result. Speed can be more important than accuracy. In some cases self-refereeing may well work. But in other cases it could lead to wrong or incomplete research to be ‘published’. In such a system, results from proven researchers may be seen as more authorative. Whether that is a good thing I’ll leave for others to decide.

    • Clearly, traditional journals such as MNRAS chucking out peer review is a non-starter; the journals would soon fill up with crackpot stuff.

      Note that arXiv explicitly states that they have moderators, not referees. They do keep out obviously crackpot stuff, but that doesn’t mean that everything which gets through is good.

      I’m willing to believe that your papers might not improve much due to refereeing, but I doubt that that is true in general.

      Note also that not everyone is allowed to post to arXiv. For a struggling young researcher from a third-world country or whatever, acceptance by a major traditional journal is often the only way to obtain permission to post to arXiv (and, as far as I know, even this is not automatic).

      The stuff on arXiv is of relatively high quality primarily because most of it is intended for traditional journals.

    • “If we had a journal with rigorous critical peer review that published only the best 10% of what it was sent, that would be valuable. But this doesn’t exist, and no-one is proposing it.”

      But this would negatively affect bread-and-butter research which can be essential for the Really Important Papers.

      Some journals do mark some articles as “Editors’ Choice” or whatever to denote articles which they think are at the top of the barrel.

  4. I do think there is a real problem with being asked to peer-review a paper that has already appeared on arXiv. I put a lot of effort into reviewing papers in the hope that my suggestions will improve them, but if the paper is on arXiv first, it’s largely wasted effort as most people will read the unreviewed version. Fortunately, almost none of the papers I receive to review have been posted, so to contradict the OP, it is not true that most Astronomy & Astrophysics papers are submitted to arXiv on submission, although it is common in cosmology because at the turn of the century there was a frantic race to get results out first. I think I have only had one paper that I noticed was out on arXiv first, and I simply passed on reviewing it.
    (NB I see that the OJA requires authors to post first, so don’t bother asking me to be one of your referees!).

    I agree with both John Peacock and telescoper that the whole system is moribund and maintained only because beancounters have fixed on counting peer-reviewed papers and citations as a measure of research output (which by somebody’s law it ceases to be, as soon as it’s used to inform funding decisions)… all at vast and unproductive expense to the taxpayer.

    • “(NB I see that the OJA requires authors to post first, so don’t bother asking me to be one of your referees!).”

      I think that there is a stealth mechanism to submit without arXiv (though, of course, accepted papers must go to arXiv). I might try it out at some point if I am allowed to.

      “I agree with both John Peacock and telescoper that the whole system is moribund and maintained only because beancounters have fixed on counting peer-reviewed papers and citations as a measure of research output (which by somebody’s law it ceases to be, as soon as it’s used to inform funding decisions)… all at vast and unproductive expense to the taxpayer.”

      The question is what to replace it with.

    • The somebody is probably Goodhart.

    • “I put a lot of effort into reviewing papers in the hope that my suggestions will improve them, but if the paper is on arXiv first, it’s largely wasted effort as most people will read the unreviewed version.”

      A long time ago, I started suggesting that there should be a required “status” field at arXiv and that the submission automatically expires if this is not updated every 3 months or whatever (or if the status stays the same for a year or whatever) until the stage “published” is reached (and the Journal-Ref. field correspondingly updated). This might combat Paddy’s fears to some extent.

      As I’ve said before, not everything is on arXiv; even if something is, the definitive version is not always there; even if it is, there might not be any way to determine this (save by comparing it with the published version, in which case one doesn’t need the arXiv version).

      • Phillip: Some top-cited papers in astrophysics are not published.
        Two such widely cited ones https://arxiv.org/abs/astro-ph/0207156 (which has over 1200 citations in ADS). Another is https://arxiv.org/abs/astro-ph/0108028 (close to 150 citations). Both of these are correct and widely used. Also my string theory colleague tells me that people like Ed Witten and Juan Maldacena don’t bother to submit their papers to journals. they only post them on arxiv and in fact journal editors contact them to submit their papers for publication.

      • “Some top-cited papers in astrophysics are not published.
        Two such widely cited ones https://arxiv.org/abs/astro-ph/0207156 (which has over 1200 citations in ADS). Another is https://arxiv.org/abs/astro-ph/0108028 (close to 150 citations).”

        These are exceptions.

        “Also my string theory colleague tells me that people like Ed Witten and Juan Maldacena don’t bother to submit their papers to journals. they only post them on arxiv and in fact journal editors contact them to submit their papers for publication.”

        Again, exceptions. Similarly, really famous people don’t apply for jobs; they get asked. 😐

    • Paddy: Not all referees are as conscientious as you are. Many times the acceptance/declining of a paper depends on reputation of author and/or which institute/country they are based in. The classic example ( I know of) is that when Moffat submitted his paper to PRD on proposing faster than speed of light as alternative to inflation, it was rejected. But the same journal accepted the paper by Albrecht and Magueijo . Also sometimes a paper can be benefitted by vetting from than one person, even if the referee is very careful. So certainly similar to HEP, authors should submit their papers to arxiv

      • telescoper Says:

        The biggest problem we have at the OJA is getting referees to agree, and then to produce a report by the agreed deadline. It’s much better if they simply decline the invitation than what often happens, namely that they accept but just don’t write a report.

        This is true of other journals too, but it’s worse for OJA I think because it is relatively new.

      • “Paddy: Not all referees are as conscientious as you are.”

        Is there any referee more conscientious than Paddy? 🙂

  5. Philip. These two papers are of the top of my head. I am sure there are many more. With time hopefully we shall see a greater trend of papers been put up on arxiv.

  6. Phillip Helbig Says:

    Speaking of the Open Journal of Astrophysics, can we have a minimal-example paper using openjournal.cls as well as the class file itself? Since it is heavily based on emulateapj, some things which might be obvious to ApJ authors might not be to others.

    • I see that you’re now the technical contact for questions about the style file. 🙂

      %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
      %% Copyright 2015 The Open Journal of Astrophysics
      %%
      %% Originally based on emulateapj.cls by Alexey Vikhlinin
      %% The first version of this package was written by Maxim Markevitch.
      %% Pieces of AASTeX code are used for compatibility with aastex.cls.
      %%
      %% This program can be redistributed and/or modified under the terms of
      %% the LaTeX Project Public License available from CTAN archives in
      %% macros/latex/base/lppl.txt. This means you are free to use and
      %% distribute this package; however, if you modify anything, please
      %% change the file name and remove the author’s email address.
      %%
      %% The Open Journal of Astrophysics
      %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
      %%
      %% `openjournal’ is a LaTeX2e class provided by the Open Journal of Astrophysics as a suggested template for submissions.
      %%
      %% openjournal requires revtex4.cls. If you don’t have it already, it can be
      %% downloaded from http://publish.aps.org/revtex4/ (it’s a small package).
      %% Other external packages used are latexsym, graphicx, amssymb, longtable,
      %% epsf. They should already be present in the modern TeX distributions; if
      %% not, download them from http://www.ctan.org.

    • Yes!! Can you let me know if you find somethign? Right now it seems to be incompatible with just about every article I have written!

    • telescoper Says:

      You can of course access the source files of all the papers we have published from the arXiv…

Leave a Reply to Paddy Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: