The Negative Mass Bug

You may have noticed that some time ago I posted about  a paper by Jamie Farnes published in Astronomy & Astrophysics but available on the arXiv here which entails a suggestion that material with negative mass might account for dark energy and/or dark matter.

Here is the abstract of said paper:

Dark energy and dark matter constitute 95% of the observable Universe. Yet the physical nature of these two phenomena remains a mystery. Einstein suggested a long-forgotten solution: gravitationally repulsive negative masses, which drive cosmic expansion and cannot coalesce into light-emitting structures. However, contemporary cosmological results are derived upon the reasonable assumption that the Universe only contains positive masses. By reconsidering this assumption, I have constructed a toy model which suggests that both dark phenomena can be unified into a single negative mass fluid. The model is a modified ΛCDM cosmology, and indicates that continuously-created negative masses can resemble the cosmological constant and can flatten the rotation curves of galaxies. The model leads to a cyclic universe with a time-variable Hubble parameter, potentially providing compatibility with the current tension that is emerging in cosmological measurements. In the first three-dimensional N-body simulations of negative mass matter in the scientific literature, this exotic material naturally forms haloes around galaxies that extend to several galactic radii. These haloes are not cuspy. The proposed cosmological model is therefore able to predict the observed distribution of dark matter in galaxies from first principles. The model makes several testable predictions and seems to have the potential to be consistent with observational evidence from distant supernovae, the cosmic microwave background, and galaxy clusters. These findings may imply that negative masses are a real and physical aspect of our Universe, or alternatively may imply the existence of a superseding theory that in some limit can be modelled by effective negative masses. Both cases lead to the surprising conclusion that the compelling puzzle of the dark Universe may have been due to a simple sign error.

Well there’s a new paper just out on the arXiv by Hector Socas-Navarro with the abstract

A recent work by Farnes (2018) proposed an alternative cosmological model in which both dark matter and dark energy are replaced with a single fluid of negative mass. This paper presents a critical review of that model. A number of problems and discrepancies with observations are identified. For instance, the predicted shape and density of galactic dark matter halos are incorrect. Also, halos would need to be less massive than the baryonic component or they would become gravitationally unstable. Perhaps the most challenging problem in this theory is the presence of a large-scale version of the `runaway’ effect, which would result in all galaxies moving in random directions at nearly the speed of light. Other more general issues regarding negative mass in general relativity are discussed, such as the possibility of time-travel paradoxes.

Among other things there is this:

After initially struggling to reproduce the F18 results, a careful inspection of his source code revealed a subtle bug in the computation of the gravitational acceleration. Unfortunately, the simulations in F18 are seriously compromised by this coding error whose effect is that the gravitational force decreases with the inverse of the distance, instead of the distance squared.

Oh dear.

I don’t think I need go any further into this particular case, which would just rub salt into the wounds of Farnes (2018) but I will make a general comment. Peer review is the best form of quality stamp that we have but, as this case demonstrates, it is by no means flawless. The paper by Farnes (2018) was refereed and published, but is now shown to be wrong*. Just as authors can make mistakes so can referees. I know I’ve screwed up as a referee in the past so I’m not claiming to be better than anyone in saying this.

*This claim is contested: see the comment below.

I don’t think the lesson is that we should just scrap peer review, but I do think we need to be more imaginative about how it is used than just relying on one or two individuals to do it. This case shows that science eventually works, as the error was found and corrected, but that was only possible because the code used by Farnes (2018) was made available for scrutiny. This is not always what happens. I take this as a vindication of open science, and an example of why scientists should share their code and data to enable others to check the results. I’d like to see a system in which papers are not regarded as `final’ documents but things which can be continuously modified in response to independent scrutiny, but that would require a major upheaval in academic practice and is unlikely to happen any time soon.

In this case, in the time since publication there has been a large amount of hype about the Farnes (2018) paper, and it’s unlikely that any of the media who carried stories about the results therein will ever publish retractions. This episode does therefore illustrate the potentially damaging effect on public trust that the excessive thirst for publicity can have. So how do we balance open science against the likelihood that wrong results will be taken up by the media before the errors are found? I wish I knew!

24 Responses to “The Negative Mass Bug”

  1. “I’d like to see a system in which papers are not regarded as `final’ documents but things which can be continuously modified in response to independent scrutiny, but that would require a major upheaval in academic practice and is unlikely to happen any time soon.”

    If so, then it needs to be done in a manner which makes it easy to follow changes. I don’t want to re-read a paper just to find where something was changed, even less so if the change is not relevant. Such changes need to be clearly documented, stating what, exactly, was changed, not just “updated with revised version”. Like at arXiv, previous versions need to be available and/or there should be errata which clearly indicate which text should be deleted, changed, or added.

  2. “Well there’s a new paper just out on the arXiv by Hector Socas-Navarro with the abstract”

    There is something wrong with the link!

  3. “So how do we balance open science against the likelihood that wrong results will be taken up by the media before the errors are found?”

    We need to avoid people putting out stuff too early, relying on others to do their work, then taking the credit.

    We need both. We need traditional peer review. At least in cases where the results a) are unexpected, b) depend on numerical calculations, and c) such code is time-consuming to write, the code needs to be not just publicly available but tested.

    About a third of my papers point out errors in other papers. This is an essential part of science, but not very well honoured. Even less honoured is confirming a published result. Especially in cases involving millions of hours of CPU time and person-years of work, this is really important, but hardly ever done. Part of the problem is that some journals don’t like publishing papers which merely confirm other results.

  4. I’m amazed how the author didn’t find the bug well before even submitting the results to a journal. Surely he must have run tests to see if his code can reproduce, for example, the well-known solution of the two-body problem.

  5. HI Peter, I have posted a thread about the problems with the recent paper here: https://twitter.com/Astro_Jamie/status/1100050152734248962 My model may be wrong, time will tell, and I know that science will soon work that out, however this paper is definitely not the reason why it is wrong.

    This is a good case for the problems with modern science as you discuss here. The paper which suggests my paper is wrong is itself currently unpeer-reviewed, and the initial conditions used for their paper (from which many of the problems in their paper arise) have not been made open-source. I completely agree that peer-review and open-source codes are critical components of a successful scientific system.

    Thanks so much for discussing this, and really enjoy your blog.
    All the best,
    Jamie

    • I’m neither a github nor Python man (except for the original series, of course), but I tried to find the code where the alleged error resides, but found only relatively short codes. Is there a direct link to the code where the alleged error is?

    • “This is a good case for the problems with modern science as you discuss here. The paper which suggests my paper is wrong is itself currently unpeer-reviewed, and the initial conditions used for their paper (from which many of the problems in their paper arise) have not been made open-source.”

      I agree with Jamie here: an unrefereed, non-open-source-code paper b which claims that paper a (refereed with open-source code) is wrong is getting praised too soon.

      I don’t have an opinion on who’s right. The point is that the critic shouldn’t be held to lower standards than the criticized.

  6. “Unfortunately, the simulations in F18 are seriously compromised by this coding error whose effect is that the gravitational force decreases with the inverse of the distance, instead of the distance squared.”

    True or false?

    • Unless I’m reading the code incorrectly, function update_velocities in file sim.py is indeed calculating the acceleration incorrectly. But maybe this is compensated somewhere, and I’m just not seeing it?

      • telescoper Says:

        Might it look wrong because it is in comoving coordinates?

      • I’m not a cosmologist and I’m not that familiar with comoving coordinates, but I don’t see how they could make that kind of difference.

    • I read the 20 Tweets. G=1 or not depends on the system of units, but the claim is a different one: inverse rather than inverse square. They cannot both be correct. Which is in your code? Does changing it as suggested by Hector have an effect? If so, what is the effect? If not, why not?

      These are simple questions which I think can be answered without having to plough through the code.

  7. Indeed, these are ongoing and rather intractable issues, where many false hopes and misdirections have occurred, as I touched upon in my post published at https://soundeagle.wordpress.com/2012/10/28/two-thirds-of-scientific-publications-retracted-are-fraudulent/

  8. […] 26 Feb 2019] Recomiendo leer a Peter Coles @telescoper, “The Negative Mass Bug,” In the dark, 25 Feb 2019: “This episode does therefore illustrate the potentially damaging effect on public trust that […]

  9. If you find or suspect an error in a paper, it seems better to contact the original author and, if confirmed, let that author publish an erratum. Using it to denounce a paper on arxiv seems more about point scoring than about science.

    • telescoper Says:

      There has been correspondence between these two authors, but it seems they disagree. I don’t see anything wrong with going public if that happens.

    • I think it depends on the type of error. If it is a programming error or something similar, i.e. an honest mistake, then, yes, get the author to publish an erratum (mentioning the finder in the acknowledgements). If the author refuses, though, I think that a separate paper pointing out the error is necessary.

      On the other hand, an error in the sense of some basic error of logic, leading to a wrong interpretation, or something like that, should result in a paper by the person who found the problem.

      I once read an erratum in A&A where the authors pointed out an error in their code, then in the acknowledgements thanked a colleague for painstakingly trying to reproduce their results.

      • I usually quote something for context, but didn’t here, since I assumed that mine would be the first comment under Albert’s. However, it seems that Peter replied while I was writing, so his comment appeared first.

        For the record, I was replying to Albert, not to Peter.

    • If the two teams don’t agree, yes it can become a matter for the scientific literature. I don’t know the history here.

  10. telescoper Says:

    I’m not sure why, but this post has attracted an awful lot of traffic!

  11. Giovanni Manfredi Says:

    Farnes’ paper received a lot of publicity, mainly thanks to the press release of Oxford university. May I point out that a negative mass universe (“Dirac-Milne”) was already proposed by Chardin et al in 2012: http://dx.doi.org/10.1051/0004-6361/201016103
    Numerical simulations of structure formation in this universe, along with a detailed analysis of negative-mass cosmologies, were published in Phys. Rev. D (shortly before Farnes’ paper), and the paper is available here: https://arxiv.org/abs/1804.03067.
    The type of negative masses used in our work are significantly different from those discussed by Farnes.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: