Archive for the Astrohype Category

Cosmological Dipole Controversy

Posted in Astrohype, Bad Statistics, The Universe and Stuff with tags , , on October 11, 2022 by telescoper

I’ve just finished reading an interesting paper by Secrest et al. which has attracted some attention recently. It’s published in the Astrophysical Journal Letters but is also available on the arXiv here. I blogged about earlier work by some of these authors here.

The abstract of the current paper is:

We present the first joint analysis of catalogs of radio galaxies and quasars to determine if their sky distribution is consistent with the standard ΛCDM model of cosmology. This model is based on the cosmological principle, which asserts that the universe is statistically isotropic and homogeneous on large scales, so the observed dipole anisotropy in the cosmic microwave background (CMB) must be attributed to our local peculiar motion. We test the null hypothesis that there is a dipole anisotropy in the sky distribution of radio galaxies and quasars consistent with the motion inferred from the CMB, as is expected for cosmologically distant sources. Our two samples, constructed respectively from the NRAO VLA Sky Survey and the Wide-field Infrared Survey Explorer, are systematically independent and have no shared objects. Using a completely general statistic that accounts for correlation between the found dipole amplitude and its directional offset from the CMB dipole, the null hypothesis is independently rejected by the radio galaxy and quasar samples with p-value of 8.9×10−3 and 1.2×10−5, respectively, corresponding to 2.6σ and 4.4σ significance. The joint significance, using sample size-weighted Z-scores, is 5.1σ. We show that the radio galaxy and quasar dipoles are consistent with each other and find no evidence for any frequency dependence of the amplitude. The consistency of the two dipoles improves if we boost to the CMB frame assuming its dipole to be fully kinematic, suggesting that cosmologically distant radio galaxies and quasars may have an intrinsic anisotropy in this frame.

I can summarize the paper in the form of this well-worn meme:

My main reaction to the paper – apart from finding it interesting – is that if I were doing this I wouldn’t take the frequentist approach used by the authors as this doesn’t address the real question of whether the data prefer some alternative model over the standard cosmological model.

As was the case with a Nature piece I blogged about some time ago, this article focuses on the p-value, a frequentist concept that corresponds to the probability of obtaining a value at least as large as that obtained for a test statistic under a particular null hypothesis. To give an example, the null hypothesis might be that two variates are uncorrelated; the test statistic might be the sample correlation coefficient r obtained from a set of bivariate data. If the data were uncorrelated then r would have a known probability distribution, and if the value measured from the sample were such that its numerical value would be exceeded with a probability of 0.05 then the p-value (or significance level) is 0.05. This is usually called a ‘2σ’ result because for Gaussian statistics a variable has a probability of 95% of lying within 2σ of the mean value.

Anyway, whatever the null hypothesis happens to be, you can see that the way a frequentist would proceed would be to calculate what the distribution of measurements would be if it were true. If the actual measurement is deemed to be unlikely (say that it is so high that only 1% of measurements would turn out that large under the null hypothesis) then you reject the null, in this case with a “level of significance” of 1%. If you don’t reject it then you tacitly accept it unless and until another experiment does persuade you to shift your allegiance.

But the p-value merely specifies the probability that you would reject the null-hypothesis if it were correct. This is what you would call making a Type I error. It says nothing at all about the probability that the null hypothesis is actually a correct description of the data. To make that sort of statement you would need to specify an alternative distribution, calculate the distribution based on it, and hence determine the statistical power of the test, i.e. the probability that you would actually reject the null hypothesis when it is incorrect. To fail to reject the null hypothesis when it’s actually incorrect is to make a Type II error.

If all this stuff about p-values, significance, power and Type I and Type II errors seems a bit bizarre, I think that’s because it is. In fact I feel so strongly about this that if I had my way I’d ban p-values altogether…

This is not an objection to the value of the p-value chosen, and whether this is 0.005 rather than 0.05 or, , a 5σ standard (which translates to about 0.000001!  While it is true that this would throw out a lot of flaky ‘two-sigma’ results, it doesn’t alter the basic problem which is that the frequentist approach to hypothesis testing is intrinsically confusing compared to the logically clearer Bayesian approach. In particular, most of the time the p-value is an answer to a question which is quite different from that which a scientist would actually want to ask, which is what the data have to say about the probability of a specific hypothesis being true or sometimes whether the data imply one hypothesis more strongly than another. I’ve banged on about Bayesian methods quite enough on this blog so I won’t repeat the arguments here, except that such approaches focus on the probability of a hypothesis being right given the data, rather than on properties that the data might have given the hypothesis.

Not that it’s always easy to implement the (better) Bayesian approach. It’s especially difficult when the data are affected by complicated noise statistics and selection effects, and/or when it is difficult to formulate a hypothesis test rigorously because one does not have a clear alternative hypothesis in mind. That’s probably why many scientists prefer to accept the limitations of the frequentist approach than tackle the admittedly very challenging problems of going Bayesian.

But having indulged in that methodological rant, I certainly have an open mind about departures from isotropy on large scales. The correct scientific approach is now to reanalyze the data used in this paper to see if the result presented stands up, which it very well might.

Recalibration of Ultra-High-Redshift Galaxies

Posted in Astrohype, The Universe and Stuff with tags , , , , on August 10, 2022 by telescoper

Remember all the recent excitement about the extremely high redshift galaxies (such as this and this; the two examples shown above) “identified” in early-release JWST observations? Well, a new paper on the arXiv by Adams et al using post-launch calibration of the JWST photometry suggests that we should be cautious about the interpretation of these objects. The key message of this study is that the preliminary calibration that has been in widespread use for these studies is wrong by up to 30% and that can have a huge impact on inferred redshifts.

The new study does indeed identify some good candidates for ultra-high-redshift galaxies, but it also casts doubt on many of the previous claims. Here is a table of some previous estimates alongside those using the newly recalibrated data:

You will see that in most – but not all – cases the recalibration results in a substantial lowering of the estimated redshift; one example decreases from z>20 to 0.7! The two candidates mentioned at the start of this post are not included in this table but one should probably reserve judgement on them too.

The conclusive measurements for these objects will however include spectroscopy, and the identification of spectral lines, rather than photometry and model fits to the spectra energy distribution. Only with such data will we really know how many of these sources are actually at very high redshift. As the philosopher Hegel famously remarked

The Owl of Minerva only spreads its wings with the coming of spectroscopy.

The Dark Side of ChorizoGate

Posted in Astrohype with tags , , on August 6, 2022 by telescoper

So ChorizoGate continues…

The story has now made it into numerous newspapers, from the Daily Star to The Times, and even got coverage on CNN News and the RTÉ website. More exciting still, it’s in Physics World!

In every single one of the stories I’ve seen, the image, together with the JWST connection, is attributed to Étienne Klein who is apparently very well known in France as a popularizer of science in the French language. Because he writes and broadcasts in French he is not so well known outside France. Until recently that is!

To be honest I’m quite relieved to have avoided the media notoriety surrounding ChorizoGate, especially as it means I’ve avoided being on the front page of the Daily Star! Dr Klein is welcome to the publicity, though perhaps it might backfire on him…

Since Thursday, when I posted my piece on the background to ChorizoGate, I’ve been contacted by a significant number of people (some in private and some in public, on social media) with stories about the “French scientist” who spread the above image on Twitter. Some of these are rather worrying.

In particular, in 2016, Étienne Klein was involved in a scandal in which he was shown to have plagiarized some of the material in one of his popular books. This is mentioned only briefly on his wikipedia page, but there are articles about his plagiarism here and here (in French). This story accords with many of the public and private comments I’ve seen about his habitual plagiarism. In other words, he has form. A lot of it, even if only a fraction of what I’ve been told is true.

Étienne Klein’s appropriation of a silly joke is of no consequence, but I couldn’t help wondering how someone who would do that might behave with things that actually matter. Now I think I know the answer, and it’s worrying.

Update:

The Earliest Galaxy we’ve seen?

Posted in Astrohype, The Universe and Stuff with tags , , , on July 20, 2022 by telescoper

The red smudge in the centre of this image is thought to be a galaxy with a redshift of around z=13, as seen by the NIRCam instrument on the James Webb Space Telescope. This redshift estimate is based on photometry so the object remains a candidate rather than a confirmed high-redshift galaxy, but if confirmed spectroscopically this would be the highest-redshift galaxy yet observed.

For more details on the observations and their implications see the preprint on arXiv here. It’s interesting (and challenging) that there are such bright galaxies at such an early stage of cosmic evolution, assuming of course that the redshift is correct. Photometric redshift estimates have been wrong before.

If we take the estimated redshift at face value and adopt the standard cosmological model, the lookback time to this galaxy (GLASS-z13) is about 97.6% of the current age of the Universe so we’re seeing it as it was just 330 million years after the Big Bang. It could therefore be the earliest galaxy we have seen. It isn’t very accurate to say that it is the oldest galaxy we’ve seen, as we are probably seeing it as it was when it was very young.

These observations come from JWST Early Science Release Programmes so are just a taster of what is to come. No doubt we’ll hear much more about high-redshift galaxies from JWST in future and there’s every chance that they will change our view of the high-redshift Universe in dramatic ways.

I’ll just mention here that I’m old enough to remember going to conferences where “high redshift” meant z=0.5! In those days the highest redshift objects were quasars, but they have long since been overtaken.

Science vs Marketing

Posted in Astrohype, Education, Science Politics, The Universe and Stuff with tags , , , on May 20, 2022 by telescoper

I saw a paper some months ago by former Sussex colleague Xavier Calmet and collaborators that attracted quite a lot of press coverage largely based on a press release from the University of Sussex that claimed:

Stephen Hawking’s famous black hole paradox solved after hair-raising discovery

(If you want to learn more about the black hole information paradox you could start here.)

The press release is pure unadulterated hype. The paper in Physical Review Letters is actually rather good in my opinion but it says next to nothing about the black hole information paradox. Unfortunately the Sussex press release was picked up by the BBC’s science editor Pallab Ghosh who turned it into a very garbled article. Unfortunately Ghosh has quite a lot of form when it comes to producing nonsensical takes on science results. See, for example, this piece claiming that recent results from the Dark Energy Survey cast doubt on Einstein’s general theory of relativity when they do nothing of the sort.

Fortunately in the case of the black hole paper David Whitehouse has done a good job at demolishing the “BBC’s black hole baloney” here so I don’t need to repeat the arguments.

What I will mention however is that there is an increasing tendency for university press offices to see themselves entirely as marketing agencies instead of informing and/or educating the public. Press releases about scientific research nowadays rarely make any attempt at accuracy – they are just designed to get the institution concerned into the headlines. In other words, research is just a marketing tool.

This isn’t the only aspect of the marketisation of universities. If an academic tries to organize a public engagement event or do some schools outreach activity, the chances are their institution will hijack it and turn it into a marketing exercise, aimed exclusively at student recruitment. Universities are increasingly unconcerned with education and research and obsessed with income.

Forget the phony controversies about woke politics and free speech manufactured by right-wing press. The real culture war in modern universities is between those who believe in the intrinsic value of higher education and those who see it simply as a means of generating profit by whatever means possible. As in any war, truth is the first casualty.

Still no Primordial Gravitational Waves…

Posted in Astrohype, Bad Statistics, The Universe and Stuff with tags , , , , , , on October 27, 2021 by telescoper

During March 2014 this blog received the most traffic it has ever had (reaching almost 10,000 hits per day at one point). The reason for that was the announcement of the “discovery” of primordial gravitational waves by the BICEP2 experiment. Despite all the hype at the time I wasn’t convinced. This is what I said in an interview with Physics World:

It seems to me though that there’s a significant possibility of some of the polarization signal in E and B [modes] not being cosmological. This is a very interesting result, but I’d prefer to reserve judgement until it is confirmed by other experiments. If it is genuine, then the spectrum is a bit strange and may indicate something added to the normal inflationary recipe.

I also blogged about this several times, e.g. here. It turns out I was right to be unconvinced as the signal detected by BICEP2 was dominated by polarized foreground emission. The story is summarized by these two news stories just a few months apart:

Anyway, the search for primordial gravitational waves continues. The latest publication on this topic came out earlier this month in Physical Review Letters and you can also find it on the arXiv here. The last sentence of the abstract is:

These are the strongest constraints to date on primordial gravitational waves.

In other words, seven years on from the claimed “discovery” there is still no evidence for anything but polarized dust emission…

More from the Dark Energy Survey

Posted in Astrohype, The Universe and Stuff with tags , , , , , , on May 28, 2021 by telescoper

To much media interest the Dark Energy Survey team yesterday released 11 new papers based on the analysis of their 3-year data. You can find the papers together with short descriptions here. There’s even a little video about the Dark Energy Survey here:

The official press release summarizes the results as follows:

Scientists measured that the way matter is distributed throughout the universe is consistent with predictions in the standard cosmological model, the best current model of the universe.

This contrasts a bit with the BBC’s version:

The results are a surprise because they show that it is slightly smoother and more spread out than the current best theories predict.

The observation appears to stray from Einstein’s theory of general relativity – posing a conundrum for researchers.

The reason for this appears to be that the BBC story focusses on the weak lensing paper (found here; I’ll add a link to the arXiv version if and when it appears there). The abstract is here:

The parameter S8 is a (slightly) rescaled version of the more familiar parameter σ8  – which quantifies the matter-density fluctuations on a scale of 8 h-1 Mpc – as defined in the abstract; cosmic shear is particularly sensitive to this parameter.

The key figure showing the alleged “tension” with Planck is here:

The companion paper referred to in the above abstract (found here has an abstract that concludes with the words (my emphasis).

We find a 2.3σ difference between our S8 result and that of Planck (2018), indicating no statistically significant tension, and additionally find our results to be in qualitative agreement with current weak lensing surveys (KiDS-1000 and HSC).

So, although certain people have decided to hype up a statistically insignificant l discrepancy, everything basically fits the standard model…

Do you believe there is intelligent life out there?

Posted in Astrohype, The Universe and Stuff with tags , , on February 15, 2021 by telescoper

Following on from yesterday’s post and inspired by this tweet

I thought I’d try a quick poll on what people think about the existent of intelligent life “out there”. In my experience, and in contrast to the tweet, most astronomers are actually quite open-minded about the possibility of extraterrestrial intelligence. My own view is that while it is entirely possible that there is intelligent life elsewhere in the Universe, scientific belief* should be based on evidence and we simply don’t have any evidence to support the idea. I would therefore say that I don’t “believe” but am agnostic. Sometimes “I don’t know” is the only rational answer.

You might ask me as a cosmologist whether I believe in the Big Bang theory. My answer would be “yes” because I think the evidence supports this description of the Universe. If evidence came along to change my opinion I might believe in it less or not at all. I might also change my mind if a different model came along that fits the observations better.

*In a Bayesian sense I would interpret “refusing to believe” as meaning setting a prior probability of zero, so that no amount of evidence would produce a non-zero posterior probability.

Anyway, setting aside the issue of whether there is even any evidence of intelligent life on Earth here is a poll:

Phosphine on Venus again…

Posted in Astrohype, The Universe and Stuff with tags , , , on February 8, 2021 by telescoper

Remember all the excitement last September about the claimed detection of Phosphine in the atmosphere of Venus? If you do you will remember that the claim has been contested. Now there is another paper on this matter claiming that what was interpreted as a spectral feature due to Phosphine (PH3) could actually be due to Sulphur Dioxide (SO2)). Here is the abstract of the latest paper.

You can find the full paper on the arXiv here.

P.S. I bookmarked the arXiv paper about a week ago but was very busy and forgot about it until now!

Phosphine on Venus, Water on the Moon, and Hype Everywhere

Posted in Astrohype, The Universe and Stuff with tags , , on October 28, 2020 by telescoper

To continue the ongoing saga of Phosphine in the atmosphere of Venus there’s a very strongly worded paper on the arXiv with the following abstract:

It’s one thing to question or refute another group’s result, but there’s no need to be so aggressive about it. The last sentence of the abstract is particularly unnecessary and reprehensible.

Update: the abstract has now been changed.

There has been a lot of reaction on social media from astronomers and others to the perceived “hype” of the initial discovery by the authors. I watched the press conference at the time and I think the authors spoke very sensibly about their work. Of course just because scientists are sensible that’s no reason to suppose the press will also be sensible and there was undoubtedly a great deal of hype about that result. In my experience hype is more likely to be a result of journalists wanting a sensational story and/or institutional press offices wanting to promote their institution that scientists over-egging their own puddings (though that does happen too).

I don’t mind individual scientists or groups of scientists making fools of themselves. It’s the damage to public trust in science that is the real danger here.

The hostile reaction we see in the above paper is an inevitable manifestation of an environment which encourages runaway self-publicity. This is not the only area in which this sort of toxic behaviour happens. I suppose it is mildly reassuring that it’s not only cosmologists that behave in such a way, but is this really what we want astrophysics to be like? I think we’d be better off leaving the petty point-scoring to the politicians.

Another example of hype this week – also involving a paper in Nature Astronomy – was the discovery of water on the Moon (again). The NASA publicity machine pulled out all the stops in advance of this announcement, only for the actual result to be a damp squib. Water is one of the most abundant molecules in space and I’ve lost track of how many times it has been detected on the Moon already. I suppose it is moderately reassuring that hasn’t suddenly disappeared, but from a scientific point of view it’s not all that interesting. I was particularly disappointed when the result turned out to be water, as I had bet on phosphine…

Could it be that the Stratospheric Observatory for Infrared Astronomy (SOFIA) is up for a funding review?