Archive for Science

Metrics for `Academic Reputation’

Posted in Bad Statistics, Science Politics with tags , , , on April 9, 2018 by telescoper

This weekend I came across a provocative paper on the arXiv with the title Measuring the academic reputation through citation records via PageRank. Here is the abstract:

The objective assessment of the prestige of an academic institution is a difficult and hotly debated task. In the last few years, different types of University Rankings have been proposed to quantify the excellence of different research institutions in the world. Albeit met with criticism in some cases, the relevance of university rankings is being increasingly acknowledged: indeed, rankings are having a major impact on the design of research policies, both at the institutional and governmental level. Yet, the debate on what rankings are  exactly measuring is enduring. Here, we address the issue by measuring a quantitative and reliable proxy of the academic reputation of a given institution and by evaluating its correlation with different university rankings. Specifically, we study citation patterns among universities in five different Web of Science Subject Categories and use the PageRank algorithm on the five resulting citation networks. The rationale behind our work is that scientific citations are driven by the reputation of the reference so that the PageRank algorithm is expected to yield a rank which reflects the reputation of an academic institution in a specific field. Our results allow to quantifying the prestige of a set of institutions in a certain research field based only on hard bibliometric data. Given the volume of the data analysed, our findings are statistically robust and less prone to bias, at odds with ad hoc surveys often employed by ranking bodies in order to attain similar results. Because our findings are found to correlate extremely well with the ARWU Subject rankings, the approach we propose in our paper may open the door to new, Academic Ranking methodologies that go beyond current methods by reconciling the qualitative evaluation of Academic Prestige with its quantitative measurements via publication impact.

(The link to the description of the PageRank algorithm was added by me; I also corrected a few spelling mistakes in the abstract). You can find the full paper here (PDF).

For what it’s worth, I think the paper contains some interesting ideas (e.g. treating citations as a `tree’ rather than a simple `list’) but the authors make some assumptions that I find deeply questionable (e.g. that being cited among a short reference listed is somehow of higher value than in a long list). The danger is that using such information in a metric could form an incentive to further bad behaviour (such as citation cartels).

I have blogged quite a few times about the uses and abuses of citations (see tag here) , and I won’t rehearse these arguments here. I will say, however, that I do agree with the idea of sharing citations among the authors of the paper rather than giving each and every author credit for the total. Many astronomers disagree with this point of view, but surely it is perverse to argue that the 100th author of a paper with 51 citations deserves more credit than the sole author of paper with 49?

Above all, though, the problem with constructing a metric for `Academic Reputation’ is that the concept is so difficult to define in the first place…

Advertisements

Talent versus Luck

Posted in Biographical with tags , , on March 5, 2018 by telescoper

I’ve remarked quite a number of times on the blog that I think I’ve been exceptionally lucky in my scientific career, the latest example being the good fortune that the position at Maynooth University came up precisely when it did, enabling me to relocate to Ireland.

It struck me further the other day that the people who think that science is genuinely meritocratic, tend to be those who have done well in the system rather than those who haven’t. It’s rather like the way that very rich people tend to think that they have earned their wealth and that makes them better people than those who are less well off, even when that’s demonstrably not true.

Likewise, luck plays a definite role in winning grant funding. Having been on grants panels I’m away that many very good proposals are not funded. A scoring system is generally used that introduces some level of objectivity into the process, but the fact is that a lot of proposals come out with similar scores and the ranking of these is a bit arbitrary. A slightly different panel would produce slightly different scores, but perhaps a large difference in ranking would result.

Anyway, there’s a paper on the arXiv (by Pluchino et al) with the title Talent vs Luck: the role of randomness in success and failure that
discusses the role of good fortune in scientific careers. This is the abstract:

The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking. Sometimes, we are willing to admit that a certain degree of luck could also play a role in achieving significant material success. But, as a matter of fact, it is rather common to underestimate the importance of external forces in individual successful stories. It is very well known that intelligence or talent exhibit a Gaussian distribution among the population, whereas the distribution of wealth – considered a proxy of success – follows typically a power law (Pareto law). Such a discrepancy between a Normal distribution of inputs, with a typical scale, and the scale invariant distribution of outputs, suggests that some hidden ingredient is at work behind the scenes. In this paper, with the help of a very simple agent-based model, we suggest that such an ingredient is just randomness. In particular, we show that, if it is true that some degree of talent is necessary to be successful in life, almost never the most talented people reach the highest peaks of success, being overtaken by mediocre but sensibly luckier individuals. As to our knowledge, this counterintuitive result – although implicitly suggested between the lines in a vast literature – is quantified here for the first time. It sheds new light on the effectiveness of assessing merit on the basis of the reached level of success and underlines the risks of distributing excessive honors or resources to people who, at the end of the day, could have been simply luckier than others. With the help of this model, several policy hypotheses are also addressed and compared to show the most efficient strategies for public funding of research in order to improve meritocracy, diversity and innovation.

Comments are, as always, welcome!

Hamiltonian Poetry

Posted in Poetry, The Universe and Stuff with tags , , , , , , on January 8, 2018 by telescoper

I posted a couple of items last week inspired by thoughts of the mathematician William Rowan Hamilton. Another thing I thought I might mention about Hamilton is that he also wrote poetry, and since both science and poetry feature quite regularly on this blog I thought I’d share an example.

In fact during the `Romantic Era‘ (in which Hamilton lived) many scientists wrote poetry related either to their work or to nature generally. One of the most accomplished of these scientist-poets was chemist and inventor Humphry Davy who, inspired by his friendship with the poets Wordsworth and Coleridge, wrote poems throughout his life. Others to do likewise were: physician Erasmus Darwin; and astronomer William Herschel (who was also a noted musician and composer),

William Rowan Hamilton interests me because seems to have been a very colourful character as well as a superb mathematician, and because his work relates directly to physics and is still widely used today. Interestingly, he was a very close friend of William Wordsworth, to whom he often sent poems with requests for comments and feedback. In the subsequent correspondence, Wordsworth was usually not very complimentary, even to the extent of telling Hamilton to stick to his day job (or words to that effect). What I didn’t know was that Hamilton regarded himself as a poet first and a mathematician second. That just goes to show you shouldn’t necessarily trust a man’s judgement when he applies it to himself.

Here’s an example of Hamilton’s verse – a poem written to honour Joseph Fourier, another scientist whose work is still widely used today:

Hamilton-for Fourier

If that’s one of his better poems, then I think Wordsworth may have had a point!

The serious thing that strikes me is not the quality of the verse, but how many scientists of the 19th Century, Hamilton included, saw their scientific interrogation of Nature as a manifestation of the human condition just as the romantic poets saw their artistic contemplation. It is often argued that romanticism is responsible for the rise of antiscience. I’m not really qualified to comment on that but I don’t see any conflict at all between science and romanticism. I certainly don’t see Wordsworth’s poetry as anti-scientific. I just find it inspirational:

I HAVE seen
A curious child, who dwelt upon a tract
Of inland ground, applying to his ear
The convolutions of a smooth-lipped shell;
To which, in silence hushed, his very soul
Listened intensely; and his countenance soon
Brightened with joy; for from within were heard
Murmurings, whereby the monitor expressed
Mysterious union with its native sea.
Even such a shell the universe itself
Is to the ear of Faith; and there are times,
I doubt not, when to you it doth impart
Authentic tidings of invisible things;
Of ebb and flow, and ever-during power;
And central peace, subsisting at the heart
Of endless agitation.

Science and Innovation after Brexit

Posted in Politics, Science Politics with tags , , on September 7, 2017 by telescoper

I’ve been busy most of today so I only have a little time for a short post pointing out that the long-awaited `position paper’ about collaboration on science and innovation between the UK and EU after Brexit has now been published. Those of you intending to remain in the United Kingdom if and when it leaves the European Union might be interested in reading it. I say `might be’ rather than `will be’ as it doesn’t really say anything concrete about anything.

Here’s the overall summary:

In preparing to leave the EU, one of the UK’s core objectives is to “seek agreement to continue to collaborate with European partners on major science, research, and technology initiatives”. It is the UK’s ambition to build on its unique relationship with the EU to ensure that together we remain at the forefront of collective endeavours to improve the world in which we live. The UK believes this is in the joint interest of the UK and EU, and would welcome discussion on how best to shape our future partnership in this area.

The answer to the last bit is, of course, easy. The best way to shape our future partnership in this area is unquestionably for the United Kingdom to remain in the European Union. This document says as much itself. As with most of these papers it consists primarily of a long list of the benefits in this area that the United Kingdom has enjoyed as a direct result of our membership of the European together with a desire to keep most of them after our departure. It offers no real ideas as to how to square the many circles that would involve. In particular, many EU schemes, including those funded by the European Research Council, depend on the freedom of movement the European Union guarantees. Given the leaked Home Office document outlining how it intends to deter EU citizens from coming here I don’t see how we can possibly remain an attractive destination for scientists, or anyone else for that matter.

Meanwhile, today, Parliament is debating the European Union Withdrawal Bill which, if passed, would give the Government sweeping powers – the so-called `Henry VIII’ powers – to bypass Parliament and directly repeal or amend any law it doesn’t like the look of without debate. This is exactly the right-wing power grab that many of who voted Remain feared would happen. If this Bill passes without significant amendment then we can say goodbye to our parliamentary democracy. The parallel with the Enabling Act of 1933 that gave absolute power to Adolf Hitler is frightening.

March for Science – Cardiff

Posted in Politics, Science Politics with tags , , , on April 21, 2017 by telescoper

MFS
Just a quick note to say that tomorrow I’ll be attending the Cardiff March for Science, which is one of a series of events happening around the world. I quote:

The March for Science is a celebration of science.  It’s not only about scientists and politicians; it is about the very real role that science plays in each of our lives and the need to respect and encourage research that gives us insight into the world.

The Cardiff March starts with a rally at 10am on the steps of the Senedd in Cardiff Bay and is followed by a march around the bay to Techniquest for a science event there to which families with children are particularly welcome. It should be a fun occasion  There’s a science-themed fancy dress competition. I’ll be going as a middle-aged man with a beard.

For further details see here or follow the Twitter feed:

 

 

Lord Rees on the Threat to UK Science

Posted in Politics, Science Politics, The Universe and Stuff with tags , , , on October 6, 2016 by telescoper

In case you missed the comments by Lord Rees on Newsnight in the wake of the announcement of this year’s Nobel Prizes for Physics, here is a video.

Martin is always impeccably polite but I sense he must have been outraged by the statements made by Home Secretary Amber Rudd at the Conservative Party Conference this week, some of which seem to have been taking directly from Mein Kampf. Prior to this interview, the most extreme word I’ve ever hard Martin use was “reprehensible” – and that on an occasion when he was clearly angry. His use of the word “deplorable” here is very significant.

Quite apart the threat to science, I have to admit I’m extremely worried about the direction this country is taking. Perhaps someone should tell Prime Minister Theresa May that the referendum wasn’t about leaving the League of Nations and that this isn’t 1933. The parallels with Germany are striking. In that case it didn’t end with the identification and deportation of foreign workers. Yesterday Theresa May stated that anyone who describes themselves as a “Citizen of the World” is really a “Citizen of Nowhere”. I’ve never felt less at home in my own country than I do now.

A few days before the referendum a wrote a post that included this:

Of course I’m not saying that all those who want the UK to Leave the EU are fascists. Far from it. Many – indeed the majority – are reasonable, civilised people. But like it or not, if you vote Leave you’re voting the way the far right want you to vote. I for one will not take a single step in that direction. Fascism only needs a foot in the door. I fear that the domestic political consequences of BrExit will give it far more than that. Once they get hold of it, we’ll never get our country back.

My fear is even more real now than it was then.