Political Correlation

I was just thinking that it’s been a while since I posted anything in my bad statistics category when a particularly egregious example jumped up out of this week’s Times Higher and slapped me in the face. This one goes wrong before it even gets to the statistical analysis, so I’ll only give it short shrift here, but it serves to remind us all how feeble is many academic’s grasp of the scientific method, and particularly the role of statistics within it. The perpetrator in this case is Paul Whiteley, who is Professor of Politics at the University of Essex. I’m tempted to suggest he should go and stand in the corner wearing a dunce’s cap.

Professor Whiteley argues that he has found evidence that refutes the case that increased provision of science, technology, engineering and maths (STEM) graduates are -in the words of Lord Mandelson – “crucial to in securing future prosperity”. His evidence is based on data relating to 30 OECD countries: on the one hand, their average economic growth for the period 2000-8 and, on the other, the percentage of graduates in STEM subjects for each country over the same period. He finds no statistically significant correlation between these variates. The data are plotted here:

This lack of correlation is asserted to be evidence that STEM graduates are not necessary for economic growth, but in an additional comment (for which no supporting numbers are given), it is stated that growth correlates with the total number of graduates in all subjects in each country. Hence the conclusion that higher education is good, whether or not it’s in STEM areas.

So what’s wrong with this analysis? A number of things, in fact, but I’ll start with what seems to me the most important conceptual one. In order to test a hypothesis, you have to look for a measurable effect that would be expected if the hypothesis were true, measure the effect, and then decide whether the effect is there or not. If it isn’t, you have falsified the hypothesis.

Now, would anyone really expect the % of students graduating in STEM subjects  to correlate with the growth rate in the economy over the same period? Does anyone really think that newly qualified STEM graduates have an immediate impact on economic growth? I’m sure even the most dedicated pro-science lobbyist would answer “no” to that question. Even the quote from Lord Mandelson included the crucial word “future”! Investment in these areas is expected to have a long-term benefit that would probably only show after many years. I would have been amazed had there been a correlation between measures relating to such a short period, so  absence of one says nothing whatsoever about the economic benefits of education in STEM areas.

And another thing. Why is the “percentage of graduates” chosen as a variate for this study? Surely a large % of STEM graduates is irrelevant if the total number is very small? I would have thought the fraction of the population with a STEM degree might be a better choice. Better still, since it is claimed that the overall number of graduates correlates with economic growth, why not show how this correlation with the total number of graduates breaks down by subject area?

I’m a bit suspicious about the reliability of the data too. Which country is it that produces less than 3% of its graduates in science subjects (the point at the bottom left of the plot). Surely different countries also have different types of economy wherein the role of science and technology varies considerably. It’s tempting, in fact, to see two parallel lines in the above graph – I’m not the only one to have noticed this – which may either be an artefact of small numbers chosen or might indicate that some other parameter is playing a role.

This poorly framed hypothesis test, dubious choice of variables, and highly questionable conclusions strongly suggest that Professor Whiteley had made his mind up what result he wanted and simply dressed it up in a bit of flimsy statistics. Unfortunately, such pseudoscientific flummery is all that’s needed to convince a great many out there in the big wide world, especially journalists. It’s a pity that this shoddy piece of statistical gibberish was given such prominence in the Times Higher, supported by a predictably vacuous editorial, especially when the same issue features an article about the declining standards of science journalism. Perhaps we need more STEM graduates to teach the others how to do statistical tests properly.

However, before everyone accuses me of being blind to the benefits of anything other than STEM subjects, I’ll just make it clear that, while I do think that science is very important for a large number of reasons, I do accept that higher education generally is a good thing in itself , regardless of whether it’s in physics or mediaeval latin, though I’m not sure about certain other subjects.  Universities should not be judged solely by the effect they may or may not have on short-term economic growth.

Which brings me to a final point about the difference between correlation and causation. People with more disposal income probably spend more money on, e.g., books than people with less money. Buying books doesn’t make you rich, at least not in the short-term, but it’s a good thing to do for its own sake. We shouldn’t think of higher education exclusively on the cost side of the economic equation, as politicians and bureaucrats seem increasingly to be doing,  it’s also one of the benefits.


Share/Bookmark

Advertisements

9 Responses to “Political Correlation”

  1. It completely ignores the pseudo-economy of virtual money making by the financial sector. Is economic growth actually a measurement of tangible developments such as employment, manufacturing, medical advances etc. or is the significant part of economic growth made up by how many zeros somebody adds to a balance sheet at a central bank?

    Contributions to economic growth by new graduates such as myself probably wont be seen for another 20-30 years, where I’m reaching the top of my career and actually leading/making advancements. Up until then you could say that I’ll be in ‘maintenance’ within the economy – keeping things ticking over and supporting already present infrastructure.

    Of course how many people reach the top of their careers to lead, make advancements and improve the overall economy depends upon the support system in place i.e. what’s available today, to draw people into the sector, and how many people employed within the sector; more people means a greater scope of more pioneers.

    It really is simple stuff.

    The best example I can think of is the LASER. Blue-sky thinking that game the world a technology it had no need for. A technological push rather than pull. Today’s world would be impossible without the laser. Many say they could function without it, only to forget about their CD players, telecoms, hospital treatment etc. We need more of this technological/scientific push for the future economy.

  2. Bryn Jones Says:

    Peter,

    One issue that irritates me is inaccurately or sloppily presented data.

    The figure you reproduced from the published analysis has one axis labelled “Economic growth % (2000-8)”. From memory I know that advanced industrialised economies had G.D.P. growth rates of typically 1.5 to 3.5% per annum before the recession, and some developing countries had rates larger than this. So the growth between 2000 and 2008 would be roughly 10 to 30% for advanced industrialised countries. Clearly the graph shows the mean percentage growth per annum between 2000 and 2008. That is not what it states.

    The information about the correlation coefficient on the graph gives the value but no error estimate.

    Bryn.

  3. If economic growth is measured in GDP, then I’m not sure that I would expect to really see a correlation between science graduates and economic growth. I looked up GDP to refresh my memory on wikipedia, since it’s been awhile since I took macroeconomics, and really, I don’t think GDP is a good indicator of “growth”. GDP = private spending + investment (purchasing of tools basically, not financial vehicles) + government spending + (exports-imports). So where is the growth? Well, I do recall that in the US, growth was primarily caused by foreign investment and government spending. But, really, that is just an illusion. The primary place of growth should really be simply investment. Investment in GDP is supposed to be investment in infrastructure, factories, technology, tools, etc. I mean, how can spending really be considered growth? Doesn’t it depend on what people are buying? If people buy a bunch of shiny new things like tv’s, big SUVs, weapons, and fastfood, how does this really contribute to growth? In fact, the same can be said for investment, actually. If businesses are investing in bad tools, and unhelpful infrastructure, then how can it really be considered growth?

    I think the bottom line is, that measurement of growth doesn’t really have a very good methodology. Measuring it doesn’t seem to be very objective. Perhaps measuring standard of living would be a better method, which GDP is actually supposed to measure, but I don’t see how it could be that great a tool for measuring standard of living. Besides, what does “standard of living” really mean? If a person isn’t starving, isn’t sick, and has books to read, and can work on applying their knowledge to the improvement of the human condition, then isn’t that the highest standard of living that is possible? If that’s the case, then I could see how science graduates would contribute to this fuzzy concept of standard of living…but I think this idea of economic growth would really need to be clarified before trying to draw a scientific conclusion from it.

  4. telescoper Says:

    In view of the number of comments on this article trapped by my spam filter, let me remind readers that I do not accept anonymous comments using fake email addresses.

  5. It is obvious that as a Professor of Politics Paul Whitely has a vested interest in showing that investment in Higher Education – but not only STEM subjects is beneficial.

    I think that we simply need to consider what the actual end result of most students studies is in order to gauge the effectiveness (although statistical studies are more scientific and obviously useful when conducted properly).

    Many science undergraduates go on to complete original research as PhD students which may then be built upon and lead to new technologies and economic growth – in science even a negative result is important progress.

    Compare this to the theses of Arts students which are often read by very few people and of little or no application in the real world. I mean if you look at Arts and Science research almost any given time period, which has had the greater effect – the works produced by the Arts or the works of Science?

    Even the more applied parts of the Arts such as the social sciences are pretty useless as what purpose does it really serve to come up with studies showing that, for example, inequality decreases the quality of life in general if no one is willing to implement any changes based on the research and there are just as many studies and thinktanks with opposing studies as the subject cannot be objectively settled.

    I realise I have been talking about research but in Physics it seems most students do go on to do PhD’s, and those that go into industry are often employed far more easily, and in underfilled areas of our economy (even if it is in the detested City) than Arts graduates.

    Why should the taxpayers money be used to subsidise students to study subjects which will neither benefit themselves (due to poor prospects upon graduation) or society (poor prospects make the return via taxation minimal, whilst there is little tangible non-monetary benefit to society as in the case of new technologies developed by often poorly-paid scientists)?

    I know this might just sound like another Arts vs Science troll, but every time I have seen the argument for Blue Skies research made successfully it has been almost entirely dependent upon cases in the sciences (especially physics with the laser and also with important parts of the internet and medical imaging from particle physics). I have yet to see anyone successfully argue why we should subsidise Arts students and research.

  6. telescoper Says:

    I’m quite surprised that no astronomer has yet suggested dividing the points into Type I (above the red line) and Type II (below the red line). I’m sure each group separately has a significant value of the correlation coefficient.

  7. Anton Garrett Says:

    A professor of what? Modern history is a proper academic subject. But the idea that politics is fit to be studied in isolation alongside mathematics, philosophy, history, physics, biochemistry, English literature, Classics is risible. ‘Management’ is another shallow impostor.

    Anton

    • telescoper Says:

      According to his CV, he’s “Professor of Government”. I wonder how much experience he has of being in government?

      In the post I suggested that perhaps he should stand in the corner wearing a dunce’s cap, but working at the University of Essex is probably pretty much equivalent to that anyway.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: