Citation Analysis of Scientific Categories

I stumbled across an interesting paper the other day with the title Citation Analysis of Scientific Categories. The title isn’t really accurate because not all the 231 categories covered by the analysis are `scientific’: they include many topics in the arts and humanities too. Anyway, the abstract is here:

Databases catalogue the corpus of research literature into scientific categories and report classes of bibliometric data such as the number of citations to articles, the number of authors, journals, funding agencies, institutes, references, etc. The number of articles and citations in a category are gauges of productivity and scientific impact but a quantitative basis to compare researchers between categories is limited. Here, we compile a list of bibliometric indicators for 236 science categories and citation rates of the 500 most cited articles of each category. The number of citations per paper vary by several orders of magnitude and are highest in multidisciplinary sciences, general internal medicine, and biochemistry and lowest in literature, poetry, and dance. A regression model demonstrates that citation rates to the top articles in each category increase with the square root of the number of articles in a category and decrease proportionately with the age of the references: articles in categories that cite recent research are also cited more frequently. The citation rate correlates positively with the number of funding agencies that finance the research. The category h-index correlates with the average number of cites to the top 500 ranked articles of each category (R2 = 0.997). Furthermore, only a few journals publish the top 500 cited articles in each category: four journals publish 60% (σ = ±20%) of these and ten publish 81% (σ = ±15%).

The paper is open access (I think) and you can find the whole thing here.

I had a discussion over lunch today with a couple of colleagues here in Maynooth about using citations. I think we agreed that citation analysis does convey some information about the impact of a person’s research but that information is rather limited. One of the difficulties is that publication rates and citation activity are very discipline-dependent so one can’t easily compare individuals in different areas. The paper here is interesting because it presents an interesting table showing how various statistical citation measures vary across fields and sub-fields;  physics is broken down into a number of distinct areas (e.g. Astronomy & Astrophysics, Particle Physics, Condensed Matter and Nuclear Physics) across which there is considerable variation. How to best to use this information is still not clear..



7 Responses to “Citation Analysis of Scientific Categories”

  1. Opinionated Moron Says:

    “citation activity are very discipline-dependent”
    Indeed. Just to add that cosmology can no longer be considered a single discipline. Even in theoretical cosmology, various groups
    working in different areas don’t have exactly same culture.

    In case of e.g. early Universe researchers the number of co-authors
    is often less than five. The groups mainly focus on model
    building using mainly analytical work. The number of papers
    written by individuals tend to be relatively high. Since doing such
    calculations do not require expensive computing facilities,
    many authors from less affluent countries work in such areas
    e.g. dark energy, inflationary models etc.
    The citation by individuals depends on the total number of researchers working in that area. Hence, such
    authors typically tend to have higher citations.

    In contrast model testing using data involves teams with a much higher number of coauthors. The citations of individuals
    is relatively difficult to work out as it is distributed among team
    members which also include people who are observers or instrumental builders.

  2. It’s like the Drake equation: there are many factors so, yes, all else being equal, more citations means more impact, but all else is rarely, if ever, equal. There are many other factors not mentioned above.

  3. Jim Dunlop Says:

    Citations – a tricky business. However, whatever way one tries to spin it, if citations are low, at some level there is no dodging the point that no-one cares what you have done. Of course, that doesn’t mean it isn’t good work.

  4. Francis Keenan Says:

    Not only are citation rates discipline-dependent, but sub-discipline and perhaps even sub-sub-discipline. Within ‘astrophysics’ I believe that e.g. exoplanets, stellar physics, cosmology have different citation rates, and same with e.g. quantum information and atomic collision processes in atomic physics.

    For REF2014 there was a (HEFCE?) publication on citation rates by discipline and also by sub-discipline, as I recall? Even these were quite general.

  5. Opinionated Moron Says:

    Publishing wrong results can improve citation e.g.,
    neutrinos with speed greater than light, gravity waves from
    cmb, circles in cmb maps etc. One of the many side-effects
    of citation driven research.

  6. I never really thought too much about the differences in citation practice in different disciplines until I started writing HoS papers (i.e., papers on the history of 20th century physics). I quickly found that an interesting historical paper can attract a large number of readers and yet receive a very small number of citations. I suspect this happens because only a small fraction of the readership write historical papers of their own. By contrast, a technical paper might attract a small number of readers, but most of those readers will cite the paper, as it is relevant to their own work.
    I first noticed this when one of our EPJH papers established a record number of downloads for the journal – the same paper has been cited less than 12 times!

    • Indeed. I enjoy reading such papers, and might even write some in the future. I think they do make an “impact” through being read, but there is often little reason to cite such a paper. Not only do citation practices also vary from discipline to discipline, but also from institute to institute, as does the question “when does one move from being mentioned in the acknowledgements to being an author”.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: