Google Citations

Just time for a quick post this morning to pass on the news that Google Citations is now openly available. I just had a quick look at my own bibliometric data and, as far as I can tell, it’s pretty accurate. As well as total citations, Google Scholar also produces an h-index and something called the i10-index (which is just the number of papers with more than 10 citations). It also gives the corresponding figures for the past 5 years as well as for the entire career of a given researcher.

I’ve bragged blogged already about my most popular paper citation-wise, which has 287 citations on Google Scholar, which doesn’t exactly make it a world-beater but I’m still quite please with its impact. What I find particularly interesting about that paper is its longevity. This paper was published in 1991, i.e. 20 years ago, but I  recently looked on the ADS system at its citation history and found the following:

Curiously, it’s getting more citations now than it did when it was first published. I’ve got quite a few “slow burners” like this, in fact, and many of the citations listed for me in the last 5 years actually stem from papers written much earlier. Unfortunately, although I think this steady rate of citation is some sort of indicator of something or other, this is exactly the wrong sort of paper for the Research Excellence Framework, as it is only papers that are published within the roughly 5-year REF window that are taken into account. It would be more useful for the REF panels if the “5-year” window listed citations only to those papers actually published within the last five years. I wonder how the panel will try to use this limited information in assessing the true quality of  a paper?

I should also say that although this paper is, by a large margin, the nearest I’ve got to the citation hit parade, I don’t think it’s by any means the best paper I’ve ever written.

Another weakness is that Google Scholar doesn’t give a normalized h-index (i.e. one based on citations shared out amongst the authors of multi-author papers).

Still, you can’t have everything. Now that this extremely useful tool is available (for free) to all scientists and other denizens of the interwebs, I re-iterate my point that the panels involved in the assessing research for the Research Excellence Framework should use it rather than the inferior commercial versions, which are much less accurate.



18 Responses to “Google Citations”

  1. As I’ve said before, using bibliometry to assess scientific quality is not much better than using the top-10 music charts to assess quality in music. Still, if it must be done, I think that the g-index is better than the h-index (which at least avoids many problems with more traditional schemes; the g-index retains these and adds another advantage or two).

    As you point out, splitting the credit between the authors is an important item which is missing, but is essentially impossible to do automatically. Unless authors put percentages after their names (and of course if the add up to 100%!), I see no way around this.

    A related problem is the threshold above which one moves from being mentioned in the acknowledgements to being a co-author. (This problem would go away if the percentages were mentioned, since someone who had just crossed the threshold would have a low percentage).

    (I don’t think the number of pages in an article should enter into the calculation at all. Apart from the fact that the number of words per page varies from journal to journal, at the end of the day it is the citations which count (or, at least, which are counted); if a 50-page paper really is 50 times (or whatever) more important than a 1-page paper, presumably that would be reflected in the citation count. Also, the number of pages in the paper is only a rough indicator of the amount of work which went into the science which is written up in the paper.)

  2. I’m not a big user of Google. I use it as a search engine, and sometimes Google Maps, and a while back I registered just for the purpose of being able to comment on some blogs (figure out which ones!) which require one to log in (and sometimes use that profile elsewhere, since it links to one of my favourite pictures). I’ve never used Gmail or other more “advanced” features. The email address I used to register the profile mentioned above works for the citation stuff as well, so I decided to give it a go.

    Extremely simple to get up and running, at first glance quite accurate and a sensible user interface. It looks like everything which should be there is there and nothing which shouldn’t be there is there. (Apparently, not all fields are covered, since it doesn’t seem to pick up my non-astronomy publications.) This despite the fact that the precise form of my name varies from paper to paper (Phillip, P., Ph., P.J. etc).

    There is some dodgy conversion from LaTeX, but it’s probably difficult to get that right all the time. One conference proceeding appears under the bizarre title “The Netherlands”. OK, that does appear in the list of institutes, but it’s not clear why (only) this publication shows such a goof.

    I listed all previous affiliations. Nice would be separate boxes for current and previous affiliations. I also didn’t see the total number of papers; that would be interesting, if only to see how much there is to browse.

    Also interesting is that while most (all?) proceedings contributions have the proper indication of where they were officially publshed, only some journal papers do, while others mention arXiv, although all the journal papers are on the arXiv and almost all the arXiv stuff was published in a journal. It would be interesting to know why this is the case (apparently, it depends on the journal).

  3. I went over to Google Citations to have a look at how accurately it handled my papers. Sadly it doesn’t work as well in my case as in yours: it only picked up about 30% of the citations recorded on InSpire or the NASA ADS system (which agree reasonably well with each other).

    As I’m at the stage of my scientific life where (rightly or wrongly) number of citations appears to matter quite a lot when it comes to job prospects, Google Citations is some way from being my favourite reference tool!

    • What is your field? Are ADS and Google so different only for you or also for colleagues in the same field? (In my case, a brief check has Google showing more citations than ADS.)

  4. Greg M. Gupton Says:

    First, I’m happy and relieved that Google is continuing to invest in Scholar. It had felt like a stagnant project without priority in Google’s overall plan. So … Thank!

    Sadly, the new ‘Citations’ accuracy is poor for my own work. One paper is scattered across ten (10) …um… formulations and ‘Citations’ states that 79 papers cite it. I just verified that this “79” includes (also) key word hits in the body-text without a citation in the papers’ reference sections. Also, I’ve verified that 115 papers (and likely more) do actually list my/this paper in their reference sections.

  5. Had a look at this – it seems wierd. For most of my papers google gives a higher citation rate than NASA ADS (e.g. 537 for Dunlop & Peacock 1990 versus 441 citations on NASA ADS), but then my third most cited first-author paper which has 309 citations on ADS basically disappears on Google – I found it down at 45 citations! There seems no way to fix this sort of thing so it is a bit worrying – clearly some sort of serious bug as there is nothing specially wierd about where this paper was published (MNRAS) or where most of the citations come from.

    • Generally, google citations gives higher numbers than ADS for mine; I assume that’s because it includes sources not included in the ADS database. The fact that some papers have many fewer on google citations is therefore perplexing and worrying. There may be a simple bug that can be fixed. If so, that will require looking at individual papers and trying to see what’s gone wrong. I suggest reporting such anomalies. I didn’t find any with mine.

      • like jim i have a bunch of fragmented paper citations. the overall total is ~20% higher than ADS (which makes me worry) and even after editing the ~40% of dross out… things didn’t settle down. i had a long list of papers with ~10-50 citations where gCite (or whatever we should call it) returned 0.

        for astronomy – ADS is still the gold standard – but of course REF isn’t just about astronomy… and this may be better than SCOPUS (although WoK has got better in my view).

      • telescoper Says:

        I’m not worried about the fact that Google returns higher numbers than ADS because the database it’s drawing from is much much larger; I know that some of my papers are cited in fields outside physics and astronomy so this doesn’t surprise me.

        However, the missing and/or fragmented papers are a worry, and it’s useful to report them to find out what’s going wrong.

      • …the problem is that it appears to overcite highly cited papers [perhaps due to unidentified duplicate citations from arxiv and the corresponding journal papers?] and undercite the newer/less cited stuff. at least compared to ADS – where i can easily check the quality/type of the citing articles (few/none of which are outside “astronomy” for me).

        so i think its got a long way to go before i’ll be using it instead of ADS (although i do admit i use general google searches instead of ADS for trying to find general material, rather than specific papers).

  6. Looks like those for whom it works (maybe more or less coincidentally) should keep an eye on it in the future; even if it is reasonably correct (and how does one verify that in practice?) now it might not be in the future. On the other hand, the fact that ADS and Google differ, without further investigation, means that at least one of them is wrong.

  7. Maybe I was blinded a bit by the fact that it was easy to set up and has a nice interface. I also notice some of my papers being split (say, in one case with a Roman numeral and in another case with an Arabic numeral in the title). Before you conclude that it shows too few citations, you need to make sure that the paper in question doesn’t appear more than once in the list under slightly different titles.

  8. Avery Meiksin Says:

    I found it rather less reliable than ADS, as have others. In particular, looking at the lower end of the Google citations shows lots of missing papers that the ADS catches in my case. There’s no obvious pattern to why Google missed them: papers in standard journals that it recognizes over a range of years don’t show up.

    Still, it’s a start. A couple of useful features that could give it an advantage over the ADS is to allow the user to group papers and have the statistics run on separate groups (eg, by user-selected
    topics or years). A feature ADS has that Google lacks is the facility to tally citations only by refereed papers, something the REF should do (if it doesn’t plan to already).

  9. I see Peter is still listed as being at Cardiff. 😐

  10. It has gone downhill. The new interface is worse than the old one, automatic updates can no longer be confirmed first, automatic updates are wrong, and the helpdesk ignores users.

    Yeah, for some it shows more citations than ADS. Yes, ADS is not always current. But people should check how much of Google Scholar is correct.

    I have a very simple, very clear-cut problem. Repeated attempts to contact the support people have resulted in nothing.

    Google used to be an innovative company, like Saab used to be. Where is Saab now?

    • Monthly update: they redesigned the look and feel for no obvious reason, it is worth than before, and really simple mistakes are not corrected, even when email is sent to the address which is supposed to help for things like this.

      It looks like whoever set this up left the company and it just keeps running but is not maintained anymore.

      The fact that Google could easily search for comments like this one and contact me for further information (not that they don’t have all that they need) shows that they don’t really care about this.

      Anyone who uses Google instead of ADS or some other good service should not be allowed to make decisions based on it. It’s not a question of who has the most references, or updates the list the most quickly etc., but rather a question of who (Google!) does things obviously wrong, like merging two papers which are not the same paper, thus distorting the citation count for both.

      This is really, really basic bibliometry stuff. If they can’t get this right, how can one trust them on other stuff?

    • By coincidence, just two days shy of three years since my previous rant. Still the same problems, the interface is now even worse, and I have still not found out how to contact anyone to fix obvious bugs.

      If I make changes myself, they are revoked. There is no rhyme nor reason to the mistakes made.

      Has anyone had any success in contacting a real person who knows how to fix these problems?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: