Who needs critics? Or peer review for that matter…

No time for a proper post today so I’m going to rehash an old piece from about six years ago. In particular I direct your attention to the final paragraph in which I predict that peer review for academic publications will soon be made redundant. There has been quite a lot of discussion about that recently; see here for an example.

Critics say the stangest things.

How about this, from James William Davidson, music critic of The Times from 1846:

He has certainly written a few good songs, but what then? Has not every composer that ever composed written a few good songs? And out of the thousand and one with which he deluged the musical world, it would, indeed, be hard if some half-dozen were not tolerable. And when that is said, all is said that can justly be said of Schubert.

Or this, by Louis Spohr, written in 1860 about Beethoven’s Ninth (“Choral”) Symphony

The fourth movement is, in my opinion, so monstrous and tasteless and, in it’s grasp of Schiller’s Ode, so trivial that I cannot understand how a genius like Beethoven could have written it.

No less an authority than  Grove’s Dictionary of Music and Musicians (Fifth Edition) had this to say about Rachmaninov

Technically he was highly gifted, but also severely limited. His music is well constructed and effective, but monotonous in texture, which consists in essence mainly of artificial and gushing tunes…The enormous popular success some few of Rachmaninov’s works had in his lifetime is not likely to last and musicians regarded it with much favour.

And finally, Lawrence Gillman wrote this in the New York Tribune of February 13 1924 concerning George Gershwin’s Rhapsody in Blue:

How trite and feeble and conventional the tunes are; how sentimental and vapid the harmonic treatment, under its disguise of fussy and futile counterpoint! Weep over the lifelessness of the melody and harmony, so derivative, so stale, so inexpressive.

I think I’ve made my point. We all make errors of judgement and music critics are certainly no exception. The same no doubt goes for literary and art critics too. In fact,  I’m sure it would be quite easy to dig up laughably inappropriate comments made by reviewers across the entire spectrum of artistic endeavour. Who’s to say these comments are wrong anyway? They’re just opinions. I can’t understand anyone who thinks so little  of Schubert, but then an awful lot of people like to listen what sounds to me to be complete dross.

What puzzles me most about the critics is not that they make “mistakes” like these – they’re only human after all – but why they exist in the first place. It seems extraordinary to me that there is a class of people who don’t do anything creative themselves  but devote their working lives to criticising what is done by others. Who should care what they think? Everyone is entitled to an opinion, of course, but what is it about a critic that implies we should listen to their opinion more than anyone else?

(Actually, to be precise, Louis Spohr was also a composer but I defy you to recall any of his works…)

Part of the idea is that by reading the notices produced by a critic the paying public can decide whether to go to the performance, read the book or listen to the record. However, the correlation between what is critically acclaimed and what is actually good (or even popular) is tenuous at best. It seems to me that, especially nowadays with so much opinion available on the internet, word of mouth (or web) is a much better guide than what some geezer writes in The Times. Indeed, the   Opera reviews published in the papers are so frustratingly contrary to my own opinion that I don’t  bother to read them until after the performance, perhaps even after I’ve written my own little review on here.  Not that I would mind being a newspaper critic myself. The chance not only to get into the Opera for free but also to get paid for spouting on about afterwards sounds like a cushy number to me. Not that I’m likely to be asked.

In science,  we don’t have legions of professional critics, but reviews of various kinds are nevertheless essential to the way science moves forward. Applications for funding are usually reviewed by others working in the field and only those graded at the very highest level are awarded money.  The powers-that-be are increasingly trying to impose political criteria on this process, but it remains a fact that peer review is the crucial part of the process. It’s not just the input that is assessed either. Papers submitted to learned journals are reviewed by (usually anonymous)  referees, who often require substantial changes to be made the work before the work can be accepted for publication.

We have no choice but to react to these critics if we want to function as scientists. Indeed, we probably pay much more attention to them than artists do of critics in their particular fields. That’s not to say that these referees don’t make mistakes either. I’ve certainly made bad decisions myself in that role,  although they were all made in good faith. I’ve also received comments that I thought were unfair or unjustifiable, but at least I knew they were coming from someone who was a working scientist.

I suspect that the use of peer review in assessing grant applications will remain in place for a some considerable time. I can’t think of an alternative, anyway. I’d much rather have a rich patron so I didn’t have to bother writing proposals all the time, but that’s not the way it works in either art or science these days.

However, it does seem to me that the role of referees in the publication process is bound to become redundant in the very near future. Technology now makes it easy to place electronic publications on an archive where they can be accessed freely. Good papers will attract attention anyway, just as they would if they were in refereed journals. Errors will be found. Results will be debated. Papers will be revised. The quality mark of a journal’s endorsement is no longer needed if the scientific community can form its own judgement, and neither are the monstrously expensive fees charged to institutes for journal subscriptions.

5 Responses to “Who needs critics? Or peer review for that matter…”

  1. John Peacock Says:

    > Actually, to be precise, Louis Spohr was also a composer but I defy you to recall any of his works…

    Well, Spohr is a very fine composer. He was thought the equal of Beethoven in his day; even if we wouldn’t now agree with this judgement, many of his works are unfairly neglected. Best of all perhaps is his chamber music: I’d particularly recommend the Nonet or the Octet for mixed strings & wind. And you can see where he’s coming from with the finale of the 9th: it has the feeling of an experiment, where some parts come off brilliantly but others are just weird.

    • telescoper Says:

      It’s interesting how some reputations survive and others don’t. This morning I woke up to Hummel’s delightful Trumpet Concerto on Radio 3. He was also considered a giant in his time, but less so now…

  2. A critic — at least, a good critic — serves much the same role that a museum docent does. She can tell you not just what a piece makes her feel, but can explain what lines of creative thought the piece belongs in, and how it typifies and how it explores them. The critic can provide the context of the artist and the artwork, can point out things the piece does very well, point out what it does poorly. It isn’t just about `is this any good’, but, `what is this’ and `why is this worth audience’s attention’, and (if it isn’t a new piece) `how has this affected other artworks’.

    A critic reviewing a new piece is likely to get at least some of this wrong, possibly hilariously wrong. After all, the reasonably full context won’t be known for years, possibly generations. That’s part of what makes some critics hilarious from our perspective. An off-base criticism will collapse under the weight of time, if it doesn’t self-immolate. A perceptive criticism will show the reader something new about an artwork.

  3. I would be a lot more in favour of peer review it worked at all as intended.

    Last week I read a 3 page Nature article – which was fine, but lacking in details. Why? Because most of the information was buried in 53(!) pages of supplementary material, that was rather less well written compared to the main paper (and here I am being polite).

    This sort of thing should be exactly what peer review should avoid. But it seems not only have all the incentives gone wrong in terms of how and what gets published, but the peer review process too.

  4. Phillip Helbig Says:

    “However, it does seem to me that the role of referees in the publication process is bound to become redundant in the very near future. Technology now makes it easy to place electronic publications on an archive where they can be accessed freely. Good papers will attract attention anyway, just as they would if they were in refereed journals. Errors will be found. Results will be debated. Papers will be revised. The quality mark of a journal’s endorsement is no longer needed if the scientific community can form its own judgement, and neither are the monstrously expensive fees charged to institutes for journal subscriptions.”

    I’m not so sure. I still subscribe to paper magazines, even though in many cases the same content—and more—is available electronically. Why? Higher signal-to-noise. Some of us can remember when too little information is a problem. Now, the problem is too much information. We need filters, and acceptance by a respected journal is a reasonably good filter.

    Yes, your scenario is technically possible. But we all have too little time as it is. Do you really plan to read papers which have not been accepted by a reputable journal, even if you are not familiar with the author(s)? And check them out at the level a good referee would?

    Another point is that it is not clear what criteria are used by arXiv in determining who can post there in what category. The system is not transparent and there is no appeal process. Rocking the boat risks being banned, which is the modern-day equivalent of excommunication. So it is a stretch to say that “it is easy to place electronic publications on an archive”. Yes, on an archive, but not on the arXiv, which is the only one that matters. (In some cases, a monopoly is a good idea, and this is one. One attraction of arXiv is that it is a one-stop shop. A paper is there, or it is not relevant.) Unless you want to restrict the community to people who already have a good reputation, which I don’t think is your intent.

    I recently had a couple of papers accepted by MNRAS (one on arXiv, the other coming probably this weekend). The referee had some useful comments which improved the papers. I don’t think that I would have received such feedback from the general community.

    Also, a good referee can point out mistakes or the fact that the topic has already been covered by someone else (and verified by yet someone else) and thus avoid publication of wrong and/or redundant papers.

    Since referees work for free, I’m not sure what the issue of journal costs has to do with this.

    You have discussed an “Open Journal of Astrophysics” or whatever. If it is not refereed, it is not needed if it is essentially a pointer to arXiv papers, unless you include some other submission mechanism, which will need at least a bit of work done by someone. If it is refereed then, yes, it might be a good alternative, but again it needs an alternative submission mechanism in addition to arXiv. For various reasons, some people cannot, or don’t want to, post to arXiv before acceptance, or perhaps their institute does not allow it.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: