Guest Post – Bayesian Book Review

My regular commenter Anton circulated this book review by email yesterday and it stimulated quite a lot of reaction. I haven’t read the book myself, but I thought it would be fun to post his review on here to see whether it provokes similar responses. You can find the book on Amazon here (UK) or here ( USA). If you’re not completely au fait with Bayesian probability and the controversy around it, you might try reading one of my earlier posts about it, e.g. this one. I hope I can persuade some of the email commenters to upload their contributions through the box below!


The Theory That Would Not Die: How Bayes’ Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy

by Sharon Bertsch Mcgrayne

I found reading this book, which is a history of Bayes’ theorem written for the layman, to be deeply frustrating. The author does not really understand what probability IS – which is the key to all cogent writing on the subject. She never mentions the sum and product rules, or that Bayes’ theorem is an easy consequence of them. She notes, correctly, that Bayesian methods or something equivalent to them have been rediscovered advantageously again and again in an amazing variety of practical applications, and says that this is because they are pragmatically better than frequentist sampling theory – ie, she never asks the question: Why do they work better and what deeper rationale explains this? RT Cox is not mentioned. Ed Jaynes is mentioned only in passing as someone whose Bayesian fervour supposedly put people off.

The author is correct that computer applications have catalysed the Bayesian revolution, but in the pages on image processing and other general inverse problems (p218-21) she manages to miss the key work through the 1980s of Steve Gull and John Skilling, and you will not find “Maximum entropy” in the index. She does get the key role of Markov Chain Monte Carlo methods in computer implementation of Bayesian methods, however. But I can’t find Dave Mackay either, who deserves to be in the relevant section about modern applications.

On the other hand, as a historian of Bayesianism from Bayes himself to about 1960, she is full of superb anecdotes and information about
people who are to us merely names on the top of papers, or whose personalities are mentioned tantalisingly briefly in Jaynes’ writing.
For this material alone I recommend the book to Bayesians of our sort and am glad that I bought it.

7 Responses to “Guest Post – Bayesian Book Review”

  1. Anton Garrett Says:

    Yes that’s me Phillip. As to whether it is online – ask Peter.

  2. telescoper Says:

    The paper was published in Comments on Astrophysics, Vol. 17, p23, but that journal doesn’t have a website. Unfortunately I’ve lost the .tex file for the paper so I can’t post it to the arXiv. As far as I’m aware nobody else has put it up on the web…

  3. Anton Garrett Says:

    It was on the anthropic principle but we also enjoyed ourselves by bashing non-Bayesian probability and the Copenhagen wimp-out interpretation (and some other views) of quantum mechanics. I published a paper on the anthropic principle in a proceedings of the Maximum Entropy conference series which certainly had some overlap. The basic idea is that the Bayesian view of probability is the razor needed to excise the ample nonsense written about the anthropic principle from the plentiful good sense that has also been written about it.

  4. telescoper Says:

    I have a hard copy which I could scan, I guess, but I’m wary of copyright infringement if I put it on the web anywhere…

  5. Anton Garrett Says:

    Peter: While I know that the journal in question had a sensible editor, your comment shows EXACTLY what is wrong with science publishing: a senior scientist dare not put his own work up on the Web for fear of breaking copyright. That same Web will soon end this state of exploitation, I trust.

  6. Dear Anton

    Thanks for pointing us to this book.
    I will certainly give it a read especially since I teach a graduate
    course on Bayesian Data Analysis here at the University at Albany and
    new perspectives on the history will be useful.

    It is disappointing that the Cox-Jaynes thread in the history is
    missing since it is the one that I have found that has led to *deep
    understanding*. I was at a NASA meeting a few years ago and a person
    (a good guy who I will not name) from the ISBA crowd gave a tutorial
    and at the end mentioned several introductory texts. Absent from the
    tutorial was any mention of Cox, Jaynes, etc. I spoke up and
    mentioned this important thread (which as you probably know led to
    many of us at NASA Ames: Cheeseman, Stutz, Scargle, Wolpert, myself,
    etc.). I also mentioned Devinder’s text as particularly accessible
    and useful, as well as MacKay’s, Gregory’s and Bretthorst’s. I also
    recall at the ISBA meeting in Chile in 2004 or so Arnold Zellner
    scolding the crowd for neglecting 1/2 of the picture: entropy and
    information. As revered as he was by that community, his outburst was
    laughed off as the rantings of a crazy old man.
    Sad really.

    As John Skilling can attest, I myself have found Cox’s insights to be
    profound. I have taken them ever so seriously and with surprising and
    delightful results. The lesson I have taken away is that all one
    needs to do is quantify the order in what you are looking at.
    Consistent quantification may not be useful, but it can’t be wrong!
    This makes it a great place to start. The result is that the
    constraint equations enforcing consistent quantification will result
    in the laws that govern the quantified system. This is precisely why
    laws reflect an underlying order—laws derive from that order.

    This idea of generalizing an algebra has led not only to a theory of
    questions (which, for those of you who have been following, I now
    understand how to handle properly), but also has led several of us
    (Goyal, Skilling and myself) to a derivation of the Feynman rules of
    quantum mechanics based on similar symmetries (inspired, in part, by
    the earlier works of Tikochinsky, Gull and Caticha), as well as a
    novel derivation of special relativity and the geometry and
    dimensionality of space (which was quite unexpected and quite unlike
    anything I have yet found):

    Goyal, Knuth, Skilling, Phys Rev A, 2010 ( )
    Goyal, Knuth, Symmetry 2011 ( )
    Knuth, Bahreyni 2010 ( )
    Skilling, Knuth 2010 ( )

    Propelled and emboldened by these advances, my colleague Keith Earle
    and myself (the most unlikely of suspects, as neither of us are
    theorists by training) are working on a derivation of the Dirac equation and have already obtained new insights as well as a glimpse into how to handle both electromagnetism and gravity in concert with quantum mechanics. Keith is working from the middle up ( ) and I am working from the bottom to the middle. This is what I mainly am focused on now.

    At this point in time, the ideas of Cox have inspired us to derive far more than probability theory!

    Bayes was just the beginning.
    Cox and Jaynes, taken seriously, have led to deep understanding and
    their insights have now taken us light-years beyond statistics.
    (I haven’t even begun to mention the advances in Maximum entropy)

    Don’t despair, THE BOOK hasn’t been written yet.
    I seriously feel that in the long run, this thread through the long
    story of probability will turn out to be the most amazing one!

    Kevin Knuth

  7. Anton Garrett Says:

    Phillip: Isn’t the point that you submit to the arXiv *before* you sign away copyright to a publisher? I don’t know (do you Peter?) if publishers ever threatened not to publish stuff that had appeared on a scientist’s website, or on the arXiv in its early days; but if so then we won that one.

    The original function of academic publishers was to disseminate information. Then a quality control element came in (via the refereeiing system). Now the internet has totally removed the original function. It should not be beyond the scientific community to work out some kind of quality control method for online publishing at a tiny fraction of the cost of today’s journals, and end the iniquity of signing away copyright in our own work.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: