Understanding Uncertainty Hardcover – 3 Oct 2006

The common conceptual approaches to probability are often categorized (e.g. in Hacking's textbook An Introduction to Probability and Inductive Logic) as dogmatic Bayesian (Lindley being a prominent adherent), dogmatic frequentist, and eclectic (your reviewer). The bulk of this book is lengthy discussions, with light mathematics and hypothetical examples, of some of the basic textbook topics in probability and statistics. The first third goes slowly from verbal discussion of uncertainty to the fundamental calculus of probabilities, justified by 5 methods: "comparison with a standard, avoidance of Dutch books, the use of scoring rules, an extension of logic, action in the face of uncertainty". The remainder continues with topics such as Bayes rule, Simpson's paradox, variation and the Binomial/Poisson/Normal distributions, randomized experiments, expected utility, decision trees, hypothesis testing and statistical models, and reflections on the uses of probability and statistics in legal and scientific contexts. Discounting philosophy, Bayesian dogmatism makes little difference to these topics except for some terminology (obeying the calculus of probabilities is "coherence") and the introduction of exchangeability ("most people would consider the series of tosses of a [thumbtack] to be exchangeable") and de Finetti's theorem.

The distinctive stylistic feature is a professorial lecture style; precise tending toward pedantic, and discursive tending toward convoluted digression. An illustrative sentence: "Some physicists can be contemptuous of attempts by sociologists to be scientific, but often the contempt is unjustified because of a failure to recognize the difficulties of experimentation in the latter field compared with the precision attainable in their own, often at considerable expense."

For a reader already exposed in typical textbook fashion to the basics, and wishing to solidify their own conceptual view of these foundational issues, this book might be very helpful, simply because its discursive nature draws attention to many aspects not discussed in typical textbooks. On the other hand the author's stated goal is ".... to explain in terms that motivated, lay persons can understand, some of the discoveries about uncertainty made in academe, and why they are of importance and value to them, so that they might use the results in their lives". I find it hard to imagine that many of the "politicians, journalists, lawyers and managers" cited as lay persons would persevere through the book. In my experience such people would be much more receptive to case studies than to balls in urns. They may tolerate one hypothetical case to illustrate a conceptual point, but if you cannot describe an example of real substance they will not take you seriously.

For a concise, thorough, unbiased treatment of similar foundational issues, albeit in more textbook style, see Hacking's book mentioned above. For casual browsing of numerous real-world topics, to see how statisticians think about the world, see the Understanding Uncertainty web site (no relation!) or Senn's book Dicing with Death: Chance, Risk and Health. Or just read a good introductory text that focusses on real-world examples, such as Statistics, 4th Edition.

This review compares this outstanding but expensive new entry to the author's previous text, the second edition of making decisions: Making Decisions, 2nd Edition. You might wonder what difference it makes since the Decisions book is still nearly $100 and this new volume is over $80. The difference is that Making Decisions is a 20+ year old, and can be snagged in good shape for $10 bucks on occasion-- a significant savings.

The first question would be-- what has changed in the past couple decades that would make an investment in UU instead of MD worthwhile? The obvious answer is computing power-- many of the earlier Bayesian models are now more complex and robust. But most folks looking at decision models according to the author's target audience will be wanting an advanced intro to decision theory, not having access to Crays wired in series anyway. The theories themselves have changed very little conceptually or mathematically, although the modeling has certainly become more robust and complex.

So, before you invest in one of the other, you've got to ask yourself about your own goals and utility functions. The new UU explores a greater, less focused variety of "balls in urns" (a decision example started in MD on how we cope with uncertainty) but follows the same basic Bayesian process-- weight the decisions using probability, compare those to your utility function (the risks vs. rewards for you personally), then decide more "quantitatively."

This approach generally fits in the big rubric of thinking and deciding more clearly-- avoiding cognitive and emotional biases, and using crisp, more quantitiative inner decision tools. The nice thing about both books is that the author uses brilliant prose and conceptual language rather than math equations to get us to that skill set. The author is clearly a probability guy, and his Bayesian philosophy shows through, but doesn't stifle the discussion. It does limit "decisions" to a certain focus, and if you want to augment that with a much bigger toolkit, including creativity, analogy, semiotics, recursion, and other topics not covered here, check out: Intuition Pumps And Other Tools for Thinking.

Conclusion: If you are using this book as a "personal" decision text for self study, the previous text has everything this has in that frame. If you are more interested in understanding the whole field of uncertainty from a stats frame, the new book is worth it, as it covers more than the "optimum" ways to look at choices, including more recent research and more academic topics in the field, not all as relevant to personal choices, but certainly important if you're applying the ideas to economics, politics, and additional areas ancillary to how we make our individual choices. For less quantitative but still very fun explorations of how we evaluate things like free will or the latest government crisis, intuition pumps above gives many more practical rules of thumb and "how thinking works" examples than these texts. Both are highly recommended for the right utility functions!

Emailer: YES, I recommend this book if you are taking a more technical course on Bayes. No current math oriented texts give the level of intuitive grasp of this author, and his prose descriptions of the math will help a LOT with the tougher math. Many of these are more thought experiments than real world case examples, but they sink the point in in many ways the formulas and equations alone can't get to due to their level of abstraction. The previous text doesn't cover the range of topics you're encountering in your course-- this does.

I started by reading the prologue, then the epilogue, and then I could not let go of this book any more: The thinking is razor sharp, the beauty of the writing breathtaking. Dennis Lindley has a phenomenal gift to explain complex concepts to a broad audience. The content is relevant to all of us involved and interested in the underpinning of decision-making. If only the second round of this oeuvre (ie the result of another lifetime of thinking), understanding uncertainty in a context of conflict, had already been written, preferably by somebody as brilliant and eloquent as Dennis Lindley.

Dr. Lindley is clearly aware that conceptual clarity should come first and the mathematical expression of that clarity should come second. This is in contrast to many other technical writers, who believe that their mathematical expressions are automatically a substitute for conceptual clarity.

Dr. Lindley's vigorous assertion that uncertainty must be addressed via probability is a bit odd. Dr. Lindley may have intellectual foes against whom he is struggling valiantly, but those of us on the academic and statistical sidelines are left scratching our heads and wondering what the fuss is about. Fortunately, the dogmatic tone that the author adopts on this issue doesn't detract from the overall quality of the exposition.

This book is definitely going in my library.

I read this book in disbelief. The disbelief that it took me so long to find it, the disbelief that someone can think so clearly and express it so clearly and finally disbelief that I am now slowly becoming capable of understanding probability, its uses and laws.

This book walks you slowly thru why probability is used, how you should apply it to daily decisions and how it is applied to the sciences, applied, medical and otherwise. This is simply a terrific book and I wish I could give its author a big bear hug for writing it.