The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty [Hardcover]

"The Flaw of Averages describes a set of common avoidable mistakes in assessing risk in the face of uncertainty. It helps explain why conventional methods of gauging the future are so wrong so often, and is an accessory to the recent economic catastrophe. Once grasped, these ideas can lead us to more effective forecasting and decision making."

Those are the claims of the opening words of Sam Savage's preface, and he sets out to do just that in excellent style. The book is part textbook, part manifesto and part biography, introducing the characters Sam has met along the way and who have contributed to his insight. The textbook is unlike any I have read before. It is dealing with a highly mathematical subject, but in a very accessible and non-mathematical way. Concepts are explained in plain English--I think I will borrow some of the wording for my own consulting practice--with lots of real-life applications and a scattering of brilliant cartoons by Jeff Danziger. The manifesto is a programme for introducing probability management as the key part of any strategic decision-making process.

Savage introduces the concepts of probability and statistics with five "mindles" (easy for the mind to grasp, as a handle is easy for the hand to grasp), in the process exposing a number of "red words" (the first is "utility theory"), technical terms that professional mathematicians and statisticians have used for too long to bamboozle the public. The red words are converted to everyday words that we are already familiar with (eg, "risk attitude").

We are introduced to uncertainty versus risk, where two people might look at the same uncertainty but, with different risk attitudes, might perceive very different risks. An uncertain number is a not a single number, but a shape (a "distribution") with a range of possibilities. Adding together uncertain numbers leads to distributions that are taller and thinner, the effect of diversification (the weak Flaw of Averages). If you start doing calculations with uncertain numbers, be aware that the average output is almost certainly not the calculation done with average inputs (the strong Flaw of Averages). The average (or expected) profit is very likely less than the profit associated with average demand. The average (or expected) duration of a complex project is likely to be more than that implied by the average duration of the component tasks. Finally, and this is where the world economy came unstuck, uncertainties are interrelated.

Savage identifies the seven deadly sins of averaging (actually he identifies eleven, with the twelfth being "Believing there are only eleven deadly sins"). He then explores how uncertainty impacts all areas of life with a wide range of real, illuminating and entertaining examples. Several are drawn from the finance world, both personal and corporate, but he looks also at national security, climate change, healthcare and the difference between the sexes (it's all down to females having a diversified portfolio of two X chromosomes, whereas men have only one).

The solution to the Flaw of Averages is to embrace uncertainty and take advantage of the cheap and ubiquitous computing power that surrounds us. Monte Carlo simulation can allow us to play with thousands of scenarios and quickly and easily see how the decisions we take can affect the distribution of the outcomes we care about.

Savage proposes a role for probability management (and a Chief Probability Officer) to be in charge of developing probabilistic forecasts and ensuring that the same forecasts are used by all who need them. This involves creating stochastic information packets (SIPs) managing a central database of them, a scenario library with relationships preserved (SLURP). If all business units drew their forecasting assumptions from the database, and there are technological innovations that make doing this easy in Microsoft Excel, for instance, organisations could consistently manage the uncertainty face.

The book closes with the comment, "While every organisation faces unknown unknowns, there are also risks somewhere in those organisations that are known, but which are nevertheless not managed. It is these risks, perceived but not managed, that cause most of the destruction." Amen to that.

Corny acronyms aside, this is a great book that needs to be read by every leader making decisions in the face of uncertainty, and by everyone supporting the leader with forecasts and analysis. And every firm needs a Chief Probability Officer.

This was recommended reading as part of my MBA course alongside Sam Savage's other book Decision Making with Insight. Whilst there are some really important concepts in this book, the definitions are poor and the concepts badly explained. This book in particular focuses on trying to be funny at the expense of explaining properly concepts such as the Flaw of Averages. The examples are pretty good, but the book is flooded with them at the expense of getting to the point around the concept they are trying to explain. In short this book could be much more concise and deliver a much more powerful message without losing the 'light hearted' intention of the author.

I came across Sam Savage a few years ago when looking for software to conduct Monte Carlo analyses. Sam's view on statistics is refreshing and anyone who makes business decisions (or even wants to better understand decision making in private life) will find this book well worth reading.