The Drunkard's Walk: How Randomness Rules Our Lives [Hardcover]

I bought this on a whim based on an Amazon-recommendation, and I wasn't disappointed. Mlodinow gives a very apporachable introduction to the science of randomness, and pitches the subject matter at the right level.

Unlike other books that have dated badly, the examples that the author uses to illustrate his points are recent and relevant, and communicated with a light-hearted sense of humour that turns a dry subject matter into something surprisingly engaging. I got through it quite happily in a couple of sittings.

It loses a star as occasionally Mlodinow does go into a little too much detail on the history of probability and the personal lives of the mathematicians on whose theories this book is based. I didn't think this was necessary (or particularly interesting) and would have liked to have seen this space filled instead with wider content. Nevertheless, the vast majority of the book is abosrbing and so is well worth picking up.

I was surprised to learn that the Greeks did not have a theory of probability. Their belief "that the future unfolded according to the will of the gods" and their taste for "absolute truth" did not encourage the study of chance. Where pristine philosophy failed, the more grubby pastime of gambling succeeded in motivating probability theory. And, in true statistical style, it only took a handful of gamblers out of a large enough sample to get things going.

Today we might as well be Greeks for all that we understand or even recognize uncertainty. Even if we do not share the view that everything happens for a reason, it is still easy to ignore the role chance plays in our lives. We humans, with our big brains and clever language and propensity for story telling, are well equipped for this kind of failure. When it comes to recognizing randomness, we can be "outperformed by a rat". If this fact piques your curiosity or lowers your self-esteem, read on, and this superb book should satisfy one and restore the other. It is anything but a drunkard's walk through an intellectual maze. Mathematics, the social sciences, psychology, economics, brain studies, all contribute to the modern understanding of this fascinating area. By the end, several important ideas should have become straightened out into the intellectual equivalent of broad, tree-lined avenues, and you might agree with a quotation from Max Born: "Chance is a more fundamental conception than causality."

First off, do not panic. Even a Harvard professor specializing in probability and statistics admits we're not cut out for this kind of thinking - which makes Mlodinow's achievement in writing an entertaining book from which you can actually learn something all the more remarkable. For example, I've come across the Monty Hall problem before, and thought I'd understood it, sort of, although it was like having to read a novel by following the words with my finger. This time, it was easier, partly to do with the way in which Mlodinow introduces the concept of the sample space and breaks down the problem into manageable pieces, and partly because his style is so engaging. It helps that he writes in the first person, and is neither afraid to draw on personal experience nor cringe making when he does so.

One major theme is the "fundamental clash between our need to feel we are in control and our ability to recognize randomness." Research by scientists like Kahneman and Tversky shows how deep-rooted this is. Most of us have been duped by optical illusions, but while these "seldom have much relevance in our everyday world" cognitive biases or systematic errors, on the other hand, "play an important role in human decision making." For example, confirmation bias occurs when we attempt to prove our ideas correct instead of searching for ways to prove them wrong, and "it presents a major impediment to our ability to break free from the misinterpretation of randomness."

Abstract notions are never allowed to wander far before being pinned down by concrete illustrations, often taken from remarkably current affairs. There are two graphs - proper sciency pictures with numbers and axes and everything - which are striking in their portrayal of a startling truth: they show the performance of fund managers over two five-year periods, and while one is a nice orderly ranking from good to bad, the other looks "like random noise". You could have no better illustration of the small print that past performance is no guide to future returns - so why do we pay huge fees to these so-called experts to manage our money, when a large chunk of their "performance" is down to luck? It is salutary to learn that even Wall Street superstars cannot consistently beat the average market return. "People systematically fail to see the role of chance in the success of ventures": the CEO of Merrill Lynch could one year "be celebrated as the risk-taking genius responsible" for the company's success and then, "after the credit market collapsed, derided as the risk-taking cowboy responsible" for its failure. These are important lessons to learn, especially now that even red-blooded capitalists are beginning to question the stratospheric pay packets of financiers.

We need to move beyond "the deterministic view of the marketplace" in which "it is mainly the intrinsic qualities of the person or the product that governs success." The "nondeterministic view" - not confined to the stock market - holds that "there are many high-quality but unknown books, singers, actors, and what makes one or another come to stand out is largely a conspiracy of random and minor factors - that is, luck. In this view the traditional executives are just spinning their wheels." Such a wholesale change in our thinking seems too much to hope for, given how much "we rely on gut instinct" in everyday life and how tempting it is to see purpose where there is none, to "pay lip service to the concept of chance" but to "behave as though chance events are subject to control."

Uncertainty is a modern sin that dare not speak its name. There are always pundits on hand to explain the past and prophesy the future, to nurture some of society's "shared illusions". If you want to "learn to view both explanations and prophecies with skepticism" then the "Drunkard's Walk" is an excellent introduction.

If you wonder about the relevance of the statistics and 'proof' that inundate our lives then this book puts them in their place. With a nice bit of history about probability up to the present and how outcomes are not as predictable as we would like to think. It certainly added to my own feeling that with our politicalisation of 'statistical proof' in society that the wool maybe being pulled over our eyes. Everyone aught to know about probability and randomness. And this is a good place to start - an enjoyable read.