The Wisdom of Crowds: Why the Many Are Smarter Than the Few (Unabridged) [Audio Download]

The central premise of the book is an interesting one - that taking the average of a large number of individual viewpoints is likely to give a better result than taking the view of a single (or small number) of experts.

The paragraph above is, of course, an oversimplification. The book does go into rather more detail about when this works, and when it doesn't.

The approach in the first half of the book overall seemed OK - there were points in the first chapter where I thought that the author had fallen into some basic errors, but reading on made me realise that, no, he had considered them and had responses to the questions that I thought needed answering.

However, there are, to me, two flaws in the book.

Firstly, the second half of the book is too lightweight. He applies his theory to a whole bunch of situations (shorting of stocks, elections and so on), but doesn't present any proof of his ideas. He seems to assume that we're all so convinced in the first half that he can spend the second half saying "now we know this is true, this is what it means."

... which takes me to the second flaw.

Someone rather more intelligent than I - from memory Richard Feynman - wrote a long and interesting article about scientific rigour, and how when you do one expirement that appears to suggest a theory, you don't assume the theory is right because it fitted that one set of data. Instead, you specifically design other experiments that TEST the theory.

The problem with this book is that there appears to have been a single "real world example" which suggested the theory, but that every other example is a "class room experiment by Professor X in which a series of students did Y..."

Sadly, in a book that purports to relate to research going back 50 years, it is hard to believe that there are so few examples of the principle that can be brought to bear.

In having said all that, I have a nagging feeling that the theory is right, and that the evidence is there, but that the book was over-pruned to get down to a certain size. If this is correct, I wish he'd given more material showing why he was right, and less speculation about what would happen if he were.

This book confounded my expectations. I generally dislike books like this but this one is interesting, provocative and stimulating. I do not wholly agree with the central thesis but even in the sections that I disagreed with there was enough interesting material to hold my attention, it was refreshing to need to question one's assumptions and to think about the points being made.

What is the Wisdom of Crowds?

The book defines it rather loosely suggesting that groups make better decisions if certain conditions are met. The conditions are: diversity (to ensure that different information is used to make the decision), independence and (a certain type of) decentralisation (to ensure that no one person is dictating the decision and that people are using their own private information) together with a way of summarising the different opinions into a collective view.

This loose definition allows the book to address a huge range of topics. It does not build a coherent case attempting to support and justify the central thesis. Instead it relies on a more anecdotal approach - examining situations where crowds can be wiser and situations where they fail to be wise - it is a biography of an idea rather than a manifesto.

To provide some structure three different types of problem are identified: cognition problems, co-ordination problems and co-operation problems. However, even within these broad areas large and rather diverse sets of problems are examined. To assist in the analysis a wide range to techniques are utilised including psychology, statistics and game theory.

The book makes great play of the ideas being counterintuitive and surprising; in some of the examples this is true, in others it seems to be seriously stretching the point. For example, the story about finding the lost submarine is interesting and surprising, as is the speed with which the market identified the company at fault for the Space Shuttle disaster. Less surprising are the examples which boil down to applied game theory, statistics or the fundamental nature of markets.

The most important (practical) problem with the thesis is that the conditions required for the wisdom of crowds to apply are very difficult to meet. The book recognises this and devotes considerable space to situations where crowds fail to be wise because of this. For me, these are probably the best sections of the book. It is very clear that the wisdom of crowds does not mean management by committee (as committees almost always fail to meet the necessary conditions). It is also very sharp on the culture of the 'expert' and is very clear about the need for dissent and the importance of (intellectual) diversity. The section on NASA is chilling and excellent.

In spite of the issues this is still a fascinating book. There is a huge range of stories and examples about how the wisdom of crowds can work and how it can fail spectacularly. I found it a thoroughly engaging and interesting book.

James Surowiecki writes the "Financial Page" column in The New Yorker, where he's consistently able to come up with unusual takes on seemingly familiar topics, and he has a great knack for making business stories compelling and entertaining as well as understandable. But the column is only a page long, and I always wondered how Surowiecki would do if he was able to develop his ideas and arguments more fully. Luckily, "The Wisdom of Crowds" lives up to all my expectations. It's wonderfully readable, full of terrific stories, funny, and its basic argument -- that groups, under certain conditions, can make better decisions than even the smartest individuals -- is counterintuitive without being willfully contrarian.

The roots of the argument obviously stem from the way markets work -- buyers and sellers find each other and reach efficient outcomes without anyone being in charge, while the stock market (at least some of the time) does as good a job as possible of setting prices. But what I really like is the way Surowiecki extends this argument way beyond business and markets, showing how collective wisdom can be seen (and can potentially be used) in a host of other situations, including the racetrack, on the Internet, and on city streets. He also does a good job of drawing out the possible implications of this for everything from the U.S. intelligence community to the way companies are run.

This is definitely a big-idea book, but the author is cautious in laying out his evidence, and is careful to show that groups, even if they're potentially wise, are often stupid and dangerous. The chapter on small groups in particular, which focuses on NASA's mismanagement of the Columbia mission, is powerful stuff, and useful to anyone interested in how to run a meeting well (or badly, for that matter). The least satisfying part of the book is the chapter on democracy, where Surowiecki shies away from pushing his conclusion to its logical end. But on the whole, this is just a wonderful book, elegant and enlightening.

If you're interested in this book, it's also worth checking out Paul Seabright's "The Company of Strangers" and Robert Wright's "Nonzero."