Proofiness: The Dark Arts of Mathematical Deception [Hardcover]

One of the benefits of retiring from my career as a statistician is that I no longer feel it's my personal responsibility to alert friends and colleagues to the myriad ways they are being misled or deceived by the kind of abominably poor summarization of data that's pretty much the norm these days. It's just as well - who wants to be *that guy*, the crank at the table who people start to inch away from surreptitiously, avoiding eye contact all the while?

Not that I endorse misleading or deceptive data presentation - far from it. Now more than ever, as we all struggle to make sense of the avalanche of information that constantly assails us, the capacity for critical, intelligent interpretation is vital. So it's important to be able to see through the most prevalent fallacies in data interpretation, not to mention data presentation strategies deliberately intended to mislead. This latest book by Charles Seife has the laudable goal of educating the reader about some of the most common types of statistical malpractice out there, continuing a tradition established by such authors as Darrell Huff ("How to Lie With Statistics"), John Paulos ("Innumeracy"), Edward Tufte, or the authors of last year's highly successful "The Numbers Game" (Michael Blastland and Andrew Dilnot).

Unfortunately, though "Proofiness" is a well-intentioned book, it suffers from a fundamental crisis of identity. There is a major gap between what "Proofiness" promises and what Seife actually delivers. The first hundred pages cover roughly what one might expect: graphical deception by use of misleading labels or scales, comparison of apples and oranges (e.g. dollar amounts unadjusted for inflation, absence of an appropriate control group, regression to the mean), cherry-picking of data, the tendency to interpret mere random variation as systematic, nonsensical conclusions obtained by extrapolating beyond the range of observed data, overstatement of the precision of measurements, the way in which humans are hard-wired to misinterpret risk and deal poorly with calculations involving risk. Seife's exposition of these topics is lively and clear (with the major caveat discussed below). About halfway through the chapter on risk, however, he makes a major detour. His discussion of the malfeasance of those involved in the Enron debacle, the Bernie Madoff pyramid scheme, the failures at AIG, Citigroup and other institutions, and the subsequent bailout efforts has almost nothing to do with statistical trickery, focusing instead on the public policy and regulatory issues raised by the financial meltdown.

The next chapter, "Poll Cats" does return to the issues involved in conducting accurate sample surveys and presenting the data appropriately, with a reasonably clear discussion of systematic error versus random error. However, the following two chapters, "Electile Dysfunction" and "An Unfair Vote", taking up some 80 pages, really have little to do with data-related issues. Instead they provide a review of events surrounding the Florida vote count in the 2000 presidential election, the six-month circus that took place before Al Franken was eventually declared winner in the 2008 Minnesota Senate race, and a review of historical and present-day gerrymandering efforts whenever congressional redistricting comes up for discussion. Not that Seifen's review of the relevant events, and the issues they raise, is not interesting - but it is largely editorial comment on political events and, as such, it seems to belong in a different book, as does the appendix in which he discusses electronic voting. In making this criticism, I take the view that fraud, malfeasance and corruption stemming from poor public policy, faulty regulatory mechanisms, or inadequate enforcement of existing protections, really are subjects for a different kind of book than that initially described by Seifen. Though the author does return to his initial remit in the final two chapters (discussing abuse of probability and statistical arguments within the judicial system, and for propaganda purposes), overall the book does not make a coherent whole.

Then there's the caveat mentioned above, regarding Seife's exposition methods, which turns out to be a serious one, enough to prevent me from giving this book my endorsement, despite its good intentions. It's the author's predilection for coining cutesy neologisms that not only add nothing to the discussion, but actually end up seriously muddying the exposition. It's evident right there in the book's faux-cute title, "Proofiness". I wish I could say that the author offers a rigorous definition of exactly what he means by this invented term, but he doesn't. It remains unhelpfully vague throughout the book. Sadly, it's not the only example of authorial neologism run amok. "Disestimation", "Potemkin numbers", "randumbness", "regression to the moon", and the horrendous coinage "causuistry"; each of these is a neologism that adds nothing to the discussion. Many of them lack a clear definition, or when a definition is offered, the term just seems to muddy the waters. For instance, Seife uses "disestimation" to mean "overstatement of the precision of a number or measurement", indicating an error based in randomness. But the 'dis'-prefix clearly suggests a systematic error, as does the parallelism with "misestimation", a term which statisticians routinely use to indicate a systematic error. And while one applauds the author's efforts to educate his readership about the error of mistaking correlation for causation, the term "causuistry" is simply an abomination. I'm not sure where this recent trend for authors to invent their own faux-cutesy terminology, where none is needed, originates (possibly Malcolm Gladwell bears some of the responsibility), but it needs to stop.

Though I am sympathetic to the author's stated aims, his execution was such that I cannot endorse this book. Anyone interested in this important topic would be far better served by reading The Numbers Game: The Commonsense Guide to Understanding Numbers in the News, in Politics, and inLife by Michael Blastland and Andrew Dilnot.

In Proofiness, science journalist and NYU journalism professor Charles Seife decries the tactic of using numbers to lie. Not just using numbers to bolster one's argument. But, in his words, to use fake numbers to prove falsehoods. To use bogus mathematical arguments to prove something that we know in our heart is true - even when it's not.

Seife does not just condemn proofiness as a mistake in logic. He thinks that numbers have a mystical power. That phony numbers have the appearance of absolute truth, of pure objective fact. So we can, and do, wrongly use them to prejudice people.

Proofiness, Seife believes, is the raw material that arms partisans to fight off the assault of knowledge. To clothe irrationality in the garb of the rational and the scientific. So, he says, proofiness is a dark art of deception.

That makes, Seife believes, proofiness one of the biggest problems we face. He says our society is awash in proofiness. Using a few powerful techniques, thousands of people are crafting mathematical falsehoods to get us to swallow untruths. In fact, proofiness is destroying our democracy by deception.

Seife makes some good arguments. And Proofiness is well-written and provokes thought. But does he show that proofiness is a danger to democracy? That proofiness is at the root of many of the problems we face today? In my opinion, not hardly. On this point, Proofiness needs a little more proof.

Take the example that Seife uses to lead off the book. In February 1950, Senator Joseph McCarthy said in a speech in Wheeling, West Virginia that: "I have here in my hand a list of 205 . . . a list of names that were made known to the Secretary of State as being members of the Communist Party and who nevertheless are still working and shaping policy in the State Department."

Seife claims that McCarthy's use of the number 205 was a jolt of electricity that shocked Washington into action against communist infiltrators. He says that the very fact that McCarthy attached a number to his accusations imbued them with an aura of truth. The numbers gave McCarthy's accusations heft; they were too substantial, too specific, to ignore.

Really? So if McCarthy had left out the number 205 (which it appears from the quote that he almost did), and just said he had in his hand a list of names, then McCarthy's claims would not have had the attention they got?

I don't think so. Number or not, McCarthy's rhetorical device was powerful - "I have here in my hand a list of names." And that speech was just one part of the complex historical picture of McCarthyism. McCarthy did not need to use the dark art of proofiness to do what he did.

While Seife's focus on dark arts and deception seems overblown, Proofiness did make me think about how to weigh purported proof of complex issues. He gives some examples of how people use numbers to deceive:

-- Falsifying numbers (This is what Seife claims Joseph McCarthy did. McCarthy said he had 205 names. Then later it was 57 names. Then 81. Seife claims that McCarthy had no names. Not a single one.)

-- Comparing apples with oranges

-- Cherry-picking data

-- Apple polishing (Giving technically correct, but deliberately misleading, numbers.)

-- Potemkin numbers (These are phony statistics based on wrong or nonexistent calculations.)

-- Disestimation (Giving too much meaning to a measurement, and not qualifying it enough.)

Seife's analysis is clever, and his examples well chosen. Still, I'm not sure that he breaks much new ground here. After all, many have long warned us to watch out when someone cites numbers to prove a point. Even Homer Simpson knows that "people can come up with statistics to prove anything." And the proverb "lies, damn lies, and statistics" has been around for at least a century.

So while I enjoyed Proofiness, I would have liked Seife to plow more new ground on some issues that he only touches on. For example:

-- What do you do when things by their nature cannot really be proven? Do humans cause the earth's climate to change? Did the $787 billion stimulus help? How can you prove that you are correct on these critical issues, no matter which answer you choose? If you cannot prove you are correct, what do you do? Nothing? Or should you rely on what Stephen Colbert derided as "truthiness" (the inspiration for Seife's title Proofiness): "the truth that comes from the gut, not books."

-- Seife focuses on how others deceive us with numbers. But human beings are notoriously susceptible to self-deception. How can we avoid the trap Paul Simon warns us about in song: "All lies and jest. Still, a man hears what he wants to hear and disregards the rest."?

-- We humans find it hard to connect cause with effect. We see relationships that are not there. There's nothing deceptive about this. It's just human nature. Even the smartest among us fall prey to this, as seen by two-time Nobel prizewinner Linus Pauling and his strong but apparently mistaken beliefs about vitamin C. To avoid this problem, should we abandon faith, ignore our guts and only believe things that have been proven?

In short, Proofiness is a book worth reading. Agree with him or disagree, Seife will make you think, and that makes the book an important one. But Proofiness could have been better.

I think I was expecting something more in the spirit of "How to Lie with Statistics", the small 1954 book by Darrell Huff. In other words, I was hoping to find some techniques to improve critical thinking and show how mathematics can be used both to deceive or to uncover fraud. Generally the book focuses on the former. In the process, it glosses over the concept of margins of error in polls, without explaining standard deviations, confidence intervals, or how the margin of error might depend on the results of the poll.

The invention of cute new terms like Potemkin numbers, disestimation, and causuistry was rather awkward. The confusion of casuistry and causuistry was rather perplexing. It would have been more appropriate to discuss Granger Causality tests for example. Perhaps some discussion of techniques for improving polling results for sensitive questions like those presented in Daniel Corstange's article, "Sensitive Questions, Truthful Answers", which recently won the Warren Miller Prize awarded by one of the top Political Science journals, would have been useful.
I often felt as though I was being subjected to a passionate speech to the crowd, urging us to "nuke" all the numbers. Emotional appeals are at least as suspect as those adorned with numbers.
While I thoroughly enjoyed the author's book "Zero: The Biography of a Dangerous Idea", "Proofiness" left much to be desired.