Junk Science Judo: Self-defence Against Health Scares and Scams [Hardcover]

Junk Science Judo. Self-Defense Against Health Scares and Scams by Steven J. Milloy, Cato Institute, Washington, DC, USA, 2001, ix-xii + 218 pp.

A superb manual for understanding health claims and detecting fraud, "Junk Science Judo" is written in a punchy, easy-to-read style that allows the mathematically challenged, like myself, to interpret the usual bell-curve statistics that are used for environmental health threats and diet threats. One may also use this approach to judge the benefits of prescription drugs or alternative treatments. He shows how to look for p values of 0.05 or less, and he shows how to use the 95% confidence intervals (CI) to see whether the low or high limit crosses the reference value of 1.00. If it does, Milloy suggests disregarding any result that is claimed.

In epidemiological or ecologic studies, Milloy suggests discarding those results in which the relative risk (RR) is between 0.5 and 2.00, whichever applies. Milloy goes far beyond the usual cautions that an association is not necessarily a cause. He is contemptuous of, but not totally dismissive of epidemiology. For Milloy, the descending pecking order of research on health hazards is: clinical trials, cohort studies, case-control studies, and ecologic studies. He describes publication bias, confounding, and "tainted experts". He reminds us that mice are not little people, and that there is a safe dose of everything.

Milloy encourages intelligent contact with the sources of health advice to support or refute the recommendations, and he recommends a number of sources of supposedly reliable health information. His criticism of medical journals, which includes the defects of peer-review, is the most detailed I have seen in print, and very well-taken, in my opinion. He skewers broadcast media on their emphasis on ratings rather than accuracy or balance. He deplores the motivation of many activists. Many specific examples are given, from Alar to radiation.

This book is extremely valuable for helping anyone who is not a medical or epidemiological specialist to judge the value of health or health-threat claims.

Now the bad news. Milloy does not question the use 1-tailed statistics, or the failure of many relationships to follow a bell-curve (Kauffman, 2001). Milloy failed to warn of the common deception of reporting an effect of something on a certain health condition without the inclusion of total death rates. Milloy's fine example of prostate cancer "non-prevention" by eating foods high in selenium (p164) may mislead people into ignoring the benefits of selenium supplementation, which has been shown in a clinical trial to lower the RR of all cancers to 0.83 (95% CI 0.47-0.85), and is most effective against lung, prostate, and colorectal cancers (Clark et al., 1996). Milloy downplays the dangers of trans fats (Oomen et al., 2001), but his own example shows that the highest two quintiles of consumption are not healthful (p165). Milloy made the mistake of writing that the absence of a biological explanation for the claimed effects of electric and magnetic fields means that they have no effects (p76); readers of JSE know that such judgments based on lack of knowledge are faulty. ....
Joel M. Kauffman
...

References

Clark, L. C. et al. (17), (1996). Effects of Selenium Supplementation for Cancer Prevention in Patients with Carcinoma of the Skin. A Randomized Controlled Trial. Journal of the American Medical Society, 276, 1957-1963.

Kauffman, J. M. (2001). Article of Interest. Journal of Scientific Exploration, 15(4), 575-576.

Oomen, C. M., Ocké, M. C., Feskens, E. J. M., van Erp-Baart, M.-A. J., Kok, F. J., & Kromhout, D. (2001). Association between trans fatty acid intake and 10-year risk of coronary heart disease in the Zutphen Elderly Study: a prospective population-based study. The Lancet, 357, 746-751.

Junk Science Judo is filled with examples of questionable conclusions drawn from data from a wide variety of sources. Milloy relates how zealous lawyers can draw false or misleading inferences from scientific data in order to extract large rewards from lawsuits against defendants with deep pockets, often with collaboration from those in the "scientific" community who stand to profit from their "research" (what better way to justify large government grants to continue research into some promising area?). Most telling are the cases of the lawsuits involving breast implants and Agent Orange, where a total absence of evidence was ignored by courts and juries eager to punish unpopular defendants and to reward those who were definitely disadvantaged (but probably not as a result of either the implants or Agent Orange).

Despite a generally good job of writing a primer on how not to be taken in by faulty conclusions or misleading inferences of cause and effect, Chapter 11 of this book contains one significantly dubious conclusion of the author himself. In a discussion of the statistical technique of meta-analysis (a technique where the results of numerous small studies are combined to simulate one large study), Milloy repeatedly takes stabs at this respected technique. His criticisms appear to be based mainly on questions raised in an editorial in a medical journal where the editor cited poor data quality and publication bias (i.e., only research studies showing significant results tend to get published) as reasons for questioning conclusions from meta-analysis. Although the editor is correct, it does not follow that all meta-analysis is therefore useless. Given enough well-executed studies without bias in the findings, meta-analysis can be a very useful and enlightening tool.

I highly recommend that all consumers of information either purchase this book or at least borrow it from someone, and read it thoroughly. I suspect that you will never again be able to read uncritically any account of some purported new crisis brought on by zealous lawyers, journalists or politicians. When some "scientific" conclusion appears to be counterintuitive, it is most likely incorrect. Subsequent research may, in fact, show that the conclusions were incorrect, but (since this is not exciting) the correction is likely not to be brought to the public's attention. Since it is impossible to prove a negative, the burden of proof should be on the scientific community to exercise caution in releasing insignificant findings to the press. Junk science may sell newspapers or gain an audience for the evening news, but it is devastating when it causes major disruptions in the lives of people who either aren't given the whole picture or who can't interpret the information when it is presented.

The best defense against junk science is an informed public. Read this book and you will definitely become better informed!

Bear in mind that this book is probably not for the trained scientist. The author takes time to explain basic research concepts for the casual reader, and also interjects his own attitude to make the book more fun to read. However, for those readers who do want to check up on the author's science, the 17-page listing of 314 cited studies should leave no question that the author beckons the audience to follow his own advice by examining his research.

If I had one complaint about this book, it would be that the author often relies too heavily on quoted material. I think the author is trying to show that he was not skewing the cited studies, as would the individuals he criticizes in the book. While effective in this regard, the practice sometimes results in a monotonous pattern. The author's comments on the quoted studies become predictable midway through the book as the reader learns to identify the glaring problems in cited research methods.

In the end, this book was an excellent read for the choir to which it preaches. I would recommend it as an excellent resource for anybody who wants ammunition to fight junk science, or to back up political debate with embarrassing proof of the ineptitude on which many health/environmental policies are based. Anybody who cares about ridding public policy of special interest politics should read through this book once and keep it handy for later use as a reference tool.