1 of 2 people found the following review helpful
A flight of fancy,
Verified Purchase(What is this?)
This review is from: Tarnished Gold: The Sickness of Evidence-based Medicine (Kindle Edition)
I suppose I have to give this one star as some of the facts are correct. But most of the book displays the authors' misunderstandings of medical science and statistics.
The introduction is peppered with frank errors, for example that very large clinical trials are incapable of identifying the causes of diseases. They are not designed to do that! Then, I came across the term `orthomolecular medicine', which is megadose vitamin therapy by another name. This is the obsession which Linus Pauling developed in his later years, and is seen by most nutritionists as overt quackery. The foreword is written by perhaps the leading quack from the field. He says that evidence based medicine claims to provide certainty about treatments. If he knew the tiniest thing about it he would know that there are no certainties in science or medicine.
Hickey are Roberts are obsessed with the notion that large scale trials do not tell us anything about individual patients. Their illustration is that you can't select clothes for people by averaging their measurements. This is a frankly idiotic interpretation of science. Trials are all about estimating probabilities, which we get from deriving data from populations. Those derived statistics are then used in conjunction with specific information about a patient, to make a clinical decision. Clothes are not selected without taking measurements!
There are no plausible proposals in the book for replacing EBM. The authors make the mistake that analogies are OK for explaining an argument, but they don't necessarily prove the point. They like exposing what they think are fallacies, but fall victim to the false analogy fallacy. However I have now found out what drives their vitriolic attack. They had a spat with the US National Institutes of Health some years ago, about orthomolecular medicine (AKA megadose vitamin therapy, qv), a thoroughly discredited brand of quackery as we know. Hickey and Roberts display all the attributes of the `prophet crying in the wilderness'. This vitamins saga crops up repeatedly, so they are obviously smarting over it. I just came across a reference to Dr Devra Davis and her book The Secret History of the War on Cancer. I did a little research on Dr Davis - yes, she is another conspiracy theorist, who has been raking over the mobile phone and cancer canard.
Hickey and Roberts trot out a litany of EBM abuses, but do not realise that when people fail to follow the EBM model, they do not invalidate it. Time and again we are told that it's wrong `arbitrarily' to discard data. They do not seem to know that selection of studies for meta-analysis and systematic review (they don't appear to know the difference) is very far from arbitrary, but based on quality. As information specialists they ought to know about signal to noise ratio. There is a vast amount of noise in medical data, because humans are complex and we can't possibly know all that is going on inside them. By rejecting poor quality studies we reduce the noise and hence make the signal easier to spot.
The authors` other obsession is with Bayesian statistics. Nothing wring with that, but they describe their `ideal' very small Bayesian study, and do not realise that it is doing no more that a conventional frequentist study would do. In particular, they totally ignore the need for the prior probability in their study, which is a fundamental factor in Bayesian analysis. They seem to be certain that Bayesian methods are anathema to EBM, which is rubbish - they are just a way of analysing data (although you would normally design your study with analysis in mind).
They say that EBM ignores case reports. No, no, no. Case reports and in particular case series are useful indicators of whether there is a phenomenon that should be formally investigated. Where there is doubt however, case reports on their own are of very little value.
There is a long section on their favourite topic, megadose vitamin C. The evidence they cite goes back to Dr Fred Klenner in the 1950s, who claimed to have treated polio successfully. The authors claim that the results were replicated, but all of this evidence appears to be case reports, ie it is anecdotal. There is a lot of bleating about how expensive it would be to run the huge study that would satisfy the EBM crowd, but my question is this. Why has nobody carried out the small cheap Bayesian study that Hickey and Roberts were extolling earlier in the book? There has been no attempt by any of the megadose vitamin proponents to do any randomised comparative study on any scale, however small. The entire argument relies on anecdotal evidence. Hickey and Roberts also argue for focussing on effects that are obvious and don't require huge studies to detect small effects. They say the effects of megadose vitamin C are dramatic. So why no study designed on that basis? Do they protest too much?
The obsessive objection to large scale trials belies what actually happens in drug development. The pathway starts small, very small. After studies in healthy volunteers to get initial assessments of safety, pharmacokinetics etc, studies of typically 100 patients are carried out to test whether the drug actually has a clear effect. This is the sort of `sticks out like a sore thumb' test that Hickey and Roberts recommend. They don't appear to have noticed that it happens already, and has done for decades.
The authors include a huge number of references, but a great many are misquoted and misinterpreted, by my quick reckoning. Many references are to books and speculative review material by others, rather than to rigorous science.
There is a fuller review at [...]
Tracked by 1 customer
Sort: Oldest first | Newest first
Showing 1-2 of 2 posts in this discussion
Initial post: 18 Jan 2014 09:48:52 GMT
Donald Scott says:
So much of this review suggests that your mind was already made up before you read the book? What you write makes sense but is fanciful all the same! You write.....
'The obsessive objection to large scale trials belies what actually happens in drug development. The pathway starts small, very small. After studies in healthy volunteers to get initial assessments of safety, pharmacokinetics etc, studies of typically 100 patients are carried out to test whether the drug actually has a clear effect. This is the sort of `sticks out like a sore thumb' test that Hickey and Roberts recommend. They don't appear to have noticed that it happens already, and has done for decades'.
Looking at some of the huge fines that the pharmaceutical giants have had to pay for misleading doctors and patients by withholding data on medicines undergoing trials suggests that you have a heavy bias towards looking away from the unsavoury practices of (some unquantifiable number) of clinical trials. As a result anyone looking at your review should take it with a pinch of salt...or maybe Gaviscon!
In reply to an earlier post on 18 Jan 2014 12:47:47 GMT
L. Rose says:
You are conflating two issues. True, the pharma industry is being quite rightly castigated for lack of transparency with clinical trial data. But companies have not been fined for that (even though perhaps they should have been). They have been fined in the main for off-label promotion, ie selling drugs for conditions that have not been approved by regulators.
I freely admit that I read the book with a sceptical mind. That's because there is such a vast body of knowledge about EBM that I considered it highly implausible that such knowledge could be overturned. I was curious to know what was driving these authors, and I found it. They are obsessed with megadose vitamin therapy, which has been repeatedly discredited. There is no clinical trial evidence to support this therapy, so Hickey and Roberts have sought to discredit the EBM system. It is a common ploy to attack the process that has undermined one's dearly held beliefs.
That the drug development process is imperfect and riddled with abuses does not invalidate it. Most prescription drugs have mediocre effects, some are useless, and a minority have dramatically positive effects. This week India announced that polio has been eradicated. EBM did this, coupled with the mobilisation of thousands of workers in the field who were committed to the objective. Smallpox is extinct worldwide - similar story.
How is my writing fanciful? It is a careful analysis. I'm happy to declare a professional interest, based on some 40 years of working with the pharma industry. I have seen the good, the bad and the ugly. As an independent consultant I have several times had to warn clients that they are abusing the system (they never took any notice). I even reported one to the drug regulator (who never did anything). But the fact remains that, properly applied, EBM works.
It's interesting that the current calls for clinical data transparency come from EBM supporters and not from the likes of Hickey and Roberts (or from the alternative medicine fraternity either). See http://www.alltrials.net/. This is because these people know that EBM is powerful and valid, and they are aggrieved to see it abused.
Maybe my review here has suffered from brevity. For the full argument go to http://majikthyse.wordpress.com/2013/05/0
‹ Previous 1 Next ›