The Randomized Trial Fantasy: How We Know What We Know

By David Katz, M.D.
I suppose I might be more expert in randomized controlled trials if I had ever had the actual opportunity to fetch a pail of water without one when my foot caught on fire, as I’ve said I would do. I can’t say I’m sorry that hasn’t happened.

I feel qualified to opine on the topic just the same. I have designed, conducted, and published dozens of such trials. I have written two textbooks about them, too, one addressing details of methodology, the other addressing both that, and its application to clinical decisions. I know a thing or two about randomized trials.

So here’s the punch line: I know a thing or two without a need for randomized trials, too.

There is a fantasy taking over the world of nutrition, especially acute in the aftermath of the contentious Dietary Guidelines release, that nobody really knows anything. The arguments are made at times by seemingly expert people, although we often find they are either not the experts they pretend to be, or are badly conflicted. Or, sometimes, both.

One of the shibboleths with which this camp routinely differentiates itself is the contention that all reliable knowledge — in science, at least — results from randomized, controlled trials (RCTs). Implied, if not stated, is that RCTs are not just necessary and better, but presumably, infallible. The argument continues that such trials are glaringly absent in nutrition, and then finishes with the flourish: We therefore know nothing about nutrition. I can only guess how much Big Food loves this sequence.

It is, however, nonsense, from start to finish. We know plenty about the basic care and feeding of Homo sapiens, in part from excellent RCTs, but by no means limited to them.

For starters, RCTs do only a very specific job, although admittedly, they can do it uniquely well. They are designed to answer questions when there is considerable uncertainty about the best or right thing to do. In the absence of such uncertainty, RCTs quickly bog down in ethical problems. We have, for instance, no RCTs of treating gunshot wounds to the chest or abdomen, versus watching them bleed to see what happens. We have no RCTs of actual vs. sham emergency surgery in this circumstance, or comparisons of trauma surgery to Gregorian chants.

Similar reasoning extends well beyond the bounds of the emergency department. We have no RCTs of spraying water on a house fire vs. watching it burn to see which saves more of a family’s possessions. We have no RCTs of spraying water vs. spraying gasoline.

These silly examples aren’t as silly as they seem. They point out two serious flaws in the RCT fantasy: (1) for ethical reasons, you simply cannot always run a RCT, and (2), when you do run one, the answer is only ever as good as the question.

Randomization, technically, is a methodologic defense against something called confounding, which …read more

Source: More Fitness

You may also like...

Leave a Reply