4 questions to help you assess new medical studies

When you hear about a new medical research finding on the news, should you sit up and take notice or turn off the set? Next time you hear or read about a new study, ask the following questions.

There are many different ways to study a scientific question, but the two you hear about most often on the news are observational studies and randomized controlled trials.

• You've already read about observational studies and why the results of a single such study don't prove much. When many of these studies produce similar results, however, their collective findings can influence how doctors advise and treat patients.
• The real "gold standard" of research — as good as research gets — is the randomized controlled trial, or RCT. In a typical RCT, researchers assign people randomly to receive either a treatment or a placebo (the placebo group is the "control" group).
• Changes that occur or don't occur in the treatment group offer powerful evidence about whether a therapy works.
• The results of a single RCT must be reproduced by other scientists before they can be trusted, but RCTs are the most reliable way to test a new medical therapy.
• Keep in mind, though, that RCTs are expensive to conduct and may not be feasible in some cases due to ethical concerns. For example, you will never see scientists test the cardiovascular benefits of alcohol consumption by asking a group of non-drinkers to start sipping martinis every night.

A study showing that a new drug lowers blood glucose in diabetic lab rats is great news — but only for diabetic rats. What's good for a rat doesn't always turn out to benefit humans.

• Even when a drug is tested in humans, scientists know that age, sex, race, income bracket, education level, and other factors all influence the outcome of studies.
• In other words, if you're a 45-year-old woman, the results of a study that included only men over 70 may not apply to you.

Imagine you flipped a coin 10 times, and it came up heads on eight occasions and tails the other two. Would you assume that a coin tossed in the air has an 80 percent chance of landing heads up? Of course not.

• As every schoolchild learns, this result occurred by chance. If you flipped the coin 1,000 times, you'd probably find that it turned up heads roughly half the time.
• The same thing is true in scientific research. A study involving hundreds or thousands of patients is more likely to produce a reliable result than an experiment that includes a dozen subjects.

In the case of many drug studies, it's the pharmaceutical industry. That doesn't mean the study is worthless, but you may want to view results with some skepticism. There is a correlation between who is funding the study and the results you get.

• For instance, a 2007 review found that studies paid for by drug companies produced positive results 84 percent of the time. By comparison, just 62 percent of independently funded studies turned out positive.
• The same research paper reported that studies sponsored by the drug industry were more likely to be small and lack control groups.

New medical studies must always be taken with a grain of salt. Make sure you're asking the right questions to determine what the results really mean.