Putting Medical Science Under the Microscope, Part 3

by Elizabeth | Break the Status Quo, For Doctors and Physicians

We put a lot of faith in medications. We expect them to be safe. We expect them to cure illness. We expect them to make us healthier. But the truth is many medications aren’t as safe or as effective as we like to think they are.

The problem is that there is positive bias in the medication development process. In other words, the process of inventing and testing a new medication, from basic research to final FDA approval, is skewed in favor of proving a medication works.

Here are a few examples how this positive bias affects the medications which are produced.

#1. Researchers aren’t required to report the results of every study they do on prospective new medications. And so they don’t. If a trial shows a medication doesn’t work, they don’t disclose the results. As long as a pharmaceutical company produces evidence that a medication does work, it can get approval for the medication, even if they have conducted tests which show the opposite results.

#2. When researchers develop and test new medications, they specifically choose outcomes which are most likely to prove the medication works. In the case of chronic diseases which develop over years, where it will take months or years to see improvement with medication, researchers can choose to monitor for an outcome which can be measured more quickly and easily than waiting years for research to be finalized.

Here’s a completely hypothetical example to demonstrate this phenomenon. Let’s say a pharmaceutical company is testing a medication which, if taken daily, has the potential to prevent death from having a stroke. However, though strokes are devastating and a common cause of death, the risk of any individual person dying from a stroke is relatively low — roughly 4 people out of every 10,000 die from strokes each year in the United States. Which means that tens of thousands of people would need to be tested, potentially for a year or longer, to see if the medication is helpful. A clinical trial of this size is prohibitively expensive and complicated.

Instead the pharmaceutical company can choose a proxy outcome to measure. Let’s say, again hypothetically, that people who have died of strokes have high levels of neurotransmitter X (a completely made-up molecule) in their blood. If the new medication lowers the levels of neurotransmitter X, a change which can be measured more easily, the medication can be approved without specific proof that it lowers risk of dying from a stroke. Of course, advertising for the medication can’t specifically claim that the medication lowers the risk of dying from stroke, but advertising can imply the link between high levels of neurotransmitter X and dying of a stroke as well as suggesting the benefits of lowering neurotransmitter X.

I will be the first to admit that this example is simplistic, but it does demonstrate how researchers can pick and choose how to do experiments in order to get the results they want.

#3. Researchers and pharmaceutical companies don’t report always the adverse events they encounter in a clinical trial. People who have volunteered to be subjects in pharmaceutical research can’t and don’t always complete the studies. Sometimes people drop out for reasons which have nothing to do with the study, but when they drop out because of toxicity or other side effects, the information isn’t consistently reported.

#4. The average number of people who are given a medication in pre-approval studies is 1,500, which is a small number of people when you consider that many medications target conditions affecting millions of people. A cohort size of 1,500 people is sufficient to prove that a medication works, but it isn’t large enough to detect and fully catalog the potential side effects of medications. For example, if one out of every 5,000 people will get a severe reaction to the new medication, researchers are unlikely to detect this reaction if the medication is tested on a fraction of that number. But when the medication is prescribed to one million people, 200 people could have a severe reaction.

#5. Clinical trials are designed to limit the types of people who are treated so that the effects of the new medication can be studied in isolation. In selecting subjects for clinical trials, people with other chronic illnesses or who are on other medications are typically excluded from being in a trial. However, when a new medication is released, it is frequently used in a much broader range of patients, often with much more complicated health situations. How the medication works in a person on other medications or with several diagnoses isn’t discovered during the medication testing process.

#6. Researchers are required to show that the effects of a potential medication are statistically significant, meaning that the results do not occur by chance. The concept of statistical significance is fundamental to scientific research. However, just because a medication meets the criteria of statistical significance doesn’t mean it make a real difference in a person’s health. There are scores of medications which, if you analyze the data, produce very small changes.

Here’s an example: The FDA recently approved a new ointment to treat atopic dermatitis (eczema). In order to receive the approval, the pharmaceutical company performed two trials. In the one of the trials, the new medication was a mere 13% better than inactive placebo and in the other trial it did about half as well, giving just 7% improvement. And in both trials, fewer than 1/3 of the patients got better at all, even with the medication. Statistically, the medication works, but for practical purposes 7% or 13% improvement doesn’t make much difference.

We are bombarded with news about medical breakthroughs and with marketing messages touting the benefits of new medications. There is no doubt that these treatments benefit many people. But there are enough pitfalls and potential loopholes in the process of medication research and approval that we shouldn’t trust any medication — no matter how new or promising it seems — to completely eradicate disease and to be completely safe. We need to be aware of the limitations of the medication approval process and the ways in which data about new medications can be manipulated to show good outcomes.

When it comes to medications, especially the latest, greatest wonder drug, we need to be savvy customers. After all, our health depends on it.