Making Sense of Clinical Trials: A Guide for BioHorizon Clients

We have all been there. A long-awaited announcement concerning a recently completed clinical trial has just been released. Patients, physicians, investors, executives–anyone with a vested interest in the success or failure of a new drug or medical device–scour a company’s press release, trying to make sense of it. Does the treatment work?  Does the drug make a difference?  Will the device ever be approved? 

The questions may be obvious, but the answers–frustratingly–are often not.

When trying to interpret the seemingly elusive language and technical complexities of a clinical trial report, it is all too common for the experience to end in confusion and decision paralysis. Even worse is the potential for important decisions to be made based on a flawed analysis.

It doesn’t have to be this way. You just need the right information, and the right approach–and this is exactly what our team of expert clinical trial analysts has provided for you below. We’ve demystified the process of interpreting clinical trial results for you in an easy-to-follow, 5-step guide. Using our approach, you will improve your analytical skills, and have the clarity and confidence to make better decisions. So, let’s get started!

How to Read and Interpret Clinical Trial Results

Whether the clinical trial results are coming from a company press release, a conference abstract, or a peer-reviewed medical journal, following these 5 steps every time will improve your analysis and interpretation of trial results.

1.Determine the Trial Type: Preclinical or Clinical (Phase 1, 2 or 3)?

Is this a preclinical trial where the intervention is tested in a lab using an animal model, or a clinical trial where the intervention is tested using human participants?  Preclinical trials are important in determing whether a drug is safe and potentially effective enough to enter human trials. Preclinical studies are only the first step in a lengthy process that investigators must complete to move towards approval.

If this is a clinical trial, is it a  phase 1, phase 2 or phase 3 clinical trial?  Each stage of development is designed to show the investigators and the regulator something different.

Phase 1 clinical trials use a very small number of participants and represent the first use of the drug in humans.  Participants may be healthy volunteers or patients with an incurable condition. The focus of phase 1 trials is demonstration of safety, determination of dose range that can be safely administered, and to gain an understanding of how the new drug is metabolized in humans. Phase 1 trials do not tell us about the effectiveness of the new drug.

Phase 2 clinical trials involve a somewhat larger number of patients with the condition the drug is targeting for the purposes of determining the relative efficacy of different doses and frequencies of administration as well as providing more safety data. If the drug appears to work and has acceptable toxicity, investigators will begin the much larger and more expensive phase 3 clinical trial.

Phase 3 clinical trials (sometimes referred to as ‘pivotal clinical trials’) are designed to compare the new drug with an existing drug known to be effective.  They involve hundreds if not thousands of patients randomly divided into ‘treatment’ (patients get the new drug) or ‘control’ (patients get an existing effective drug).  Most phase 3 trials are randomized and controlled, hence the term Randomized Controlled Trials or RCTS.  In most cases new drugs must be shown to be as safe and effective (if not more so than existing therapies) in a phase 3 clinical trial before approval is obtained.

2. If  the clinical trial is phase 3, make sure it is a randomized, double blinded, controlled trial.

Most phase 1 and 2 trials are not RCTS.  Randomization (random assignment of trial participants to treatment or control), double blinding (investigators and patients don’t know whether they got treatment or not), and controlled (one group gets treatment as is compared to a second group which did not) are the foundations on which the RCT is built. They are critical for the demonstration of a drug’s effectiveness and the prevention or mitigation of  the clinical trials worst enemy–bias. 

Think of bias as a tendency of a measurement to deviate in one direction from the true value; when it is allowed to run rampant through a clinical trial, it can obscure true beneficial effects of a drug, or worse–hide harmful ones. There may be some very limited medical interventions where double blinding is not possible (surgical interventions, for example). However, checking a trial for randomization, blinding, and a control group is quick, easy andvery important. It should be one of the first things you do in your analysis in order to ensure that the investigators have taken all possible steps to minimize bias and to give you the highest degree of confidence in the trial’s findings.

3. Describe the primary endpoint or outcome of the trial. 

The primary endpoint is the main outcome of interest that the investigators were trying to measure.  It’s the trial’s “ reason for being”, so to speak.  It might be survival in a cancer trial, blood pressure change in a anti-hypertensive drug trial, blood glucose in an oral hypoglycemic trial etc. etc. If you can’t determine the primary endpoint from your information source, you need to dig deeper–most likely you’ll need to find an additional information source (for example, the company website, clinicaltrials.gov, etc.)

Ever wonder why a positive sounding news release is met with an unexpected negative reaction (by investors, for example)?  It’s often because the primary endpoint was not met; that is, the treatment effect that the investigators were hoping to measure was not found.

4. Determine if statistical significance was achieved for the primary endpoint.

If statistical significance* was achieved for the primary endpoint, it’s usually stated explicitly by the sponsor if it’s a press release, or included in the results section of an abstract or peer reviewed article.  It’s pretty safe to say that if it is not stated explicitly, it didn’t happen! 

Be careful of language like ‘results in the treatment group trended towards significance’, or ‘the treatment group improved’ with no reference to statistical testing.  Regulators and physicians like treatments that produce statistically significant improvements in primary endpoints, enough said.

*Statistical significance: the probability that an observed outcome of an experiment or trial is not due to chance alone.

5. Look for evidence of poor trial quality.

Before clinical trials are started, all aspects of the trial must be laid out in painstaking detail in the trial protocol.  The number and type of patient to be include in the trial, the precise treatment they will be given, the frequency and type of medical follow-up to be received by each participant, even the statistical test that will be used to analyse the results are all examples of the details included in a clinical trial protocol.   Once the clinical trial is underway, the protocol should not change. If the protocol changes, be very, very leery of the conclusions reached. Protocol changes will often be difficult to detect so close attention must be paid to company news releases, etc. 

Next, look for a description of participant withdrawals in both the treatment and control groups.  Withdrawals occur in all trials to some degree; however, large numbers of drop-outs in a treatment group with no explanation is always cause for concern and should serve as a flag to proceed with caution!

So, there you have it.  5 steps we would like you to take every time you read a report of a clinical trial:

1. Determine the trial type
2. If a phase 3 trial – make sure the trial is randomized, double blinded and controlled
3. Describe the primary endpoint or outcome
4. Determine if the drug produced a statistically significant improvement in the primary outcome
5. Look for evidence of poor trial quality

Following the steps above will enable you to more accurately interpret clinical trial results. This will allow you  to make better, more informed decisions.