As of September 28, 2020, there were 2342 COVID-19 treatment trials around the world.
- Trials conducted thus far have many methodological limitations and have largely generated low quality evidence.
- One of the most salient limitations is the small size of trials.
- Large trials are needed to not only generate statistically and clinically meaningful findings, but also capture the nuanced nature of COVID-19.
- Different subpopulations may have different outcomes from the disease and respond differently to treatments, which calls for large trials to identify treatments appropriate for different individuals.
- Large randomized controlled trials that are well-designed, well-coordinated, and leverage a pragmatic and adaptive design can provide necessary evidence of effectiveness of treatment for COVID-19 in a timely manner.
- Our acronym S.I.G.N.A.L identifies key strategies for conducting meaningful clinical trials during a public health crisis.
“The medical research world is responding to the covid-19 pandemic at breathtaking speed. There has been a maelstrom of global research, with mixed consequences. Positives include the greater provision of open access to covid-19 studies, some increased collaboration, expedited governance and ethics approvals of new clinical studies, and wider use of preprints. But many problems have become evident. Before the pandemic, it was estimated that up to 85% of research was wasted because of poor questions, poor study design, inefficiency of regulation and conduct, and non or poor reporting of results. Many of these problems are amplified in covid-19 research, with time pressures and inadequate research infrastructure contributing.” — Glasziou et al., 2020 (1)
The Rush to Find a Cure: Progress Thus Far
After more than six months of the COVID-19 pandemic, there are no signs of the spread of SARS-CoV-2 slowing down with the second wave already hitting many countries. Globally, there have now been more than 1 million deaths and more than 33 million cases and counting – which has taken a tremendous toll on developing and developed countries alike. The healthcare systems and economies have experienced unprecedented demands and are close to being crippled in many places around the world. With this dire reality, the global scientific community has been rushing to find a cure for COVID-19. While the progress up to now has been remarkable, considering the total scope of the research highlights the fallacies of conducting a few years’ worth of work in a few months. The design and the methodological rigour of many of the studies thus far have been compromised, which questions their utility for providing high-quality evidence to support the use of potential treatments. One of the biggest and most salient criticisms has been the lack of well-designed and large randomized controlled trials (RCTs) for COVID-19 treatments. Not only is there a strong rationale for conducting these trials in light of emerging knowledge about SARS-CoV-2, especially in terms of the differential impact it has on different populations, but also false claims about efficacy of treatments made based on findings of non-randomized studies. Expedited trials that draw inaccurate conclusions based on low-quality evidence are not only a huge waste of resources, but can also significantly endanger the lives of millions of people around the world. Joining forces in the scientific community to conduct well-designed and large RCTs is our best bet for battling the rush to find a cure for COVID-19.
“Of the >2,000 planned drug studies examining COVID-19 treatments (hps://www.covid-trials.org), most have delivered lile or no directly useful information. Exceptions include two large, adaptive, pragmatic trials, RECOVERY and SOLIDARITY, which combined have randomized >20,000 patients to assess the effects of several treatments on mortality, and the US National Institutes of Health ACTT trial, which randomized 1,059 patients to assess the effect of remdesivir on time to disease resolution.” — Tikkinen et al., 2020 (2)
Current Treatment Trials for COVID-19
As of September 28, 2020, there have been 2342 COVID-19 treatment trials. Exhibit 1 shows countries with more than 50 trials, with the largest number of trials being conducted in the United States (3).
Exhibit 1: Countries with More than 50 COVID-19 Treatment Trials as of September 28, 2020 (3)
Country Total trials (N=2342) n (%)
United States 473 (20.2)
China 387 (16.5)
Iran 277 (11.8)
India 188 (8)
Spain 168 (7.2)
France 130 (5.6)
United Kingdom 106 (4.5)
Italy 89 (3.8)
Brazil 75 (3.2)
Germany 71 (3)
Canada 65 (2.8)
Egypt 60 (2.6)
Mexico 59 (2.5)
As of August 21, 2020, over 310 trials were reviewed by the Food and Drug Administration (FDA), many of which received Investigational New Drug Application (IND) Approval to administer their respective investigational drugs to humans (4).
Despite the remarkable and commendable progress made by the scientific community from around the world, there continues to be significant methodological concerns regarding the existing body of evidence for COVID-19 treatments. In this context, the value of well-conducted and large RCTs has been increasingly emphasized as they have the power to overcome the current limitations observed in COVID-19 treatment trials.
“the outcomes for many of these [COVID-19 treatment] trials involve time to symptom resolution, improvement of laboratory or radiographic abnormalities, or reduction in the use of mechanical ventilation. Few of the studies will be sufficiently powered to detect a difference in mortality. Although these are important clinical outcomes, and use of mechanical ventilation is associated with mortality, it will be important to objectively assess and accurately describe the outcomes from ongoing trials and what the results potentially mean in terms of improving overall survival. In addition, for trials with unblinded treatment allocation and unblinded outcome assessment, interpretation of findings, such as symptom resolution, may be problematic.” — Bauchner & Fontanarosa, 2020 (5)
“…drug responses can be influenced by factors tied to race and ethnicity, such as genetics. More than that, diversity in research is necessary to understand the biological underpinnings of diseases that, for example, make some people more susceptible or responsive to treatment.”
“While published trial data typically breaks down participants by demographics, the categories aren’t uniform. Still, it’s clear that most high-profile U.S. trials have been relying on largely White test pools. More than 70 per cent of participants in a late-stage Gilead Sciences Inc. trial of Remdesivir, used to treat Covid-19, were White. Black people accounted for less than 12 per cent.” — Brown, 2020 (6)
Why We Need Large Randomized Controlled Trials
In our previous INSIGHT “The Rush to Find a Cure: Are We Sacrificing Quality For Speed?”, we have discussed a wide range of methodological challenges that have been identified in COVID-19 treatment trials thus far. These challenges include, but are not limited to:
- Small number of patients
- Lack of patient important outcomes
- Lack of randomization, concealment of generated sequence, and blinding in RCTs
- Lack of steps necessary to minimize confounding in non-randomized studies
While the value of conducting expedited research during a pandemic cannot be denied, implementing shortcuts from a methodological perspective defeats the purpose of the scientific endeavor as it does not generate high-quality evidence.
From the beginning of the pandemic, there have been calls for large and well-conducted RCTs as they can validate the effectiveness, safety profile, and adverse effects of potential treatment candidates for COVID-19 (7). Small trials can lack statistical power to detect realistic, moderate treatment effects that are clinically significant, which may in turn lead potentially worthwhile interventions to be dismissed early in the therapeutic evaluation process (8,9). This is particularly concerning as initial non-promising results for interventions may prompt researchers to move onto test other treatment candidates for a disease, which may be tempting during the current pandemic and the rush to find a cure for COVID-19. Maggioni et al. (2020) explained that given our current knowledge of SARS-CoV-2, a moderate treatment effect is expected from interventions and to reliably demonstrate this effect, several
thousand patients are needed – which is a far cry from the majority of studies that have been conducted for COVID-19 treatment candidates (10). Another challenge of underpowered small trials is that if a statistically significant effect is found, this is most likely due to chance and an overestimation of the treatment effect size (9). This is also problematic, as it may prevent further trials from occurring, especially if it is considered unethical to randomize more patients if the treatment is effective according to statistical significance (9). Notably, earlier in the pandemic, statistically significant findings from several small RCTs have led chloroquine and hydroxychloroquine to gain much momentum in the media as potential treatment candidates for COVID-19, despite the clear lack of statistical power in these trials (11).
Beyond issues of statistical and clinical significance of findings from small RCTs, generalizability of findings from these trials also present a big challenge. This is highly relevant in the context of COVID19 as some sub-populations are at higher risk of more severe disease, including those older than 60 years and immunocompromised individuals (12). In order to identify the effectiveness and adverse effects of COVID-19 treatment candidates in different subgroups defined by age, sex, comorbidities, and severity of disease, large RCTs are needed. This is crucial for determining the most appropriate
treatment for patients. At a broader level, there is much need for sociodemographic diversity in the COVID-19 treatment trials. Individuals with different racial and ethnic backgrounds who have different life circumstances may experience different outcomes once infected with SARS-CoV-2, and may respond differently to different treatments (10). For example, African American individuals in the US have disproportionately carried a greater burden of the pandemic with many deaths, which can be due
to a combination of socioeconomic deprivation, lack of access to health care, and higher rates of comorbidities (11). Yet, they remain severely underrepresented in COVID-19 treatment trials. In fact, most of the high-profile trials in the US so far mostly included White participants (6). Although there are no magic numbers to determine the exact proportions of participants from different sociodemographic backgrounds to be included in the trials, they should be representative of the general population the
treatments are meant for (6). Given the global scope of the pandemic and the diverse populations impacted by COVID-19, the value of well-conducted and large RCTs is paramount to take an equitable approach to identifying suitable treatments for everyone. This is particularly relevant as more trials are
moving towards later stages of testing, and increasing the pool of participants to further test the safety and effectiveness of COVID-19 treatment candidates (6).
“During the COVID-19 pandemic, several false claims of efficacy have emerged from non-randomized comparisons (often misleadingly referred to as ‘real-world evidence’), and it has been refreshing to see how perfectly such weakly founded claims can be swept aside by evidence from properly conducted, large-scale, randomized trials.” — Tikkinen et al., 2020 (2)
Lessons from Current Large Randomized Trials
Two recent large RCTs for COVID-19 treatment candidates have demonstrated that the use of oldfashioned randomization, combined with established clinical-trials networks and innovative use of modern information technology can provide rapid and reliable answers needed during a pandemic (2). These were the RECOVERY and SOLIDARITY trials, and the following are the key characteristics that distinguish them from the majority of other COVID-19 treatment trials (Exhibit 3). Both of these are adaptive and pragmatic trials conducted through international collaborative work based on the World Health Organization R&D Blueprint (2). Both of these trials are large enough to test the effects of several treatments on mortality, which was a key limitation of several underpowered small trials conducted previously.
The key strength of both these trials is the large study size, which is crucial for overcoming the risk of several small trials not generating strong evidence on the relative effectiveness of potential COVID-19 treatment candidates (15). Notably, after much media buzz about hydroxychloroquine, both the RECOVERY and SOLIDARITY trials provided evidence of ineffectiveness of this drug within a few months of beginning the trials. This efficiency highlights the importance of investing in large RCTs because not only will they generate the critical treatment answers we need in a timely manner, but if they are well-coordinated, they will also prevent the waste of massive amounts of resources in these truly dire times. Exhibit 4 shows key strategies for designing meaningful clinical trials during a public health crisis.
Exhibit 4: Six Key Steps for Conducing Meaningful Clinical Trials During a Public Health Crisis
S Start with a question that matters
I Initiate local, national and international collaborations
G Give enough time to plan and incorporate methodological safeguards in study design
N Narrow down the scope of the study to keep it simple; focus on patients-centered outcomes
A Adopt a pragmatic approach and exercise flexibility in study design
L Learn from emerging evidence from diverse sources and plan next steps accordingly
OE Community Perspectives on COVID-19 Treatments
We conducted a poll within the OE community to gain their perspectives on COVID-19 treatments. Overall, 62% of the participants expect a treatment will be available by the end of fall 2021, whereas 38% of the participants expect a treatment to be available by the end of summer 2021.
Contributors
Mohit Bhandari, MD, PhD
Dr. Mohit Bhandari is a Professor of Surgery and University Scholar at McMaster University, Canada. He holds a Canada Research Chair in Evidence-Based Orthopaedic Surgery and serves as the Editor-in-Chief of OrthoEvidence.
Ayesha Siddiqua MSc, PhD
Ayesha Siddiqua has a Masters and a PhD from the Health Research Methodology Program in the Department of Health Research Methods, Evidence, and Impact at McMaster University.
References
1. Glasziou PP et al (2020). Waste in covid-19 research. BMJ; 369: m1847. DOI: 10.1136/bmj.m1847
2.Tikkinen et al (2020). COVID-19 clinical trials: learning from exceptions in the research chaos. Nature Medicine. DOI: 10.1038/s41591-020-1077-z
3.Cytel (2020, September 28). Global Coronavirus COVID-19 Clinical Trial Tracker. Retrieved from https://www.covid-trials.org/
4. U.S. Food & Drug Administration (2020, September 28). Coronavirus Treatment Acceleration Program (CTAP). Retrieved from
https://www.fda.gov/drugs/coronavirus-covid-19-drugs/coronavirus-treatment-acceleration-program-ctap
5. Bauchner H & Fontanarosa (2020). Randomized Clinical Trials and COVID-19 Managing Expectations. JAMA; 323(22): 2262-2263. DOI: 10.1001/jama.2020.8115.
6. Brown KV (2020, July 30). COVID-19 clinical trials aren’t very diverse and that’s a problem. Retrieved from https://nationalpost.com/news/world/covid-19-clinicaltrials-arent-very-diverse-and-thats-a-problem
7. Khan Z et al (2020). Anti COVID-19 Drugs: Need for More Clinical Evidence and Global Action. Advances in Therapy; 37(6): 2575-2579. DOI: 10.1007/s12325-020-01351-9
8. Altman DG & Bland JM (1995). Absence of evidence is not evidence of absence. BMJ; 311(7003): 485. DOI: 10.1136/bmj.311.7003.485
9.Guyatt GH et al (2008). In the Era of Systematic Reviews, Does the Size of an Individual Trial Still Matter? PLoS Medicine; 5(1): e4. DOI:
10.1371/journal.pmed.0050004
10. Maggioni A et al (2020). Key words to be adopted for COVID-19 research A return to simple, large, randomized trials. European Heart Journal; 2020(0): 1-2.
DOI: 10.1093/eurheartj/ehaa322
11.Alexander PE et al (2020). COVID-19 coronavirus research has overall low methodological quality thus far: Case in point for chloroquine/hydroxychloroquine.
Journal of Clinical Epidemiology; 123: 120-126. DOI: 10.1016/j.jclinepi.2020.04.016
12. World Health Organization (2020). COVID-19: vulnerable and high risk groups. Retrieved from https://www.who.int/westernpacific/emergencies/covid19/information/high-risk-groups#:~:text=COVID%2D19%20is%20often%20more%20severe%20in%20people%2060%2Byrs,who%20are%20at%20most%20risk
13. RECOVERY trial (2020). Retrieved from https://www.recoverytrial.net/
14. RECOVERY trial (2020). Randomized Evaluation of COVID-19 Therapy (RECOVERY). Retrieved from https://www.recoverytrial.net/files/recovery-protocol-v7-0-2020-06-18.pdf
15. World Health Organization (2020). “Solidarity” clinical trial for COVID-19 treatments. Retrieved from https://www.who.int/emergencies/diseases/novelcoronavirus-2019/global-research-on-novel-coronavirus-2019-ncov/solidarity-clinical-trial-for-covid-19-treatments
Citation:
(October 10, 2020- No 23)
OrthoEvidence. ‘Randomized Trials’ During the SARS-CoV2
Pandemic: What Went Wrong? OE Insight. Available from: https://myorthoevidence.com/Download/68f3de38-7c0f-44fa-8095-d8c8972e9bff?vgo_ee=2UNi3BwdKSCCXrre6%2BvV2yH8YZLl63WNbSgmQq3NzPQ%3D
