As COVID-19 continues to evolve, frontline health care workers are eager for real-world evidence (RWE) to help guide and reinforce treatment decisions in real time. While randomized controlled trials (RCTs) have traditionally been a strong tool for assessing the efficacy and safety of a medicine, RWE provides important insight on a treatment’s use in clinical practice that can complement findings from clinical trials.
The rapid pace of the COVID-19 pandemic has increased reliance on RWE to help inform clinical management of the disease. To address these needs, various organizations have turned to RWE to help inform real-time decision-making on COVID-19 interventions.
This was the case for Gilead Sciences’s remdesivir, which the U.S. Food and Drug Administration (FDA) approved as the first antiviral treatment for COVID-19. Remdesivir’s approval was based in part on the Adaptive COVID-19 Treatment Trial 1 (ACTT-1), which was a randomized, double-blind, placebo-controlled trial with time to recovery as the primary endpoint. With these findings, remdesivir became the standard of care in the U.S. for COVID-19 treatment.
As the pandemic progressed, it became increasingly clear that assessing whether remdesivir had a mortality benefit was important. Although ACTT-1 showed a trend towards reduction in mortality that was not statistically significant, and a substantial reduction in mortality in low-flow patients in a post hoc analysis, the trial was not large enough to assess mortality with sufficient statistical power. With remdesivir considered a standard of care, further placebo-controlled trials were considered unethical. Therefore, real-world data (RWD) became central in the evaluation of the potential mortality benefit of remdesivir.
To explore this, scientists from Gilead and Aetion collaborated on a real-world effectiveness study comparing remdesivir-exposed patients to matched referent patients using a pre-specified analytical plan. The findings were presented in a poster at this year’s World Microbe Forum, and we describe our approach below.
Intent of the study
The goal of this study was to characterize remdesivir’s real-world effect on mortality as well as hospital discharge for people hospitalized with COVID-19. This comparative effectiveness study examined clinical outcomes of 49,712 adults hospitalized with COVID-19 using de-identified U.S.-based patient data from HealthVerity, analyzed in collaboration with Gilead.
Process for the analysis
Remdesivir is an intravenous medication, administered in the hospital. Therefore, HealthVerity data was selected for this analysis, which includes hospital chargemaster data linked to claims, electronic medical records, and lab data. This is a large, regularly updated data source, and we observed that the chargemaster data provided as complete a view as possible of the real-world utilization and in-hospital outcomes of remdesivir.
To build the study population, we identified hospitalized adults newly diagnosed with COVID-19 between May 1, 2020, and May 3, 2021, then selected patients who received at least one remdesivir infusion. To define the control group, we matched referent patients to the remdesivir patients using risk set sampling according to a range of factors—such as start date of remdesivir treatment, number of days since hospital admission, demographic characteristics, oxygen support requirements at match, ICU status, and corticosteroid use—as well as propensity score matching to further establish comparable groups.
As we were working with hospital data, we chose to focus on concrete, in-hospital endpoints that were unlikely to show bias between those in the study and control populations: the primary being in-hospital death, with a maximum follow-up of 28 days. We also assessed routine discharge from the hospital.
We used the Aetion Evidence Platform® to run the analysis, to confirm the definitions of the determined variables, and to collaborate in real time. The platform also allowed us to effectively and efficiently iterate, ensuring the Gilead and Aetion teams were aligned on each decision during implementation of the analysis plan, and to apply the same analytical tools to different subgroups.
Learnings and key findings
Throughout our process, we found that working with COVID-19 RWD comes with a unique set of challenges. For example, as understanding of COVID-19 grew, the standards of care and health care utilization varied and evolved rapidly, and outcome rates changed. The introduction of highly effective vaccines has also affected the course of the pandemic. RWD captures and reflects all of these changes, so analyses of RWD must consider this evolution.
Another learning is the importance of collaboration and knowledge sharing. The Reagan-Udall Foundation for the FDA and Friends of Cancer Research have led the COVID-19 Evidence Accelerator, a consortium that brings diverse minds and datasets together to share the approaches that work best for analysis of COVID-19 RWD. We have been regular and active participants in this group since its inception and have benefited tremendously from the thoughtful scientific exchange to help inform our approach.
As we prepared for the analysis, we had to account for specific aspects of remdesivir treatment. The drug is administered via a five-day, in-hospital, intravenous infusion, so it has a short-term exposure that requires patients to remain in the hospital. Therefore, the point in the course of a patient’s hospitalization and disease progression at which remdesivir is initiated is likely to be important. How could we account for this variability between the patient cohorts in RWD? It was necessary to carefully assess the optimal variables and approaches to address this effectively. We also had to understand the level of granularity in RWD, and how to consider factors like the number of days of exposure and changes in levels of oxygen support.
In the end, after extensive work and adaptation of our approach to ensure as robust an analysis as possible, we found that remdesivir is associated with a significant reduction in mortality—by 23 percent—in patients hospitalized with COVID-19 across all baseline oxygen requirement subgroups, from patients requiring no oxygen support at the time of remdesivir initiation to those needing invasive mechanical ventilation. We also observed a significantly improved likelihood of hospital discharge after completion of five days of remdesivir infusion, an effect that was most marked among patients with the lowest oxygen support requirements.
Impact and future areas of focus for COVID-19 RWE studies
By complementing the findings from the ACTT-1 RCT with RWE, we were able to build a more complete picture of how remdesivir helps patients in real-world settings. Our study population included nearly 50 times the number of patients as the RCT, which took place in the earliest days of the pandemic. Based on the dataset and methods used, our findings reflect the real-world effectiveness of remdesivir in hospitalized U.S. patients with COVID-19.
This study also emphasizes the importance of choosing the right methods when designing an RWE study. Compared to clinical trials where randomization is built in, RWE studies require researchers to account for potential confounding and bias. Particular to studying remdesivir for COVID-19, we found that we needed to emulate an RCT as closely as possible in our RWE study to mitigate the biases that could arise if, say, one patient was administered remdesivir late in the course of their disease, and another received it early in their disease. Furthermore, with the evolution of our collective understanding of COVID-19 during the course of the pandemic, standards of care have changed, with treatment initiation occurring earlier and mortality rates decreasing, and thus comparisons must account for calendar time. Our hope is that other RWE researchers will take these methodological concerns into account in their own studies, including confounding, immortal time bias, and selection bias. Here, we used risk set sampling and propensity score matching to help balance those populations, and we believe the rigor in our methods is critical when appraising our findings.
Going forward, there is a need to understand an area of growing concern in COVID-19: long-term COVID-19. While much attention has focused on outcomes of COVID-19 infection within the first 28 days, long-term COVID-19 remains relatively poorly characterized. What are the symptoms and outcomes that define this condition, and are they different from what we know of COVID-19 today? Are there treatments available to help minimize the effects of long-term COVID? These will be areas of focus in the future, and RWE will likely play a role in answering these questions.