A recent issue of Clinical Pharmacology & Therapeutics focused entirely on real-world data (RWD) and real-world evidence (RWE), synthesizing ideas and opinions from thought leaders across sectors. The industry-wide discussion in the journal represents the rising momentum of the field—with an increase in the quantity and diversity of stakeholders joining forces to innovate and overcome hurdles, we expect the science and applications of RWE to develop even further and faster.
The contributors agreed on four major points:
1. Regulatory decision-making: Proven successes and potential applications for RWE
Using RWE to support regulatory decisions is not new in pharmacovigilance. In the paper “Real‐world data, advanced analytics, and the evolution of postmarket drug safety surveillance,” Gerald Dal Pan, M.D. M.H.S., director of the Office of Surveillance and Epidemiology in the Food and Drug Administration’s (FDA) Center for Drug Evaluation and Research (CDER), summarizes how a variety of data sources are used to generate drug safety evidence, and promises continued efforts to expand data access and advance methods to support safety surveillance and benefit-risk decision-making.
When the objective is to evaluate treatment efficacy, however, the use of RWD is still controversial. For almost 60 years, after all, randomized controlled trials (RCTs) have been the “gold standard” to demonstrate the efficacy of treatments—and are likely to remain so under many circumstances.
Can RCTs answer every question? Not necessarily. RWE is drawing new interest when a traditional RCT may be infeasible or unethical. Regulators are beginning to warm up to RWE, too, specifically the FDA and European Medicines Agency (EMA), as they explore potential use cases for RWE in regulatory decision-making. Both agencies have begun to turn to RWE for external control arms, indication expansions, and prospective or even retrospective observational studies to generate regulatory-grade evidence.
Applications for RWE don’t stop there—global health, clinical pharmacology, and medical devices are also areas highlighted for their potential in the issue.
2. Separating the good from the flawed: Methodological challenges of RWE
To establish RWE as sound evidence for regulatory decisions, researchers must meet rigorous standards for study design. Methodology is the cornerstone of working with RWD to minimize confounding and generate the best evidence possible.
While flawed RWE analyses exist due to the inherent features of the data sources, RWE cannot be categorically disregarded, writes Sebastian Schneeweiss, M.D., Sc.D., professor of medicine and epidemiology at Harvard Medical School and Brigham and Women’s Hospital, in his paper, “Real‐world evidence of treatment effects: The useful and the misleading.” It is essential, he says, to identify tools that enable reviewers to discriminate between actionable and erroneous RWE, and advocates for principled designs of RWE analyses to help minimize confounding bias and avoid known design flaws.
Careful consideration before performing the study is more likely to yield valid findings, save time and frustration, and avoid misinformation. “To convince its critics,” he writes, “advocates of RWE will need to get this right, reliably and predictably.”
Anuradha Ramamoorthy, Ph.D., and Shiew-Mei Huang, Ph.D., policy lead and deputy director respectively of the Office of Clinical Pharmacology in FDA’s CDER, echo Dr. Schneeweiss in their paper “What does it take to transform real-world data into real-world evidence?”, noting that “it is critical to determine when and how RWD analysis may lead to valid evidence generation vs. when it will not.”
3. Transparency, transparency, transparency
In “Real‐world data for regulatory decision making: Challenges and possible solutions for Europe,” the EMA identifies transparency as one of the challenges with the use of RWD to generate acceptable RWE, and emphasizes that “a detailed description of study design and data collected, and full transparency on the protocol and study report (with registration in a publicly available database) would do much to build confidence in results and avoid the confusion created by disparate results.” Similarly, in his paper, Dr. Schneeweiss mentioned that tools that facilitate full transparency will help investigators understand and assess a study efficiently and without burden.
As RWE researchers, we frequently encounter varying results across studies attempting to answer the same study question, likely due to the differences in design and analytical approach. In a recent abstract presented at ICPE 2019, researchers from Amgen and Aetion worked in parallel to evaluate the reproducibility of a study protocol with the Aetion Evidence Platform™ (AEP) and SAS®️. The initial study execution yielded discrepant study results, even though a detailed protocol was set in place prior to implementation. Only after two rounds of careful investigation into potential differences in implementation approaches was concordance achieved.
The study identified reasons for initially observed discrepancies and, based on that work, investigators were able to make recommendations on important components of study protocols to ensure transparency in research. This demonstrates why scientific rigor matters, which is also a snapshot of the reality that transparency is not easy to achieve but will be of great importance in further facilitating communications and benefiting the RWE ecosystem.
4. Diverse stakeholders expand the potential for RWD
With the rapid growth of health care digitization, new sources of data bring new opportunities for evidence to support decision-making, particularly as new stakeholders join the game. The Applied Proteogenomics OrganizationaL Learning and Outcomes (APOLLO) network published how the Department of Defense and Department of Veterans Affairs are collaborating to implement a prospective curation and translation of RWD into RWE. Multiple stakeholders also shared the progress of the RWD evolution in precision oncology and digital health.
With more data sources becoming available and improved data infrastructures enabling access, there is an urgent need to consider and plan for future data demands. The inherent heterogeneity of RWD sources across health care systems is a major technological challenge to fully realizing the power of RWE. The EMA emphasizes the importance of planning for data collection early as more data sources become available, and encourages collaboration among stakeholders planning to use RWE for decision-making, as their joined effort could help standardize and validate RWD.
As new stakeholders join the RWD learning process, they help accelerate the application of next-generation RWD. With the right momentum, collaboration among stakeholders can become a virtuous cycle. As Dr. Schneeweiss notes, RWE researchers can “illuminate the process of data generation and curation up to the point when the data are used for analysis.” The subsequent improvement in data collection empowers researchers to engage stakeholders, including patients, providers, and regulatory agencies, to optimize the caregiving procedure and provide evidence of an improved standard of care. Jacqueline Corrigan‐Curay, J.D., M.D., and M. Khair ElZarrad, Ph.D., M.P.H., director and deputy director respectively of the Office of Medical Policy within FDA’s CDER, explain that the community at large needs to evaluate how best to maximize the RWE utility actively and also reinforces its commitment to engage stakeholders in this critical effort by “developing guidance around RWD collection and quality, study design, and good clinical practice to keep pace with these advances.”