Real-World Evidence — Where Are We Now?

John Concato, M.D., M.P.H., and Jacqueline Corrigan‑Curay, J.D., M.D.

More than 5 years after the passage of the 21st Century Cures Act of 2016, the terms “real-world data” (RWD) and “real-world evidence” (RWE) are being used inconsistently and sometimes interchangeably. This imprecision has complicated efforts to assess the impact of such data and evidence and hindered attempts to track their use. 

The Food and Drug Administration (FDA), in its Framework for FDA’s Real-World Evidence Program,¹ defined RWD as “data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources” and defined RWE as “clinical evidence about the usage and potential benefits or risks of a medical product derived from analysis of RWD,” regardless of the type of study design. 

But there are two widespread misconceptions about these terms. The first is the notion that RWD and RWE were brand-new concepts in 2016. In reality, sources of data and types of study designs haven’t fundamentally changed, but electronic access to more detailed clinical data is evolving, and such information’s reliability and relevance to research are improving. The availability of more robust data on clinical factors that affect health outcomes also provides opportunities for exploring various statistical methods in lieu of randomization. Including “RWD” or “RWE” in the description of a study, however, doesn’t tell us exactly where the data came from or what kind of study architecture is involved. Providing more specifics about data sources and study design can reduce confusion over RWD and RWE.²

The second misconception is that a simple dichotomy between randomized, controlled trials (RCTs) and observational studies delineates the entire landscape of study design.³ Although randomization of treatment assignment is a key strength of RCTs, not all clinical trials are randomized; rather, their defining feature is assignment of treatment according to an investigational protocol. For example, in single-group trials, investigators assign participants to receive an intervention without randomization — and face challenges similar to those in observational studies in determining whether differences in clinical outcomes between the protocol-driven group and a comparator (“control”) group represent actual treatment effects. 

Correcting these misconceptions requires recognition that the degree of reliance on RWD varies with the type of study design and that, by definition, RCTs that incorporate RWD generate RWE (see diagram). This conceptualization confirms that even when strict eligibility criteria may limit the generalizability of trial results, trial participants exist, and their outcomes occur, in the “real world” — despite perceptions that generation of RWE occurs only outside clinical trials. Also, although the terms “clinical trials” and “observational studies” have clear meanings when used properly, the terms “interventional studies” and “noninterventional studies” have advantages in describing whether the treatment of interest was administered according to a study protocol. 

These conceptual distinctions were less pertinent when causal inferences regarding therapeutic effectiveness relied mainly on interventional studies with primary data collected in traditional RCTs. Increasingly, however, RCTs in corporate RWD, and when randomization isn’t feasible for ethical or other reasons, externally controlled trials include a comparator group derived entirely from a source of secondary data (“external” to the treatment group). Conversely, noninterventional studies that analyze primary data collected from registries are being conducted more often. 

Notwithstanding confusion regarding these terms and concepts, we at the FDA continue to evaluate RWD and RWE as we consider regulatory decisions. Indeed, the agency published four related draft guidance documents in 2021.⁴ FDA guidance on data from electronic health records and medical claims databases includes recommendations on how to select relevant data sources and define and validate study variables; other guidance provides recommendations on designing or using an existing registry to support regulatory decision making. A guidance document on data standards advises sponsors to document a rationale for changes made to ensure that RWD conform to FDA-supported data standards, and guidance on regulatory considerations describes the FDA’s expectations regarding noninterventional (observational) studies that use only RWD. 

Although data generated by digital health technologies don’t meet the strict definition of RWD if provided in the context of a clinical trial, their suitability for use in clinical studies warrants mention. Such technologies — including software applications and sensor hardware used to remotely obtain physiological or behavioral data — have an expanding role in health care and, when the data they generate are verified and valid, offer considerable opportunities for drug development.

Other FDA initiatives supporting the deployment of RWD and RWE in product development include various demonstration projects aimed at improving the usefulness of RWD, exploring methods of designing studies and analyzing data to generate RWE, or developing specific tools and techniques to assist in this process. An example is the OneSource Project (www.fda.gov/science-research/advancing-regulatory-science/source-data-capture-electronic-health-records-ehrs-using-standardized-clinical-research-data), which is developing approaches for automating the flow of structured data from electronic health records into external systems to facilitate research and narrow the divide between patient care and clinical investigations.

Although approval of drugs and biologics based on what we now call RWE predates the 21st Century Cures Act, two approvals in the past 5 years illustrate the issues we raise here. In 2021, the approval by the Center for Drug Evaluation and Research (CDER) of tacrolimus (Prograf) in combination with other immunosuppressant drugs for the prevention of organ rejection in patients receiving lung transplants (www.fda.gov/drugs/news-events-human-drugs/fda-approves-new-use-transplant-drug-based-real-world-evidence) was based on a noninterventional study comparing data from a well-established registry with data from historical controls. In addition to relying on RWE for FDA approval and aligning with patients’ and clinicians’ perspectives, the new indication for lung transplantation represents CDER’s first acceptance of an “observational study” as an adequate and well-controlled study providing the primary support for a finding of substantial evidence of effectiveness. 

In 2019, the Center for Biologics Evaluation and Research approved onasemnogene abeparvovec-xioi (Zolgensma) as an adeno-associated virus vector–based gene therapy for the treatment of patients younger than 2 years of age who have spinal muscular atrophy and a specific biallelic mutation (www.fda.gov/news-events/press-announcements/fda-approves-innovative-gene-therapy-treat-pediatric-patients-spinal-muscular-atrophy-rare-disease). This approval was based on assessment of primary outcomes (survival and achieving a functional milestone) among participants receiving the biologic product in a single-group trial and comparison of those outcomes with RWD from patients in studies of the natural history of the condition. Although RWD were less prominent here than in the tacrolimus approval, in both cases, reviewers found the data fit for use and concluded that the study design addressed the regulatory question and that the study conduct met FDA requirements.¹

The CDER approval also highlights the fact that an observational design is not synonymous with use of secondary data.2 When primary data collection occurs in noninterventional studies, such as those using registries that collect data in a standardized format, investigators may encounter fewer challenges than they do with electronic health records, medical claims, or other sources, in terms of variability in the conduct and timing of clinical assessments. More general issues related to data quality include clinical relevance and reliability (e.g., accuracy, completeness, provenance, and traceability). 

Other considerations include whether and how the Covid-19 pandemic has changed perceptions of RWD and RWE and practices related to them. In general, the pandemic has accelerated awareness and adoption of RWD and RWE, but their use was already increasing before the pandemic. In addition, though robust RWE has sometimes informed pandemic responses,⁵ challenges involved in diagnosing, treating, and reporting on a new disease can create methodologic problems (and studies can be invalid for other reasons, such as research misconduct). Overall, Covid-19 presents an opportunity to leverage RWD to inform clinical and regulatory decisions, but scientific rigor must be maintained. 

The FDA remains committed to robust policy development aligned with the 21st Century Cures Act while maintaining evidentiary standards in honoring our obligation to protect and promote public health. Focusing on the distinction between interventional studies and noninterventional studies can help researchers, sponsors, and regulators better understand and describe relevant methodologic issues. Gaining more experience, including the conduct of rigorous noninterventional studies, will help to advance drug development. 

The views expressed in this article are those of the authors and do not necessarily represent the views or policies of the Food and Drug Administration or Yale University. 

Disclosure forms provided by the authors are available at NEJM.org. 

From the Office of Medical Policy (J.C.) and the Office of the Center Director (J.C.-C.), Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD; and Yale University, New Haven, CT (J.C.). 

This article was published on April 30, 2022, at NEJM.org.

1. Food and Drug Administration. Framework for FDA’s real-world evidence program. December 2018 (https://www.fda.gov/media/120060/download).
2. Concato J, Stein P, Dal Pan GJ, Ball R, Corrigan-Curay J. Randomized, observational, interventional, and real-world-What’s in a name? Pharmacoepidemiol Drug Saf 2020;29:1514-7.
3. Collins R, Bowman L, Landray M, Peto R. The magic of randomization versus the myth of real-world evidence. N Engl J Med 2020;382:674-8.
4. Food and Drug Administration. Real-world evidence (https://www.fda.gov/science-research/science-and-research-special-topics/real-world-evidence).
5. Andrews N, Tessier E, Stowe J, et al. Duration of protection against mild and severe disease by Covid-19 vaccines. N Engl J Med 2022;386:340-50.

DOI: 10.1056/NEJMp2200089

Copyright © 2022 Massachusetts Medical Society.

For more information, email karyn.digiorgio@quantumleaphealth.org