Arnold Ventures FDA Draft Guidance RWE | June 18, 2024

June 18, 2024 

Dockets Management Staff (HFA-305) 

Food and Drug Administration 

5630 Fishers Lane, Rm. 1061 

Rockville, Maryland 20852 

Re: FDA-2023-D-5470

Filed electronically at http://​www​.reg​u​la​tions​.gov

Dear Commissioner Califf, 

Thank you for the opportunity to provide comments to the Food and Drug Administration (FDA) on the following draft guidance issued in March 2024: 

• Real-World Evidence: Considerations Regarding Non-Interventional Studies for Drug and Biological Products

Arnold Ventures is a philanthropy dedicated to investing in evidence-based policy solutions that maximize opportunity and minimize injustice. We work to develop evidence that drives reform across a range of issues including health care, education, and criminal justice. Our work within the health care sector is driven by the recognition that the system costs too much and often fails to adequately care for the people it serves.

We appreciate FDA’s efforts to help sponsors identify and address the presence of confounding and other forms of bias when planning and conducting non-interventional studies (also referred to as observational studies).

Background

Real-world data (RWD) are data relating to patient health status and health care services. These data are routinely collected from various sources, such as electronic health records, administrative claims, and registries. The 21^st Century Cures Act signed into law on December 13, 2016, is designed to help accelerate medical product development by considering new uses of RWD to generate real-world evidence (RWE). We believe that RWE cannot replace randomized interventional studies, but it can complement those studies by:

• Understanding the course of disease or standard of clinical care before designing a trial; 
• Supporting the recruitment, enrollment, and retention of representative study populations when conducting a trial; and 
• Validating post-approval surveillance

Therefore, we recommend the following modifications to the draft guidance to make clear that non-interventional studies alone cannot demonstrate substantial evidence of effectiveness, which is the standard required by FDA when approving new drugs and biologics:

1. Limit study designs to those that can determine when the drug’s effect is distinguishable from other influences on the study.
2. Verify that RWD sources are mapped onto common data models to standardize the data for comparison across study protocols.
3. Expand guidance to include more information about analytic approaches that address confounding.

Recommendations

1. Limit study designs to those that can determine when the drug’s effect is distinguishable from other influences on the study

We recommend that FDA make clear in final guidance that any proposed approach to develop RWE must address sources of confounding and avoid the risk of bias. i Given that RWE generated from RWD cannot randomize subjects to relevant active comparators, there may be differences between groups of patients that make the treatment effects indistinguishable from other influences. This leaves observed results unexplained by the interventions under study.

FDA acknowledges this draft guidance is focused on submission of observational study designs that include a proposed approach to support causal inference, as well as address confounding and other types of bias. However, we recommend that final guidance go further to acknowledge that bias cannot be mitigated through statistical models.ii Sponsors should submit to FDA all prospectively specified covariates and feasibility studies as part of the study design discussion. Furthermore, final guidance should make certain that sponsors confirm an absence of unmeasured confounding to ensure validity of inferences made from observational data.