Trials avoid high risk patients and underestimate drug harms

Abstract: "The FDA does not formally regulate representativeness, but if trials under-enroll vulnerable patients, the resulting evidence may understate harm from drugs. We study the relationship between trial participation and the risk of drug-induced adverse events for cancer medications using data from the Surveillance, Epidemiology, and End Results Program linked to Medicare claims. Initiating treatment with a cancer drug increases the risk of hospitalization due to serious adverse events (SAE) by 2 percentage points per month (a 250% increase). Heterogeneity in SAE treatment effects can be predicted by patient's comorbidities, frailty, and demographic characteristics. Patients at the 90th percentile of the risk distribution experience a 2.5 times greater increase in SAEs after treatment initiation compared to patients at the 10th percentile of the risk distribution yet are 4 times less likely to enroll in trials. The predicted SAE treatment effects for the drug's target population are 15% larger than the predicted SAE treatment effects for trial enrollees, corresponding to 1 additional induced SAE hospitalization for every 25 patients per year of treatment. We formalize conditions under which regulating representativeness of SAE risk will lead to more externally valid trials, and we discuss how our results could inform regulatory requirements."

This seems like an odd criticism.

First off it ignore the fact that if you include frail patients you’ll confound the results of the trial. So there is a good reason for it.

Second, saying “rate of SAE is higher than rate of treatment effect” is a bit silly considering these are cancer trial - without treatment there is a risk of death so most people are willing to accept SAE in order to achieve treatment effect.

Third, saying “the sickest patients saw the highest increase in SAE” seems obvious? It’s exactly what you’d expect.

First, ignoring frail patients means your trial isn't representative of the wider population, so it shouldn't be accepted for general use - only on people who were well-represented in the trial.

Second, you're ignoring the possibility of other treatment options. It isn't always the binary life-or-death you're making it, so SAEs do matter.

Third, a big part of trials is to discover and develop prevention methods for SAEs. Explicitly ignoring the people most likely to provide data valuable for the general population sounds like a pretty silly approach.