←back to thread

Trials avoid high risk patients and underestimate drug harms

(www.nber.org)
174 points bikenaga | 1 comments | 08 Dec 25 19:07 UTC | HN request time: 0s | source
Show context
bikenaga ◴[08 Dec 25 19:08 UTC] No.46196315[source]▶
Abstract: "The FDA does not formally regulate representativeness, but if trials under-enroll vulnerable patients, the resulting evidence may understate harm from drugs. We study the relationship between trial participation and the risk of drug-induced adverse events for cancer medications using data from the Surveillance, Epidemiology, and End Results Program linked to Medicare claims. Initiating treatment with a cancer drug increases the risk of hospitalization due to serious adverse events (SAE) by 2 percentage points per month (a 250% increase). Heterogeneity in SAE treatment effects can be predicted by patient's comorbidities, frailty, and demographic characteristics. Patients at the 90th percentile of the risk distribution experience a 2.5 times greater increase in SAEs after treatment initiation compared to patients at the 10th percentile of the risk distribution yet are 4 times less likely to enroll in trials. The predicted SAE treatment effects for the drug's target population are 15% larger than the predicted SAE treatment effects for trial enrollees, corresponding to 1 additional induced SAE hospitalization for every 25 patients per year of treatment. We formalize conditions under which regulating representativeness of SAE risk will lead to more externally valid trials, and we discuss how our results could inform regulatory requirements."
replies(1): >>46198178 #
refurb ◴[08 Dec 25 21:54 UTC] No.46198178[source]▶
This seems like an odd criticism.

First off it ignore the fact that if you include frail patients you’ll confound the results of the trial. So there is a good reason for it.

Second, saying “rate of SAE is higher than rate of treatment effect” is a bit silly considering these are cancer trial - without treatment there is a risk of death so most people are willing to accept SAE in order to achieve treatment effect.

Third, saying “the sickest patients saw the highest increase in SAE” seems obvious? It’s exactly what you’d expect.

replies(1): >>46199818 #
crote ◴[09 Dec 25 00:37 UTC] No.46199818[source]▶
First, ignoring frail patients means your trial isn't representative of the wider population, so it shouldn't be accepted for general use - only on people who were well-represented in the trial.

Second, you're ignoring the possibility of other treatment options. It isn't always the binary life-or-death you're making it, so SAEs do matter.

Third, a big part of trials is to discover and develop prevention methods for SAEs. Explicitly ignoring the people most likely to provide data valuable for the general population sounds like a pretty silly approach.

replies(4): >>46199909 #>>46200069 #>>46200860 #>>46202153 #
1. sfink ◴[09 Dec 25 03:14 UTC] No.46200860[source]▶
> First, ignoring frail patients means your trial isn't representative of the wider population, so it shouldn't be accepted for general use - only on people who were well-represented in the trial.

Sure, but including frail outliers does not automatically mean you can generalize to the whole population. People can be frail for a wide variety of reasons. Only some of those reasons will matter for a given trial. That means the predictive power varies widely depending on which subpopulation you're looking at, and you'll never be able to enroll enough of some of the subgroups without specifically targeting them.

The results in the posted paper seem valid to me, but the conclusion seems incorrect. This seems like a paper that is restating some pretty universal statistical facts and then trying to use that to impose onerous regulations that can't and won't solve the problem. It will improve generalizability for a small fraction of the population, at a high cost.

> Second, you're ignoring the possibility of other treatment options. It isn't always the binary life-or-death you're making it, so SAEs do matter.

Of course they do. It's a good thing we have informed consent.

> Third, a big part of trials is to discover and develop prevention methods for SAEs. Explicitly ignoring the people most likely to provide data valuable for the general population sounds like a pretty silly approach.

If your primary claim is that data from non-frail people is not generalizable to frail people, then how can you claim that data from frail people is generalizable to non-frail people? If the trials for aspirin found that hemophiliacs should get blood clot promoting medications along with it, then should non-hemophiliacs also be taking those medications?

I'm thankful we can extract some amount of useful data from these trials without undue risk. It's always going to be a balancing act, and this article proposes putting a thumb on the scale that reduces the data without even solving the problem it's aiming at addressing.