Abstract | June 2021
Background: Statistical testing in phase III clinical trials is subject to chance errors, which can lead to false conclusions with substantial clinical and economic consequences for patients and society.
Methods: We collected summary data for the primary endpoints of overall survival (OS) and progression-related survival (PRS) (eg, time to other type of event) for industry-sponsored, randomized, phase III superiority oncology trials from 2008 through 2017. Using an empirical Bayes methodology, we estimated the number of false-positive and false-negative errors in these trials and the errors under alternative P value thresholds and/or sample sizes.
Results: We analyzed 187 OS and 216 PRS endpoints from 362 trials. Among 56 OS endpoints that achieved statistical significance, the true efficacy of experimental therapies failed to reach the projected effect size in 33 cases (58.4% false-positives). Among 131 OS endpoints that did not achieve statistical significance, the true efficacy of experimental therapies reached the projected effect size in 1 case (0.9% false-negatives). For PRS endpoints, there were 34 (24.5%) false-positives and 3 (4.2%) false-negatives. Applying an alternative P value threshold and/or sample size could reduce false-positive errors and slightly increase false-negative errors.
Conclusions: Current statistical approaches detect almost all truly effective oncologic therapies studied in phase III trials, but they generate many false-positives. Adjusting testing procedures in phase III trials is numerically favorable but practically infeasible. The root of the problem is the large number of ineffective therapies being studied in phase III trials. Innovative strategies are needed to efficiently identify which new therapies merit phase III testing.
Funding
Grants and funding
Citation Shen C, Ferro EG, Xu H, Kramer DB, Patell R, Kazi DS. Underperformance of Contemporary Phase III Oncology Trials and Strategies for Improvement. J Natl Compr Canc Netw. 2021 Jun 21;19(9):1072-1078. doi: 10.6004/jnccn.2020.7690. PMID: 34153943.
