key: cord-307426-fkzk8ubd
authors: Su, Zheng
title: The Critical Role of Futility Analysis in the Pursuit of Effective Treatments for COVID-19
date: 2020-06-09
journal: Contemp Clin Trials Commun
DOI: 10.1016/j.conctc.2020.100581
sha: 
doc_id: 307426
cord_uid: fkzk8ubd

nan

The potential to stop a trial or some of the treatment arms early for futility is critical in the pursuit of effective treatments for COVID-19. A large number of experimental treatments are being studied, and it is critical to be able to eliminate ineffective treatments early in order to reallocate limited resources to more promising ones. Many of the experimental treatments under testing for COVID-19 have limited preclinical and clinical data to support their potential effectiveness. Various statistical methodologies exist in the literature for determining the futility stopping criteria at interim analyses of clinical trials. For example, Pei et al. [2] and Yi et al. [3] reviewed some methods based on conditional power and predictive power for setting the futility stopping boundaries.

A randomized trial of hydroxychloroquine as postexposure prophylaxis for COVID-19 was recently completed, which demonstrated that hydroxychloroquine did not prevent illness compatible with COVID-19 or confirmed infection when used as postexposure prophylaxis within 4 days after exposure [4] . At the third interim analysis, this trial was halted for futility on the basis of a conditional power of less than 1%. The early futility stopping of the trial provided timely information on the potential of hydroxychloroquine as postexposure prophylaxis at a time when clarity on the role of hydroxychloroquine in the management of COVID-19 was urgently needed. It may be debatable whether adopting a futility cutoff of higher than 1% for the conditional power could have led to earlier stopping of the trial.

The RECOVERY trial was launched in March to test several potential treatments for COVID-19, including hydroxychloroquine [5] . A registry analysis of hydroxychloroquine published on May 22, 2020 [6] , which was later retracted, prompted the independent Data Monitoring Committee (DMC) of the RECOVERY trial to conduct an urgent review of the data on the effects of hydroxychloroquine on mortality among hospitalized patients with COVID-19. The committee concluded that there was no cogent reason to suspend recruitment for safety reasons and recommended that the trial continue recruitment without interruption [7]. On June 4, 2020, the DMC conducted a further review of the data, which led to the conclusion of no beneficial effect of hydroxychloroquine in patients hospitalized with COVID-19 and the decision to stop enrolling patients to the hydroxychloroquine arm of the trial. A total of 1542 patients were randomized to hydroxychloroquine compared with 3132 patients randomized to usual care alone. There was no significant difference in the primary endpoint of 28-day mortality (25.7% hydroxychloroquine vs. 23.5% usual care; hazard ratio 1.11 [95% confidence interval 0.98-1.26]; p=0.10). There was also no evidence of beneficial effects on hospital stay duration or other outcomes [8] .

The early termination of the hydroxychloroquine arm was based on data from over 4000 patients enrolled and over 1000 deaths observed in the hydroxychloroquine arm and the usual care arm combined. The 28-day mortality rate was numerically higher in the hydroxychloroquine arm, and the lower limit of the 95% confidence interval for the hazard ratio convincingly ruled out any meaningful mortality benefit of hydroxychloroquine in patients hospitalized with COVID-19. More details on the interim analyses conducted up to the enrollment termination would shed light on whether enrollment to the hydroxychloroquine arm could have been stopped earlier so that more patients could have been allocated to the other treatment arms.

Futility stopping in COVID-19 trials deserves special attention from the global clinical trials community. Trial sponsors should consider establishing futility boundaries for frequent interim analyses based on well-established statistical methodologies. Institutional Review Boards should carefully assess the futility stopping criteria and the frequency of interim analyses during the protocol review process. DMCs should consider making futility recommendations based on the totality of the interim data irrespective of the futility boundaries specified in the trial protocol. In addition, DMCs should consider reviewing interim data more frequently than specified in the protocol, and consider adding additional futility analyses when the results at a pre-specified interim analysis are trending towards futility instead of waiting for the next pre-specified analysis. The tradeoff between the cost of futility interim monitoring, which is a slight loss of study power, and allocating limited resources to more promising experimental treatments should be carefully considered by the study sponsor, the IRB and the DMC in the pursuit of safe and effective treatments for COVID-19.

Futility stopping in clinical trials

Hybridization of conditional and predictive power for futility assessment in sequential clinical trials with time-to-event outcomes: a resampling approach

A randomized trial of hydroxychloroquine as postexposure prophylaxis for Covid-19

Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis