When do we need clinical endpoint adjudication in clinical trials? ARTICLE When do we need clinical endpoint adjudication in clinical trials? Claes Held Uppsala Clinical Research Center, Department of Medical Sciences, Cardiology, Uppsala, Sweden ABSTRACT Clinical endpoint adjudication (CEA) is a standardized process for assessment of safety and efficacy of pharmacologic or device therapies in clinical trials. CEA plays a key role in many large clinical trials with the aim of achieving consistency and accuracy of the study results, by applying independent and blinded evaluation of suspected clinical events reported by investigators. However, due to high costs there are different opinions regarding the use of central adjudication versus more simplified strategies or site-based assessments and whether the final results differ significantly. There is a lack of scientific evaluation of different adjudication strategies, and more knowledge is needed on the optimal adjudi- cation process and how to achieve the best cost-effectiveness. New methodologies using national registry data and artificial intelligence may challenge the traditional adjudication strategy and could potentially reduce cost considerably with a similar result. Further research and evidence in this field of clinical trials methodology are essential. ARTICLE HISTORY Received 9 July 2018 Revised 22 August 2018 Accepted 23 August 2018 KEYWORDS Adjudication; clinical trials; endpoints Introduction Clinical endpoint adjudication (CEA) is a standardized process for the assessment of safety and efficacy of pharmacologic or device therapies in clinical trials. Central adjudication plays a key role in many large clinical trials with the aim of achiev- ing consistent, accurate, independent, unbiased, and blinded evaluation of suspected clinical events reported by investiga- tors. However, there is a controversy regarding the use of central adjudication committees versus simplified strategies or site-based assessments, and, thus, the value has been debated (1,2). Arguments have been made that the final results do not differ from the investigator judgement and that it is a costly process. Despite a long history of central- ized adjudication of both cardiovascular (CV) and non-CV endpoint events, there is a need for the clinical trials com- munity to establish a set of best practices to inform how CEA committees should be structured and operated. For a long time there has not been a standardized and commonly accepted CEA process in the scientific community, and there is a need for standardization of the CEA methodology. In addition, historically the endpoint definitions have been inconsistent between studies, which has been a challenge when comparing results between studies. One example is the definition of bleeding, where there are several sets of definitions available. Often, a single study includes several bleeding definitions to adjust for this challenge. However, recently an international expert group, the Bleeding Academic Research Consortium (BARC), developed the BARC standardized bleeding criteria which are now widely accepted (3). Another important endpoint in CV outcome trials is the definition of myocardial infarction (MI), which has been updated several times by the Universal Definitions expert group (4), and a new updated definition is awaited shortly. Recently, an expert committee, led by the Food and Drug Administration (FDA), published a joint suggestion on endpoint definitions of the most important cardiovascular events (5), which is now becoming a standard. For device tri- als the Academic Research Consortium (ARC) just published suggestions for revised endpoint definitions, such as peri- procedural MIs and stent thrombosis in coronary device tri- als (6). Previous studies have shown that the rate of MI assessed by standardized adjudication or by investigator-reported results differed significantly (7). Differential ways of identify- ing endpoints, such as using triggered events or screening of laboratory data or ECG for MI, could possibly increase the detection of MI (8) or bleedings (9). At Uppsala Clinical Research Center (UCR), we have offered CEA services for clinical trials for 10 years and have now substantial experience of adjudication of outcomes, from small academic studies to registry-based randomized clinical trials and large big pharma phase III trials for registra- tion purposes. This review aims to provide an overview and highlight the advantages but also the challenges of perform- ing centralized CEA and to provide some future perspectives for the role of central adjudication in future clinical trials. Selection of endpoints When a prospective clinical trial is planned, the primary and secondary outcomes are usually defined as both a composite CONTACT Claes Held claes.held@ucr.uu.se Uppsala Clinical Research Center, Department of Medical Sciences, Uppsala, 75185 Sweden � 2018 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. UPSALA JOURNAL OF MEDICAL SCIENCES 2019, VOL. 124, NO. 1, 42–45 https://doi.org/10.1080/03009734.2018.1516706 http://crossmark.crossref.org/dialog/?doi=10.1080/03009734.2018.1516706&domain=pdf http://orcid.org/0000-0001-9402-7404 http://creativecommons.org/licenses/by/4.0/ https://doi.org./10.1080/03009734.2018.1516706 http://www.tandfonline.com of clinically relevant outcomes, such as CV death, myocardial infarction (MI), and stroke, and its separate components. In studies on antithrombotic or anticoagulant therapy, safety endpoints are mandatory, of which bleeding complication is an important and common endpoint. When total mortality is included, subcategories such as CV death and non-CV death (infection, cancer, or other organ-specific endpoints) are often being evaluated. Subcategories of CV endpoints, such as non-ST elevation MI versus ST-elevation MI (NSTEMI versus STEMI) or type 1–5 MI, or ischemic versus hemorrhagic stroke, are used for subgroup analyses. Despite the fact that clinical endpoint adjudication has been applied for several decades, the evidence is surprisingly scarce on what end- point definitions should be used and how to capture the information. Updates and revisions are necessary since the diagnostic tools such as biomarkers and new technology with highly sensitive methods develop with time. The CEA process The CEA process is believed to enhance the validity of clin- ical trial CV outcome measures through independent, sys- tematic, and standardized identification, processing, and review of CV events. However, little is published that critic- ally and consistently reports on the specifics of CEA organiza- tion and the process methodology, and there is no gold standard. The CEA charter is the document that regulates all aspects of the study specifics regarding CEA and to which there should be a reference in the study protocol. Here, the CEA committee is specified, usually composed of clinical spe- cialists in the respective fields, such as cardiology, stroke medicine/neurology, or nephrology. The endpoint office (EPO) collects clinically relevant documents, such as hospital notes, relevant ECGs, lab reports, CT reports, and angiog- raphy documentation. The EPO is usually composed of CEA coordinators, monitors, and assistants. The coordinator leads the daily work together with the CEA chair or co-chair. Collection of predefined source documents has always been challenging in large studies with a high number of endpoints. It used to be paper or fax documents, but with modern tech- nology commercial software is now available that enables the CEA to be carried out completely electronically. The EPO cre- ates PDF files with the relevant endpoint information and sub- mits the package to the reviewers after careful review for completeness (Figure 1). There is no evidence-based scientific evidence on how the flow of events and review process should be handled. However, the most common strategy used in the large global CEA centers is that two independent reviewers evaluate each potential event. If there is agreement on all aspects, the event is complete. In case of disagreement, a pro- cess of resolving this is initiated. Depending on the level of dis- agreement, it may be a committee meeting with several senior specialists coming to a consensus decision or that the two ini- tial reviewers come to a mutual agreement. Collection of the required source documents should be specified for each type of endpoint. This should be handled by a query process at each site in the study. When this is completed a package of all documents is created as a PDF file. This also often includes some pre-selected relevant data from the eCRF (often called patient profile). Registry-based studies During the last few years, registry-based randomized controlled studies (RRCT) in cardiology have become a new exciting con- cept in clinical research, using quality improvement registries for prospective randomized trials (10). Such studies are charac- terized by pragmatism, simple design, and non-complex out- come measures, such as total mortality. Endpoint detection through the use of public registries (as used in RRCTs) can be employed in randomized clinical trials instead of the more trad- itional approach with active screening and central adjudication (10–12). There are several advantages with endpoint detection through public registries, including a rapid inclusion rate, a more representative all-comer population, as well as substan- tially lower costs compared to a traditional randomized clinical trial (RCT) (13). Furthermore, official healthcare registries collect a much larger array of possible outcomes. However, registry fol- low-up is largely based on ICD codes that are dependent on the judgement of the local hospital physician and not on a sys- tematic review of study-specific event definitions. ICD codes are often fairly crude and do not catch granularities around an event, such as sub-categories of a death or an MI. Thus, they may sometimes be inaccurate, and there is also a risk of under- reporting of events in registries, such as type 2 MIs or bleed- ings. A standard RCT may thus offer a higher degree of accuracy regarding a limited number of pre-specified clinical endpoints compared to an RRCT. During the last year two RRCTs were completed in which endpoint adjudication was applied in the ascertainment of endpoints (14,15). The reason was that primary and secondary endpoints, such as bleedings and unplanned revascularization, were slightly more complex and difficult to accurately collect in standard registries. The accuracy of clinical endpoint detection using registries only, as compared to active screening, follow-up, and clinical adjudica- tion (as employed in RCTs), is currently unknown and under investigation (15). Published data from Danish registries com- paring hospital medical records with and without central adju- dication showed modest agreement on an individual patient basis, but the estimated intervention effects were similar. However, the study addressed the issue of central adjudication or its absence and did not compare active patient follow-up (as in RCTs) compared to using registries only (16). Future developments There is a great need for research in the field of CEA, from evaluation of concordance between investigators and adjudi- cation, endpoint definitions, and the optimal CEA process, to whether other ways of endpoint ascertainment can be applied, such as using ICD codes from registries. We have to compare the quality of data and concordance with the newer technologies. Technical innovations are also needed with better adjudication software, user-friendly both from the perspective of a reviewer and from the administrative side. Better and more detailed reports that can be used for UPSALA JOURNAL OF MEDICAL SCIENCES 43 monitoring of the study progress are another important aspect of improvement. Finally, electronic health records and registries now offer the potential ability to accelerate evidence development, and applying artificial intelligence methodology in the adjudica- tion of clinical endpoints and is a scenario that might become reality in the near future. Hopefully, that possibly could increase the efficacy and considerably reduce the costs. Conclusions Central adjudication of endpoints, with the aim of achieving consistent and accurate evaluation of suspected clinical events reported by investigators, is currently the gold standard in clin- ical trials. More knowledge is needed on the optimal adjudica- tion process and how to achieve the best cost-effectiveness. New methods using national registry data and artificial intelli- gence may challenge the traditional adjudication strategy and could potentially reduce cost considerably, with similar results. About the contributor Claes Held is Professor of Cardiology at Clinical Research Center, Uppsala and is serving as a faculty member. He is the Chair of the Clinical Endpoint Adjudication committee. Disclosure statement The author reports no conflicts of interest. ORCID Claes Held http://orcid.org/0000-0001-9402-7404 References 1. Granger CB, Vogel V, Cummings SR, Held P, Fiedorek F, Lawrence M, et al. Do we need to adjudicate major clinical events? Clin Trials. 2008;5:56–60. 2. Sepehrvand N, Zheng Y, Armstrong PW, Welsh R, Goodman SG, Tymchak W, et al. Alignment of site versus adjudication commit- tee-based diagnosis with patient outcomes: insights from the pro- viding rapid out of hospital acute cardiovascular treatment 3 trial. Clin Trials. 2016;13:140–8. 3. Mehran R, Rao SV, Bhatt DL, Gibson CM, Caixeta A, Eikelboom J, et al. Standardized bleeding definitions for cardiovascular clinical trials: a consensus report from the Bleeding Academic Research Consortium. Circulation 2011;123:2736–47. 4. Thygesen K, Alpert JS, Jaffe AS, Simoons ML, Chaitman BR, White HD, et al. Third universal definition of myocardial infarction. Eur Heart J. 2012;33:2551–67. 5. Hicks KA, Mahaffey KW, Mehran R, Nissen SE, Wiviott SD, Dunn B, et al. 2017 Cardiovascular and stroke endpoint definitions for clin- ical trials. J Am Coll Cardiol. 2018;71:1021–34. 6. Garcia-Garcia HM, McFadden EP, Farb A, Mehran R, Stone GW, Spertus J, et al. Standardized end point definitions for coronary intervention trials: the Academic Research Consortium-2 consen- sus document. Eur Heart J. 2018;39:2192–207. 7. Mahaffey KW, Harrington RA, Akkerhuis M, Kleiman NS, Berdan LG, Crenshaw BS, et al. Disagreements between central clinical events committee and site investigator assessments of myocardial infarc- tion endpoints in an international clinical trial: review of the PURSUIT study. Curr Control Trials Cardiovasc Med 2001;2:187–94. 8. Mahaffey KW, Held C, Wojdyla DM, James SK, Katus HA, Husted S, et al. Ticagrelor effects on myocardial infarction and the impact of event adjudication in the PLATO (Platelet Inhibition and Patient Outcomes) trial. J Am Coll Cardiol. 2014;63:1493–9. 9. Kosmidou I, Mintz GS, Jatene T, Embacher M, Redfors B, Wong SC, et al. Impact of bleeding assessment and adjudication method- ology on event rates and clinical trial outcomes: Insights from the HORIZONS-AMI trial. EuroIntervention 2018;14:e580–7. 10. Frobert O, Lagerqvist B, Olivecrona GK, Omerovic E, Gudnason T, Maeng M, et al. Thrombus aspiration during ST-segment elevation myocardial infarction. N Engl J Med. 2013;369:1587–97. 11. Frobert O, Lagerqvist B, Gudnason T, Thuesen L, Svensson R, Olivecrona GK, et al. Thrombus aspiration in ST-elevation myocar- dial infarction in Scandinavia (TASTE trial). A multicenter, prospect- ive, randomized, controlled clinical registry trial based on the Swedish angiography and angioplasty registry (SCAAR) platform. Study design and rationale. Am Heart J. 2010;160:1042–8. 12. Erlinge D, Koul S, Eriksson P, Scherst�en F, Omerovic E, Linder R, et al. Bivalirudin versus heparin in non-ST and ST-segment eleva- tion myocardial infarction–a registry-based randomized clinical trial in the SWEDEHEART registry (the VALIDATE-SWEDEHEART trial). Am Heart J. 2016;175:36–46. Site records new clinical event Collection of source documents CEC office data check Event package - final review by CEA coordinator Event package submitted to two physicians for review Incomplete source data Adjudication Reviewer 1 Adjudication Reviewer 2 Disagreements resolved at Phase II meeting or other solution Database Completed events Figure 1. Flow chart of the CEA process. 44 C. HELD 13. James S, Frobert O, Lagerqvist B. Cardiovascular registries: a novel platform for randomised clinical trials. Heart. 2012;98: 1329–31. 14. Gotberg M, Christiansen EH, Gudmundsdottir IJ, Sandhall L, Danielewicz M, Jakobsen L, et al. Instantaneous wave-free ratio versus fractional flow reserve to guide PCI. N Engl J Med. 2017; 376:1813–23. 15. Erlinge D, Omerovic E, Frobert O, Linder R, Danielewicz M, Hamid M, et al. Bivalirudin versus heparin monotherapy in myocardial infarction. N Engl J Med. 2017;377:1132–42. 16. Kjoller E, Hilden J, Winkel P, Galatius S, Frandsen NJ, Jensen GB, et al. Agreement between public register and adjudication com- mittee outcome in a cardiovascular randomized clinical trial. Am Heart J. 2014;168:197–204. e1–4. UPSALA JOURNAL OF MEDICAL SCIENCES 45 Abstract Introduction Selection of endpoints The CEA process Registry-based studies Future developments Conclusions About the contributor Disclosure statement References