key: cord-349782-djzxkus2
authors: van Kasteren, Puck B.; van der Veer, Bas; Reusken, Chantal B.E.M.; Meijer, Adam
title: Response to letter of concern by Oladimeji and Pickford of PrimerDesign
date: 2020-06-29
journal: J Clin Virol
DOI: 10.1016/j.jcv.2020.104526
sha: 
doc_id: 349782
cord_uid: djzxkus2

nan

a National Institute for Public Health and the Environment (RIVM) Center for Infectious Disease Control (CIb) Bilthoven, the Netherlands 24 June 2020

Over the past months, our experienced team of technicians has worked tirelessly to provide diagnostic support to our national healthcare community. On top of this, they have dedicated their time to providing the global diagnostic community with a much needed initial comparison of commercially available COVID-19 RT-PCR kits, which was recently published in the Journal of Clinical Virology (1).

We strongly object to the suggestion that our work was methodologically flawed, but thank Dr.

Oladimeji and Dr. Pickford of PrimerDesign (2) for providing us here with the opportunity to add some additional clarifying notes to our study.

Similar to the FIND initiative (3), although less in number, we have used a selection of archived clinical samples to obtain retrospectively an indication of the diagnostic performance of the evaluated kits (we used the terms clinical sensitivity and specificity to express the diagnostic performance with clinical samples). Our study was performed under strictly controlled circumstances to exclude influence of confounding factors on the results. RNA was extracted anew from the selected archived samples simultaneously, aliquoted, and stored at -20°C for a maximum of 4 weeks. In our extraction process we add yeast tRNA to stabilize the extracted RNA. A fresh aliquot of this single batch was thawed for each run of RT-PCR kits and our in-house reference assay, thus guaranteeing inter-assay equivalency. In case a process control for the PCR component of the kit was included, this was measured. Without exception these provided a positive result in all samples, in the case of PrimerDesign ranging between Ct 23.26-25.17. Thus indicating that the call of a negative result in samples that were in fact positive was not due to PCR issues.

Obviously, our selection of 13 SARS-CoV-2-positive clinical samples does not reflect the variation encountered in the field, especially since we specifically included several samples with relatively high Ct values. Precisely to assess clinical performance with those viral loads where differences between kits become apparent. For this reason, we deliberately refrained from expressing the observed detection rate as a percentage, as this would imply that we had determined the precise diagnostic performance in clinical sensitivity, which we did not. However, in the discussion section we do speculate that the detection rate we observe would translate to a clinical sensitivity in the field of >96% for all kits, as only 3.6% of clinical samples in our database have a Ct value >34.5 with our in-J o u r n a l P r e -p r o o f house RT-PCR and all kits detected samples with Ct values <34.5 with our in-house RT-PCR. This finding is perfectly in line with the clinical sensitivity of 100% (95% confidence interval: 93-100) found for most of the assays including PrimerDesign by the FIND initiative (4) . Notably, the FIND initiative protocol (3) and the presented results for PrimerDesign and other kits (4) do not provide information regarding the composition of the 50 positive samples included in the study and no information on viral loads using e.g. reference Ct values. It is therefore unclear to what extent these FIND samples do provide a true reflection of the variation in the field and especially on viral loads that are only detected by highly sensitive tests such as our in-house test. If only samples from patients in the hospital setting that have relatively high viral loads have been selected, every kit will score a near 100% diagnostic clinical sensitivity. We have specifically sought for the clinical performance using samples that have a viral load around the LOD of the kits.

Regarding the preliminary limits of detection (LODs) determined in our study, we do not agree that there are major differences compared to what was found by the FIND initiative (4). We agree that our assessment is less precise than that by the FIND initiative due to the lower number of replicates tested, but the point estimate still provides comparable information. In the FIND study, the vast majority of assays fall in the highest banding of 1-10 copies/reaction (4). This banding allows for a 10-fold difference between kits. In our study we observe a mere 6-fold difference between the lowest and highest LODs, which is thus again in good agreement with the results of the FIND initiative. Our finding that the RT-PCR kit with the best analytical sensitivity (Altona) did not detect the clinical samples with the highest Ct values in our in-house assay, highlights the importance of determining both analytical sensitivity using quantified standards and diagnostic clinical sensitivity using samples representing the complete range of viral loads detected in clinical practice during kit validation. Considering the controlled set-up of our comparative study we strongly disagree that the comparative LODs we reported are unverified.

In conclusion, we performed this study to confirm that commercially available COVID-19 RT-PCR kits are suitable for use in the field, not to provide a ranking of various competitors in this currently exploding field. We were content that all of the kits included in our assessment passed the basic requirements and explicitly note that more elaborate clinical evaluation in different settings (e.g. 

Comparison of seven commercial RT-PCR diagnostic kits for COVID-19

Comparison of seven commercial RT-PCR Diagnostic kits for COVID-19

Comparative evaluation of molecular tests that directly detect the nucleic acid of SARS-CoV-2

FIND. SARS-COV-2 MOLECULAR ASSAY EVALUATION: RESULTS. Available at

National Institute for Public Health and the Environment (RIVM), Bilthoven, the Netherlands Declarations of interest: none
