The COVID-19 XPRIZE and the need for scalable, fast, and widespread testing
To the Editor — The US Food and Drug Administration (FDA) Emergency Use Authorization (EUA) and Instructions for Use (IFU) documents outlining the current approved virology tests for SARS-CoV-2 are largely unstandardized. As such, there remains an urgent need for a searchable interface allowing exploration of standardized information reported in these EUA and IFU documents. To gain an improved understanding of the current testing landscape and to galvanize future test development, we present here an online tool (http://www.resiliencehealth.com/tests) that profiles current and emerging virology tests for detecting SARS-CoV-2 (Figs. 1 and 2). We also call on the research community to respond to an open COVID-19 XPRIZE competition, OpenCovidScreen, seeking to identify cheap, high-quality, scalable testing solutions.
NGS, next-generation sequencing; NAAT, nucleic acid amplification test; LFA, lateral flow assay; RT, reverse transcription; ddPCR, Droplet Digital PCR; dPCR, digital PCR; qPCR, quantitative PCR.
a, Limit of detection (LoD) for tests that reported copies/μL or copies/mL (presented in copies/μL throughout). b, EUA virology test targets. SARS-CoV-2 5′–3′ genome4 on horizontal axis, with tests on the vertical axis and colors indicating whether the test has one target (red) or multiple targets (brown) in the specified region. Each line indicates a test source (company or institution). If an EUA reported different LoDs for different targets, sample types or methods, each is displayed as a separate row. NGS, next-generation sequencing; NAAT, nucleic acid amplification test; LFA, lateral flow assay; RT, reverse transcription; ddPCR, Droplet Digital PCR; dPCR, digital PCR; qPCR, quantitative PCR; RDRP, RNA-dependent RNA polymerase; DSP, diagnostic sample preparation; NP, nasopharyngeal; OP, oropharyngeal; rRT, real-time reverse transcriptase.
As of 27 July 2020, an analysis of the FDA data on EUA SARS-CoV-2 virology tests reveal a wide range of limit of detection (LoD), spanning >5 orders of log10 differences. These metrics are of critical importance because each 10-fold increase in the LoD of a COVID-19 viral diagnostic test is expected to increase the false negative rate by 13%1.
Beyond this variable performance reported in IFUs for EUA tests, key attributes of many tests, such as primer sequences, protocol steps or viral gene targets, are either unclear or missing. Also, most approved EUAs use large multipliers (2- to 200-fold) on their own LoD for contrived or clinical samples to pass the minimum threshold for approval, based on a 95% positive/negative percentage agreement across at least 30 positive and 30 negative samples. As submissions stand, it is difficult to directly compare results and even understand how a test will actually translate into real-world or clinical settings.
Moreover, there has not been an independent assessment of these tests’ abilities or a comprehensive benchmarking of their strengths and weaknesses in different clinical settings, nor a consistent sample type (for example, nasopharyngeal, nasal, saliva) used across the EUAs. Thus, a comprehensive benchmarking effort on all methods on the market would be helpful, similar to ones conducting head-to-head studies of serological tests2 and other sites that annotate and analyze some of these tests, such as FindDx (https://www.finddx.org/), the US National Institute for Standards and Technology (NIST)’s Rapid Microbial Testing Methods Consortium (https://www.nist.gov/programs-projects/nist-rapid-microbial-testing-methods-consortium) and the COVID-19 Testing Project (https://covidtestingproject.org/index.html).
However, even if all current EUA tests for the SARS-CoV-2 virus performed with >95% sensitivity and >95% specificity, their combined capacities would still fall short of enabling large-scale, ubiquitous temporal monitoring (tens of millions per day), involving samples with varying viral load and substrates3. Also, tests with a higher LoD would not be readily applicable to pooling strategies, in which samples are by definition diluted before testing. Even the lower LoD tests, while promising for pooling, have not had independent LoD assessments. Moreover, truly city-scale or even national-level testing to decrease and control infection rates would require fast turnaround times (test result in less than one day), easy processing, and a low cost (per test and capital expense), such that a consumer, employer or government body could easily pay for multiple tests a week for each person.
To address the above issues, we have designed the COVID-19 XPRIZE competition, OpenCovidScreen (https://opencovidscreen.org/), to identify economically viable, high-quality, scalable testing options (Fig. 3). After competitors are selected on the basis of their results, methods, cost, scalability and speed, they will then be sent blinded samples to analyze. Results will be uploaded to the XPRIZE site and analyzed for overall performance, LoD and false positives. The finalists, based on their overall methods, results and innovation, will go into the clinical validation round, in which OpenCovidScreen will follow the methods laid out by the competitors to assess the reproducibility of their results. Top teams from this validation round will be awarded a prize and selected to set up and deploy testing sites, wherein OpenCovidScreen will help scale their tests and expand them into more locations, as well as to coordinate with government, industry and non-profit efforts (for example, the NIH RADx Program and Testing for America; https://www.nih.gov/research-training/medical-research-initiatives/radx).
The OpenCovidScreen competition is intended to identify fast, scalable, and cheap SAR-CoV-2 virology tests through a series of phases, starting with submitting test and method information (top), a proficiency test including blinded samples, and a clinical validation phase (middle). The top competitors identified on the basis of their methods, results and scalability will then be chosen for deployment to enable widespread usage of their tests (bottom).
We invite readers to submit solutions for this XPRIZE, which is open to participants from all around the world (https://xprize.org/testing). This prize can serve as a springboard for both new and established technologies that can enable truly global viral testing and surveillance. Successful methods will depend on the availability of reagents, resources and automation, and as such, these metrics will also be used to identify the finalists. Winning methods will help regions increase testing capabilities by orders of magnitude and thus empower schools, businesses and cities to rapidly reopen safely, as well as pioneer technologies and platforms that can be used for future outbreaks. It is crucial to deploy rapid, scalable methods capable of tracking viruses to mitigate their detrimental impacts on society. All are encouraged to help in this fight.