Which SARS-CoV-2 tests should be used?

by Pierre Nathie

Case Study

January 27, 2021

Which SARS-CoV-2 tests should be used?

Summary
- Over the course of 2020, more than 400 COVID-19/ SARS-CoV-2 tests have been released to the market, many with scant regulatory review e.g. via FDA/EUA of CE Self-certification (IVDD).
- To date, 46 of these tests have been independently, publically and reliably reviewed (e.g. CDC, FINDdx…)
- One out of four (26%) SARS-2/Covid-19 test on the market is less reliable than a coin toss i.e. their Positive Predictive Value is lower than 50%, based on these independent reviews.

Commentary:
The use of some low-specificity Rapid-Diagnostic-Tests results in a significant number of False-Positive results, triggering potentially unnecessary or heavy-handed Public Health interventions.
For example: Santa Clara (USA/CA, a local study reported 14% SARS2 prevalence i.e. 50 times the PCR-based prevalence (Published in Nature)
Knowledge is key to curb this pandemic!

1 - Published accuracy of SARS-CoV-2 tests

Methodology:

  • Inclusion criteria:
    The sensitivity and specificity of commercially available SARS-CoV-2 tests were documented from independent publications (listed in Appendix 1). They reflect the “real world” analytical performance of the tests rather than the performance printed by the manufacturers.
  • Prevalence levels:
    The PPVs and NPVs were computed at two hypothesised prevalence levels of 5% and 15%.
    • 5% SARS-CoV-2 prevalence is typical of most countries, including Australia
    • 15% SARS-CoV-2 prevalence has been reported in highly infected areas such as New York City at the pic of the infection waves.
    The computed values are listed in Table 1 of Appendix 1.


Abbreviations:

  • Lab: Pathology laboratory
  • POCT: Point-Of-Care Testing
  • NAT: Nucleic Acid Test (DNA- or RNA-based)
Covid-19 illustration 1
Covid-19 illustration 2


Positive Predictive Values of commercial SARS-CoV-2 tests at 5% prevalence graphic
Positive Predictive Values of commercial SARS-CoV-2 tests at 5% prevalence graphic
Graph #1: Positive Predictive Values of commercial SARS-CoV-2 tests at 5% prevalence


2 - Discussion

  • Antibodies tests and Nucleic Acid Amplification Tests (NAATs) are not substitutable. They detect different biomarkers that are expressed at different steps of the infection.
Evolution of SARS-CoV-2 biomarkers over time graphic
Graph #2: Evolution of SARS-CoV-2 biomarkers over time


  • A number of commercially available serological, based on the detection of IgG and IgM antibodies deliver, a Positive Predictive Value (PPV) no better than tossing a coin!


  • These low PPVs, in the context of SARS-CoV-2, are a frequent with Point-Of-Care antibodies tests. Such tests are “lateral flow tests”, also known as lateral flow immunochromatographic assays.
Covid-19 illustration 3


  • 56% of the independently-valuated Point-Of-Care SARS-CoV-2 tests fall below a 75% PPV line and constitute a waste of public money.1 They also result in misled Public Health assumptions and decisions as illustrated in Santa Clara CA Study that reported a 14% SARS-CoV-2 prevalence i.e. 50 X the PCR-based prevalence2.
    (*a 75% PPV means that 25% of positive results are falsely positive)


  • Lab-based test methods approve to be significantly more reliable, as only 14% independently-valuated Lab-CoV-2 tests fall below the 75% PPV line.


  • As supported by their “real world” performance, Nucleic Acid Amplification Tests (NAATs) deserve their title of “gold standard”.
    However they conserve that title provided that the targeted regions of the genome are conserved through the inevitable mutations of the virus.
Covid-19 illustration 3

  • However the accuracy of NAATs highly depends on the quality of preanalytical steps that are critical to the accuracy of the test. Preanalytical steps include the delicate extraction of the nucleic acids.
    Additionally, the entire system should be considered including the sampling swabs. Human factors are also influential.
Covid-19 illustration 3


  • Next generation Isothermal nucleic acid amplification tests such as RT-LAMP, Nicking Enzyme Amplification Reaction (NEAR) or CRISPR–Cas12-based lateral flow assays3 have the potential to be used at the point-of-care with sensitivity comparable to that of the current RT-qPCR tests used in pathology laboratories.
Covid-19 illustration 3




Appendix 1: SARS-CoV-2 tests analytical performance



SARS-CoV-2 Manufacturer Test Format Sensitivity Prevalence Low High
5% 15%
Specificity PPV low prev. NPV high prev. PPV low prev. NPV high prev.
AntibodiesEuroimmunSARS-COV-2 ELISA (IgG)Lab90.0%100.0%100.0%99.5%100.0%98.3%
AntibodiesMikrogenrecomWell SARS-CoV-2 IgG Lab86.4%100.0%100.0%99.3%100.0%97.7%
AntibodiesMount Sinai COVID-19 ELISA Antibody TestLab92.5%100.0%100.0%99.6%100.0%98.7%
AntibodiesOrthoAnti-SARS-CoV-2 IgG testLab90.0%100.0%100.0%99.5%100.0%98.3%
AntibodiesViramedSARS-CoV-2 Virachip IgGLab77.3%100.0%100.0%98.8%100.0%96.1%
AntibodiesRocheElecsys Anti-SARS-CoV-2Lab100.0%99.8%96.3%100.0%98.9%100.0%
AntibodiesAbbottArchitect SARS-CoV-2 IgGLab100.0%99.6%92.9%100.0%97.8%100.0%
AntibodiesBio-RadPlatelia SARS-CoV-2 Total AbLab92.2%99.6%92.4%99.6%97.6%98.6%
AntibodiesDiaSorinLIAISON SARS-CoV-2 S1/S2 IgGLab97.6%99.3%88.0%99.9%96.1%99.6%
AntibodiesAbbottAlinity i SARS-CoV-2 IgGLab100.0%99.0%84.0%100.0%94.6%100.0%
AntibodiesCDCSARS-CoV-2Lab96.0%99.0%83.5%99.8%94.4%99.3%
AntibodiesWadsworthSARS-CoV Microsphere Immunoassay for Antibody DetectionLab88.0%98.8%79.4%99.4%92.8%97.9%
AntibodiesEuroImmunIgA ElisaLab90.0%93.0%40.4%99.4%69.4%98.1%
AntibodiesEDICOVID-19 IgGLab100.0%88.7%31.8%100.0%61.0%100.0%
AntibodiesEpitopeElisaLab81.0%89.8%29.5%98.9%58.3%96.4%
AntibodiesSure-BioPOCT71.4%100.0%100.0%98.5%100.0%95.2%
AntibodiesWantaiTotal Ab ElisaPOCT93.0%100.0%100.0%99.6%100.0%98.8%
AntibodiesAutoBio DiagnosticsAnti-SARS-CoV-2 Rapid TestPOCT99.0%99.0%83.9%99.9%94.6%99.8%
AntibodiesWondfoPOCT81.0%99.1%82.6%99.0%94.1%96.7%
AntibodiesUCP BiosciencsPOCT71.4%98.1%66.4%98.5%86.9%95.1%
AntibodiesPremierPOCT81.0%97.2%60.3%99.0%83.6%96.7%
AntibodiesCellexqSARS-CoV-2 IgG/IgM Rapid TestPOCT93.8%96.0%55.2%99.7%80.5%98.9%
AntibodiesInnovitaPOCT64.3%96.3%47.8%98.1%75.4%93.9%
AntibodiesBioperfectusPOCT81.0%95.2%47.0%99.0%74.8%96.6%
AntibodiesChembioDPP Covid-19 IgM/IgG SystemPOCT93.5%94.4%46.8%99.6%74.7%98.8%
AntibodiesVivaDiagPOCT71.4%94.9%42.4%98.4%71.2%95.0%
AntibodiesDecomBioPOCT70.0%89.7%26.3%98.3%54.5%94.4%
AntibodiesLumosFebriDxPOCT93.0%86.0%25.9%99.6%54.0%98.6%
AntibodiesBiomedomicsPOCT81.0%86.9%24.5%98.9%52.2%96.3%
AntibodiesDeepBluePOCT81.0%84.3%21.3%98.8%47.6%96.2%
AntigenAbbottBINAXNOW™ COVID-19 Ag CARDPOCT93.3%99.9%98.0%99.6%99.4%98.8%
AntigenAbbottPANBIO™ COVID-19 Ag RAPID TESTPOCT86.2%99.9%97.8%99.3%99.3%97.6%
AntigenBioNoteBiocredit One Step SARSCoV-2 AntigenPOCT63.2%99.5%86.4%98.1%95.5%93.9%
AntigenCorisStandard Q COVID-19 AgPOCT84.8%98.9%79.7%99.2%93.0%97.4%
AntigenRapigenNowCheck COVID-19 Antigen TestPOCT89.2%97.3%63.5%99.4%85.4%98.1%
AntigenRoche(SD Biosensor)COVID-19 Ag Respi-StripPOCT50.0%95.8%38.5%97.3%67.8%91.6%
RNAbioMérieux SABioFire COVID-19 TestLab100.0%100.0%100.0%100.0%100.0%100.0%
RNAbioMérieux SASARS-COV-2 R-GENE® (ref 423717)Lab100.0%100.0%100.0%100.0%100.0%100.0%
RNADiaSorin Molecular, LLCSimplexa? COVID-19 Direct RT-PCR KitLab100.0%100.0%100.0%100.0%100.0%100.0%
RNAHologicPanther Fusion SARS-CoV-2 assayLab94.1%100.0%100.0%99.7%100.0%99.0%
RNARoche LtdCoronavirus LightMix® Modular SARS and Wuhan CoV E-gene assayLab100.0%100.0%100.0%100.0%100.0%100.0%
RNARoche Molecular Diagnosticscobas® SARS-CoV-2 (for use on the cobas® 6800/8800 Systems)Lab96.2%100.0%100.0%99.8%100.0%99.3%
RNAThermo Fisher ScientificTaqMan? 2019-nCoV Assay Kit v1Lab100.0%100.0%100.0%100.0%100.0%100.0%
RNAThermo Fisher ScientificTaqPath COVID-19 Combo KitLab93.8%100.0%100.0%99.7%100.0%98.9%
RNAAbbott Diagnostics Inc.ID NOW COVID-19POCT87.7%100.0%100.0%99.4%100.0%97.9%
RNACepheidXpert Xpress SARS-CoV-2POCT99.4%100.0%100.0%100.0%100.0%99.9%
Table 1: Sensitivity, specificity NPV and PPV for various commercial SARS-CoV-2 tests

Table 1: Sensitivity, specificity NPV and PPV for various commercial SARS-CoV-2 tests


Independent sources:
  • FDA4
  • FIND5
  • Science Direct6
  • medRxiv7