Advertisement

Field evaluation of a rapid antigen test (Panbio™ COVID-19 Ag Rapid Test Device) for COVID-19 diagnosis in primary healthcare centres

Published:November 12, 2020DOI:https://doi.org/10.1016/j.cmi.2020.11.004

      Abstract

      Objectives

      To our knowledge no previous study has assessed the performance of a rapid antigen diagnostic immunoassay (RAD) conducted at the point of care (POC). We evaluated the Panbio™ COVID-19 Ag Rapid Test Device for diagnosis of coronavirus 2019 disease (COVID-19) in symptomatic patients (n = 412) attending primary healthcare centres.

      Methods

      RAD was performed immediately after sampling following the manufacturer's instructions (reading at 15 min). RT-PCRs were carried out within 24 h of specimen collection. Samples displaying discordant results were processed for culture in Vero E6 cells. Presence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in cell cultures was confirmed by RT-PCR.

      Results

      Out of 412 patients, 43 (10.4%) tested positive by RT-PCR and RAD, and 358 (86.9%) tested negative by both methods; discordant results (RT-PCR+/RAD–) were obtained in 11 patients (2.7%). Overall specificity and sensitivity of rapid antigen detection (RAD) was 100% (95%CI 98.7–100%) and 79.6% (95%CI 67.0–88.8%), respectively, taking RT-PCR as the reference. Overall RAD negative predictive value for an estimated prevalence of 5% and 10% was 99% (95%CI 97.4–99.6%) and 97.9% (95%CI 95.9–98.9), respectively. SARS-CoV-2 could not be cultured from specimens yielding RT-PCR+/RAD– results (n = 11).

      Conclusion

      The Panbio™ COVID-19 Ag Rapid Test Device performed well as a POC test for early diagnosis of COVID-19 in primary healthcare centres. More crucially, the data suggested that patients with RT-PCR-proven COVID-19 testing negative by RAD are unlikely to be infectious.

      Keywords

      Introduction

      Rapid detection, effective isolation of symptomatic cases, and systematic tracing of close contacts are paramount to blunt the community spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection. Nowadays, reverse-transcriptase polymerase chain reaction (RT-PCR) is the diagnostic reference standard for coronavirus disease 2019 (COVID-19) []; however, specialized instruments and expertise are required to conduct RT-PCR assays. In addition, many countries have encountered supply shortages of RT-PCR reagents. Rapid antigen detection immunoassays (RAD) are particularly suited for point-of-care testing (POCT), as they can easily be performed and interpreted without equipment, are inexpensive, and improve turnaround times. Moreover, results returned by a recently launched antigen assay appeared to correlate better with patient infectiousness than RT-PCR results [
      • Pekosz A.
      • Cooper C.
      • Parvu V.
      • Li M.
      • Andrews J.
      • Manabe Y.C.
      • et al.
      Antigen-based testing but not real-time PCR correlates with SARS-CoV-2 virus culture.
      ]. In this field study, we evaluated the Panbio™ COVID-19 Ag Rapid Test Device (Abbott Diagnostic GmbH, Jena, Germany), a lateral flow immunochromatographic assay targeting SARS-CoV-2 nucleoprotein in nasopharyngeal specimens (NPs) for the diagnosis of COVID-19 in symptomatic patients attending primary healthcare centres.

      Material and methods

       Patients

      Between 2nd September and 7th October 2020 this prospective study enrolled 412 patients with clinical suspicion of COVID-19 (median age 31 years, range 1–91 years, 58% female), of whom 327 were adults (median age 36 years, range 17–91 years) and 85 children (≤16 years old, median 11 years, range 1–16 years), attending primary care centres of the Clínico-Malvarrosa Health Department in Valencia (Spain). Only patients with compatible signs or symptoms appearing within the prior week were recruited. The study was approved by the Hospital Clínico de Valencia (HCU) INCLIVA Research Ethics Committee.

       SARS-CoV-2 testing

      Using flocked swabs, and following appropriate safety precautions, trained nurses at each participating centre collected two NPs per patient, one of which (provided by the manufacturer) was used for RAD while the other was placed in 3 mL of universal transport medium (UTM, Becton Dickinson, Sparks, MD, USA) and delivered to the HCU Microbiology Service for RT-PCR testing. RAD assay was performed immediately after sampling following the manufacturer's instructions (reading at 15 min). RT-PCRs were carried out within 24 h of specimen collection with the TaqPath COVID-19 Combo Kit (Thermo Fisher Scientific, MS, USA) which targets SARS-CoV-2 ORF1ab, N and S genes. RNA was extracted using the Applied Biosystems™ MagMAX™ Viral/Pathogen II Nucleic Acid Isolation Kits coupled with Thermo Scientific™ KingFisher Flex automated instrument. The AMPLIRUN® TOTAL SARS-CoV-2 Control (Vircell SA, Granada, Spain) was used as the reference material for SARS-CoV-2 RNA load quantification.

       SARS-CoV-2 cell culture

      Samples collected in UTM were stored at –80°C for up to 2 weeks before being processed for culture in Vero E6 cells. Presence of SARS-CoV-2 was confirmed by RT-PCR (see footnote to Supplementary Material Table S1).

       Statistical analyses

      Agreement between RAD assay and RT-PCR was assessed using Cohen's κ statistics. Differences between medians were compared using the Mann–Whitney U-test. Receiver operating characteristic (ROC) curves were built to determine SARS-CoV-2 RT-PCR cycle threshold (CT) and RNA loads best discriminating between RT-PCR+/RAD+ and RAD– specimens. Two-sided p values < 0.05 were considered significant. Statistical analyses were performed using SPSS version 25.0 (SPSS, Chicago, IL, USA).

      Results

      Out of 412 patients, 43 (10.4%) tested positive by RT-PCR and RAD, and 358 (86.9%) tested negative by both methods, showing discordant results (RT-PCR+/RAD–) in 11 patients (2.7%) (Supplementary Material Table S1). Concordance between the two methods was good (κ 0.87, 95%CI 0.79–0.94). Overall specificity and sensitivity of RAD was 100% (95%CI 98.7–100%) and 79.6% (95%CI 67.0–88.8%) respectively. Sensitivity slightly increased (80.4%, 95%CI 66.8–89.3%) in patients with clinical courses of <5 days (Fig. 1A). Sensitivity was higher in adults (82.6%, 95%CI 69.3–90.9%) than in paediatric patients (62.5%, 95%CI 30.6–86.3%) (Supplementary Material Table S2).
      Fig. 1
      Fig. 1(A) Field performance of the Panbio™ COVID-19 Ag Rapid Test Device (RAD) according to time between symptoms onset and testing (<or ≥5 days) in a cohort of symptomatic patients with clinical suspicion of COVID-19 attending primary healthcare centres. (B) RT-PCR CT values in specimens testing either RAD + or RAD–. (C) Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) RNA loads in specimens testing either RAD + or RAD–. The AMPLIRUN® TOTAL SARS-CoV-2 Control (Vircell SA, Granada, Spain) was used as the reference material for SARS-CoV-2 RNA load quantification (in copies/mL, considering RT-PCR CTs for the N gene; the linear regression equation was: Y = –0.31 × X + 13.77; R2 = 9.89). Median and p values are shown.
      Overall RAD negative predictive value for an estimated prevalence of 5% and 10% (the incidence of COVID-19 in our Health Department during the study period was within that range) was 99% (95%CI 97.4–99.6%) and 97.9% (95%CI 95.9–98.9), respectively.
      RT-PCR CT values were significantly higher and SARS-COV-2 RNA loads significantly lower (p < 0.001) in RT-PCR+/RAD– than in RT-PCR+/RAD + specimens (Fig. 1B,C). ROC curve analyses indicated that RT-PCR CT < 25 and SARS-CoV-2 RNA loads >5.9 log10 copies/mL thresholds best discriminated between RT-PCR+/RAD+ and RT-PCR+/RAD– specimens, with a sensitivity and specificity of 100%. As expected, the overall RAD sensitivity was directly dependent upon the RT-PCR CT values (SARS-CoV-2 RNA loads) (Supplementary Material Table S3).
      The time from symptoms onset to sampling did not differ (p 0.86) between RT-PCR+/RAD + patients (median 3 days, range 1–7 days) and RT-PCR+/RAD– patients (median 2 days, range 1–6 days).
      All 11 specimens yielding discordant RT-PCR/RAD results tested negative by culture, whereas SARS-CoV-2 could be recovered from all three specimens returning RT-PCR+/RAD + results (CT 4, 14 and 16).

      Discussion

      Previous studies evaluating SARS-CoV-2 RAD tests used either retrieved specimens, which had been cryopreserved a varying number of times, or ones conducted at central laboratories, or both [
      • Linares M.
      • Pérez-Tanoira R.
      • Romanyk J.
      • Pérez-García F.
      • Gómez-Herruz P.
      • Arroyo T.
      • et al.
      Panbio antigen rapid test is reliable to diagnose SARS-CoV-2 infection in the first 7 days after the onset of symptoms.
      ,
      • Young S.
      • Taylor S.N.
      • Cammarata C.L.
      • Varnado K.G.
      • Roger-Dalbert C.
      • Montano A.
      • et al.
      Clinical evaluation of BD Veritor SARS-CoV-2 point-of-care test performance compared to PCR-based testing and versus the Sofia 2 SARS Antigen point-of-care test.
      ,
      • Scohy A.
      • Anantharajah A.
      • Bodéus M.
      • Kabamba-Mukadi B.
      • Verroken A.
      • Rodriguez-Villalobos H.
      Low performance of rapid antigen detection test as frontline testing for COVID-19 diagnosis.
      ,
      • Lambert-Niclot S.
      • Cuffel A.
      • Le Pape S.
      • Vauloup-Fellous C.
      • Morand-Joubert L.
      • Roque-Afonso A.M.
      • et al.
      Evaluation of a rapid diagnostic assay for detection of SARS-CoV-2 antigen in nasopharyngeal swabs.
      ,
      • Mak G.C.
      • Cheng P.K.
      • Lau S.S.
      • Wong K.K.
      • Lau C.S.
      • Lam E.T.
      • et al.
      Evaluation of rapid antigen test for detection of SARS-CoV-2 virus.
      ]. To our knowledge this is the first report on the performance of a RAD assay conducted at POC. Compared to RT-PCR, the Panbio™ COVID-19 Ag Rapid Test Device assay yielded an excellent specificity and a fairly good overall sensitivity (79.6%, 95%CI 67.0–88.8%), the latter slightly improved when time to testing was less than 5 days from the onset of symptoms (80.6%, 95%CI 66.8–89.3%). This figure is less impressive than is claimed by the manufacturer (93%). Based upon our findings (Supplementary Material Table S2), the sample panel evaluated by the manufacturer might have included a large fraction of specimens displaying high viral loads. Nevertheless, the overall sensitivity for the RAD assay reported herein was much closer to that found by Linares and colleagues (86.5%) [
      • Linares M.
      • Pérez-Tanoira R.
      • Romanyk J.
      • Pérez-García F.
      • Gómez-Herruz P.
      • Arroyo T.
      • et al.
      Panbio antigen rapid test is reliable to diagnose SARS-CoV-2 infection in the first 7 days after the onset of symptoms.
      ]. Sensitivity of SARS-CoV-2 RAD assays has been reported to vary between 45% and 97% [
      • Linares M.
      • Pérez-Tanoira R.
      • Romanyk J.
      • Pérez-García F.
      • Gómez-Herruz P.
      • Arroyo T.
      • et al.
      Panbio antigen rapid test is reliable to diagnose SARS-CoV-2 infection in the first 7 days after the onset of symptoms.
      ,
      • Young S.
      • Taylor S.N.
      • Cammarata C.L.
      • Varnado K.G.
      • Roger-Dalbert C.
      • Montano A.
      • et al.
      Clinical evaluation of BD Veritor SARS-CoV-2 point-of-care test performance compared to PCR-based testing and versus the Sofia 2 SARS Antigen point-of-care test.
      ,
      • Scohy A.
      • Anantharajah A.
      • Bodéus M.
      • Kabamba-Mukadi B.
      • Verroken A.
      • Rodriguez-Villalobos H.
      Low performance of rapid antigen detection test as frontline testing for COVID-19 diagnosis.
      ,
      • Lambert-Niclot S.
      • Cuffel A.
      • Le Pape S.
      • Vauloup-Fellous C.
      • Morand-Joubert L.
      • Roque-Afonso A.M.
      • et al.
      Evaluation of a rapid diagnostic assay for detection of SARS-CoV-2 antigen in nasopharyngeal swabs.
      ,
      • Mak G.C.
      • Cheng P.K.
      • Lau S.S.
      • Wong K.K.
      • Lau C.S.
      • Lam E.T.
      • et al.
      Evaluation of rapid antigen test for detection of SARS-CoV-2 virus.
      ], yet direct comparison between studies is hampered by marked dissimilarities in patient clinical characteristics and age, testing sites, type of specimen processed, and time to testing, among others.
      Interestingly, sensitivity was higher in adults than in paediatric patients. Previous studies found no age-related differences in SARS-CoV-2 RNA load in the upper respiratory tract [
      • Baggio S.
      • L'Huillier A.G.
      • Yerly S.
      • Bellon M.
      • Wagner N.
      • Rohr M.
      • et al.
      SARS-CoV-2 viral load in the upper respiratory tract of children and adults with early acute COVID-19.
      ]. Although speculative, dating of symptoms onset could have been more inaccurate in children than in adults.
      In a setting like ours, with an incidence of COVID-19 ranging between 5% and 10% at the time of study, the RAD negative predictive value was 99% (95%CI 97.4–99.6%) and 97.9% (95%CI 95.9–98.9), respectively.
      Out of 54 RT-PCR-positive specimens, 11 tested negative by RAD. In line with previous reports [
      • Pekosz A.
      • Cooper C.
      • Parvu V.
      • Li M.
      • Andrews J.
      • Manabe Y.C.
      • et al.
      Antigen-based testing but not real-time PCR correlates with SARS-CoV-2 virus culture.
      ,
      • Linares M.
      • Pérez-Tanoira R.
      • Romanyk J.
      • Pérez-García F.
      • Gómez-Herruz P.
      • Arroyo T.
      • et al.
      Panbio antigen rapid test is reliable to diagnose SARS-CoV-2 infection in the first 7 days after the onset of symptoms.
      ,
      • Young S.
      • Taylor S.N.
      • Cammarata C.L.
      • Varnado K.G.
      • Roger-Dalbert C.
      • Montano A.
      • et al.
      Clinical evaluation of BD Veritor SARS-CoV-2 point-of-care test performance compared to PCR-based testing and versus the Sofia 2 SARS Antigen point-of-care test.
      ], SARS-CoV-2 RNA load was significantly higher in RT-PCR+/RAD + specimens than in RT-PCR+/RAD– samples. In our setting, specimens with RT-PCR CT > 25 (equivalent to SARS-CoV-2 RNA loads <5.9 log10 copies/mL) returned discordant RAD/RT-PCR results.
      An important observation of our study was that SARS-CoV-2 could not be cultured from RT-PCR+ (CT > 25)/RAD– specimens. Along these lines, Pekosz and colleagues [
      • Pekosz A.
      • Cooper C.
      • Parvu V.
      • Li M.
      • Andrews J.
      • Manabe Y.C.
      • et al.
      Antigen-based testing but not real-time PCR correlates with SARS-CoV-2 virus culture.
      ] found one out of 27 RAD–/culture-positive specimens, using a highly sensitive cell culture system (VeroE6 TMPRSS2). The SARS-CoV-2 RNA load threshold associated with culture positivity herein (>5.9 log10 copies/mL) was remarkably close to other previously published results (around 106 copies/mL) [
      • Pekosz A.
      • Cooper C.
      • Parvu V.
      • Li M.
      • Andrews J.
      • Manabe Y.C.
      • et al.
      Antigen-based testing but not real-time PCR correlates with SARS-CoV-2 virus culture.
      ,
      • Wölfel R.
      • Corman V.M.
      • Guggemos W.
      • Seilmaier M.
      • Zange S.
      • Müller M.A.
      • et al.
      Virological assessment of hospitalized patients with COVID-2019.
      ,
      • Huang C.G.
      • Lee K.M.
      • Hsiao M.J.
      • Yang S.L.
      • Huang P.N.
      • Gong Y.N.
      • et al.
      Culture-based virus isolation to evaluate potential infectivity of clinical specimens tested for COVID-19.
      ,
      • Bullard J.
      • Dust K.
      • Funk D.
      • Strong J.E.
      • Alexander D.
      • Garnett L.
      • et al.
      Predicting infectious severe acute respiratory syndrome coronavirus 2 from diagnostic samples.
      ]. Other studies have reported positive cultures from upper respiratory tract specimens displaying RT-PCR CT values > 30 or even higher [
      • Huang C.G.
      • Lee K.M.
      • Hsiao M.J.
      • Yang S.L.
      • Huang P.N.
      • Gong Y.N.
      • et al.
      Culture-based virus isolation to evaluate potential infectivity of clinical specimens tested for COVID-19.
      ]. In this sense, it should be stressed that CT values returned by different RT-PCR assays for a given specimen, even among those targeting the same gene region, may vary substantially [
      • Pekosz A.
      • Cooper C.
      • Parvu V.
      • Li M.
      • Andrews J.
      • Manabe Y.C.
      • et al.
      Antigen-based testing but not real-time PCR correlates with SARS-CoV-2 virus culture.
      ]. In line with this, the lowest genome copy number that allowed SARS-CoV-2 recovery in cell culture in Huang's study [
      • Huang C.G.
      • Lee K.M.
      • Hsiao M.J.
      • Yang S.L.
      • Huang P.N.
      • Gong Y.N.
      • et al.
      Culture-based virus isolation to evaluate potential infectivity of clinical specimens tested for COVID-19.
      ] was 5.7 log10/mL, compared with 5.9 log10/mL in ours, in both the N gene being taken as the reference for SARS-CoV-2 RNA quantification.
      The main limitation of the current study is the relatively low number of cases in the series (13%); however, this could be viewed as a strength, as this figure likely represents the reality in many community settings worldwide where RAD testing is increasingly being used.
      In summary, we found the Panbio™ COVID-19 Ag Rapid Test Device to perform well as a POCT for early diagnosis of COVID-19 in primary healthcare centres. More crucially, our data suggest that patients with RT-PCR-proven COVID-19 testing negative by RAD are unlikely to be infectious. Given that false-negative RAD results may be inconsequential from a public health perspective [
      • Mina M.J.
      • Parker R.
      • Larremore D.B.
      Rethinking Covid-19 test sensitivity—a strategy for containment.
      ], a laboratory diagnostic approach that skipped RT-PCR confirmation of negative RAD tests in non-hospitalized patients would certainly alleviate laboratory workloads while RT-PCR tests are in short supply.

      Author contributions

      EA, IT, FB, DH, MM, SP, LF, AV, CSdA, JP and JC: methodology (RT-PCR and RAD) and data validation. EM and MA F–F: methodology (cell culture) and data validation. EA, IT: formal analysis. DN: conceptualization, supervision, writing the original draft. All authors reviewed the original draft.

      Transparency declaration

      The authors declare that they have no conflicts of interest. This work received no public or private funds. Our group has received private funds for research projects from Roche Diagnostics (Spain), Biomerieux Inc. (France), Pfizer (Spain) and Merck (Spain) within the last 3 years.

      Acknowledgements

      We are grateful to Abbott Diagnostics for providing the Panbio™ COVID-19 Ag Rapid Test Device kits. We thank all personnel working at Microbiology Service of Clinic University Hospital for their unwavering commitment in the fight against COVID-19. We would also like to thank María José Beltrán, Pilar Botija and Ana Sanmartín for assistance in organizing RAD testing in primary healthcare centres.

      Appendix A. Supplementary data

      The following are the Supplementary data to this article:

      References

      1. Overview of testing for SARS-CoV-2 (COVID-19).
        (Updated September 18)
        • Pekosz A.
        • Cooper C.
        • Parvu V.
        • Li M.
        • Andrews J.
        • Manabe Y.C.
        • et al.
        Antigen-based testing but not real-time PCR correlates with SARS-CoV-2 virus culture.
        medRxiv. 2020; (10.02.20205708)https://doi.org/10.1101/2020.10.02.20205708
        • Linares M.
        • Pérez-Tanoira R.
        • Romanyk J.
        • Pérez-García F.
        • Gómez-Herruz P.
        • Arroyo T.
        • et al.
        Panbio antigen rapid test is reliable to diagnose SARS-CoV-2 infection in the first 7 days after the onset of symptoms.
        MedRxiv. 2020; (09.20.20198192)https://doi.org/10.1101/2020.09.20.20198192
        • Young S.
        • Taylor S.N.
        • Cammarata C.L.
        • Varnado K.G.
        • Roger-Dalbert C.
        • Montano A.
        • et al.
        Clinical evaluation of BD Veritor SARS-CoV-2 point-of-care test performance compared to PCR-based testing and versus the Sofia 2 SARS Antigen point-of-care test.
        J Clin Microbiol. 2020; (Oct 6):JCM.02338-20
        • Scohy A.
        • Anantharajah A.
        • Bodéus M.
        • Kabamba-Mukadi B.
        • Verroken A.
        • Rodriguez-Villalobos H.
        Low performance of rapid antigen detection test as frontline testing for COVID-19 diagnosis.
        J Clin Virol. 2020; 129 (Aug): 104455
        • Lambert-Niclot S.
        • Cuffel A.
        • Le Pape S.
        • Vauloup-Fellous C.
        • Morand-Joubert L.
        • Roque-Afonso A.M.
        • et al.
        Evaluation of a rapid diagnostic assay for detection of SARS-CoV-2 antigen in nasopharyngeal swabs.
        J Clin Microbiol. 2020; 58 (e00977-20)
        • Mak G.C.
        • Cheng P.K.
        • Lau S.S.
        • Wong K.K.
        • Lau C.S.
        • Lam E.T.
        • et al.
        Evaluation of rapid antigen test for detection of SARS-CoV-2 virus.
        J Clin Virol. 2020; 129: 104500
        • Baggio S.
        • L'Huillier A.G.
        • Yerly S.
        • Bellon M.
        • Wagner N.
        • Rohr M.
        • et al.
        SARS-CoV-2 viral load in the upper respiratory tract of children and adults with early acute COVID-19.
        Clin Infect Dis. 2020; (Aug 6): ciaa1157
        • Wölfel R.
        • Corman V.M.
        • Guggemos W.
        • Seilmaier M.
        • Zange S.
        • Müller M.A.
        • et al.
        Virological assessment of hospitalized patients with COVID-2019.
        Nature. 2020; 581: 465-469
        • Huang C.G.
        • Lee K.M.
        • Hsiao M.J.
        • Yang S.L.
        • Huang P.N.
        • Gong Y.N.
        • et al.
        Culture-based virus isolation to evaluate potential infectivity of clinical specimens tested for COVID-19.
        J Clin Microbiol. 2020; 58 (e01068-20)
        • Bullard J.
        • Dust K.
        • Funk D.
        • Strong J.E.
        • Alexander D.
        • Garnett L.
        • et al.
        Predicting infectious severe acute respiratory syndrome coronavirus 2 from diagnostic samples.
        Clin Infect Dis. 2020; (May 22): ciaa638
        • Mina M.J.
        • Parker R.
        • Larremore D.B.
        Rethinking Covid-19 test sensitivity—a strategy for containment.
        N Engl J Med. 2020; (Sep. 30)https://doi.org/10.1056/NEJMp2025631