Diagnostic accuracy of loop-mediated isothermal amplification coupled to nanopore sequencing (LamPORE) for the detection of SARS-CoV-2 infection at scale in symptomatic and asymptomatic populations

Objectives Rapid, high throughput diagnostics are a valuable tool, allowing the detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in populations so as to identify and isolate people with asymptomatic and symptomatic infections. Reagent shortages and restricted access to high throughput testing solutions have limited the effectiveness of conventional assays such as quantitative RT-PCR (RT-qPCR), particularly throughout the first months of the coronavirus disease 2019 pandemic. We investigated the use of LamPORE, where loop-mediated isothermal amplification (LAMP) is coupled to nanopore sequencing technology, for the detection of SARS-CoV-2 in symptomatic and asymptomatic populations. Methods In an asymptomatic prospective cohort, for 3 weeks in September 2020, health-care workers across four sites (Birmingham, Southampton, Basingstoke and Manchester) self-swabbed with nasopharyngeal swabs weekly and supplied a saliva specimen daily. These samples were tested for SARS-CoV-2 RNA using the Oxford Nanopore LamPORE system and a reference RT-qPCR assay on extracted sample RNA. A second retrospective cohort of 848 patients with influenza-like illness from March 2020 to June 2020 were similarly tested from nasopharyngeal swabs. Results In the asymptomatic cohort a total of 1200 participants supplied 23 427 samples (3966 swab, 19 461 saliva) over a 3-week period. The incidence of SARS-CoV-2 detection using LamPORE was 0.95%. Diagnostic sensitivity and specificity of LamPORE was >99.5% (decreasing to approximately 98% when clustered estimation was used) in both swab and saliva asymptomatic samples when compared with the reference RT-qPCR test. In the retrospective symptomatic cohort, the incidence was 13.4% and the sensitivity and specificity were 100%. Conclusions LamPORE is a highly accurate methodology for the detection of SARS-CoV-2 in both symptomatic and asymptomatic population settings and can be used as an alternative to RT-qPCR.


a b s t r a c t
Objectives: Rapid, high throughput diagnostics are a valuable tool, allowing the detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in populations so as to identify and isolate people with asymptomatic and symptomatic infections. Reagent shortages and restricted access to high throughput testing solutions have limited the effectiveness of conventional assays such as quantitative RT-PCR (RT-qPCR), particularly throughout the first months of the coronavirus disease 2019 pandemic. We investigated the use of LamPORE, where loop-mediated isothermal amplification (LAMP) is coupled to nanopore sequencing technology, for the detection of SARS-CoV-2 in symptomatic and asymptomatic populations. Methods: In an asymptomatic prospective cohort, for 3 weeks in September 2020, health-care workers across four sites (Birmingham, Southampton, Basingstoke and Manchester) self-swabbed with nasopharyngeal swabs weekly and supplied a saliva specimen daily. These samples were tested for SARS-CoV-
Rapid detection of infected cases in order to limit transmission, remains challenging as most validated methods use quantitative RT-PCR (RT-qPCR) [2]. Although considered the reference standard for diagnosis, RT-qPCR is laborious and can be difficult to scale up for mass-testing; in addition, competition for reagents/equipment from many laboratories may lead to widespread reagent shortages. Initially in the outbreak, laboratories throughout the UK used primers that were designed to target sequences within the RNAdependent polymerase gene (RdRp) [3]; however, these lacked sensitivity [4]. RT-qPCR primer sets were introduced that targeted the envelope (E), nucleocapsid (N) and ORF1ab genes, which provided the necessary increased sensitivity [5,6].
Nanopore sequencing allows rapid sequencing using protein nanopores embedded in a lipid membrane [9]. Nanopore sequencing technology allows all the advantages of conventional next-generation sequencing technology, especially the capacity to perform very high (limited only by the number of barcodes available) sample multiplexing within the same sequencing run.
LamPORE is a combination of LAMP and nanopore sequencing, developed by Oxford Nanopore Technologies (Oxford, UK) [10]. This technology has a theoretical maximum capacity of 15 000 samples per GridIon Mk 1 machine (Oxford Nanopore Technologies) per 24 hours, allowing scalability and high throughput.
Use of alternative sampling strategies, such as saliva, could theoretically increase capacity over swabbing (because of less pressure on the supply chain) and increase compliance because of the less invasive nature of sampling of saliva [11].
This study aimed to assess the assay performance characteristics of the LamPORE SARS-CoV-2 Detection Assay against the reference standard RT-qPCR for SARS-CoV-2 detection in both symptomatic and asymptomatic populations from multiple independent centres.

Study design
The study consisted of a retrospective and prospective diagnostic accuracy study comparing the performance of LamPORE sequencing of the ORF1ab, N2 and E gene targets of SARS-CoV-2 against RT-qPCR of the ORF1ab and N1 gene targets of SARS-CoV-2 [12].

Participants
For the prospective study, 1200 health-care workers at high risk of asymptomatic transmission were recruited as part of a consented National Health Service (NHS) England and NHS Improvement service evaluation in September 2020. Prospective participants performed nasopharyngeal self-swabs at days 0, 7, 14 and 21 as well as daily saliva sampling for 21 days (Fig. 1). Participants were recruited from staff working within five sites: University Hospitals Birmingham NHS Foundation Trust, Birmingham Women's' and Children's' NHS Foundation Trust, University Hospital Southampton NHS Foundation Trust, Hampshire Hospitals NHS Foundation Trust and Manchester University NHS Foundation Trust. The retrospective study was undertaken by collating surplus samples from patients whose diagnostic samples had been sent to the Public Health England West Midlands laboratory for respiratory panel testing for influenza-like illness from January 2020 to June 2020 (National Research Ethics Service Committee West MidlandsdSouth Birmingham 2002/201 Amendment Number 4).

Sampling
For swab-based tests, participants underwent self-directed nasopharyngeal swabbing using flocked swabs containing viral transport medium. For saliva-based tests, participants were instructed to dribble at least 1 mL of saliva into a universal specimen container without any additive. Samples were tested immediately if returned on the day of testing or if received on Friday they were stored for a maximum of 4 days at 4 C then tested.

Clinical material used for analytical performance
To assess the limit of detection and precision, a tenfold dilution series (from 20 000 copies/mL to 0.2 copies/mL) of droplet digital polymerase chain reaction (ddPCR) quantified SARS-CoV-2 was used and tested in triplicate. A panel of respiratory viruses (Zeptometrix Respiratory Panel R2; Zeptometrix, Buffalo, NY, USA) was used to assess the specificity of the LamPORE assay.

RNA extraction
For all samples at the Birmingham site, samples were heated to 56 C for a minimum of 30 minutes to inactivate live SARS-CoV-2 and. RNA extraction was performed. For every batch of RNA extraction performed (95 samples/batch) an RNA extraction control was used consisting of heat-inactivated SARS-CoV-2 grown on Vero-E6 cells (PHE SARS-CoV-2 England reference strain).

Reference test (RT-qPCR)
Single-step RT-qPCR against the ORF1ab and N1 gene targets of SARS-CoV-2 was carried out using the CerTest ViaSure SARS-CoV-2 real-time PCR kit (CerTest Biotech SL, Zaragoza, Spain) according to the manufacturer's instructions for use on ThermoFisher Quant-Studio 5 or BioMolecular Systems MIC instruments, using 5 mL of extracted RNA per reaction [2,12].

Comparator test (LamPORE)
For each sample, 20 mL of RNA sample underwent amplification and sequencing using the LamPORE technique as per the manufacturer's protocol. A proprietary Guppy/VSEARCH/Snake-Make pipeline (algorithmic details as per James et al. [13]) that aligned reads to the viral target genes and a human b-actin (ACTB) internal control and reported results in absolute reads per sample per gene.

Sample size calculation
Sample size was determined pragmatically, based on the incidence seen in the UK at the time of the study (1%). Sample size was calculated using R code (using R 3.6.3 [14]) from the methodology of Stark et al. [15] for binary diagnostic test outcomes (a ¼ 0.05, b ¼ 0.90) setting a base sensitivity and specificity of RT-qPCR of 95% and 99%, respectively. We aimed to be able to detect a change of sensitivity and specificity of 10% in LamPORE, respectively giving a sample size greater than 9600 in the prospective cohort.

Results interpretation
The readers of RT-qPCR and LamPORE tests were blinded to any clinical information relating to study participants. For the RT-qPCR reference assay, SARS-CoV-2 was said to be detected if the following conditions were met; amplification of the kit internal control, amplification of either the ORF1ab or N gene with a cycle threshold (Ct) < 38, detection of the positive control on the sample plate, detection of the RNA extraction control on the sample plate, and no SARS-CoV-2-specific amplification in the negative control. Ct values were calculated automatically using instrument software with automatic baseline setting calculated. All curves were manually inspected by two investigators to check for quality and inhibition of reaction.

LamPORE
Aligned read counts were generated via the LamPORE pipeline against ORF1ab (labelled AS1), E1 and N2 genes, as well as a human ACTB gene internal control. Any unaligned reads were marked as Undetermined. Samples were called positive if any of the SARS-CoV-2 target genes had >50 reads/sample, indeterminate if between 20 and 50 reads and negative if < 20 reads. ACTB gene counts were not used as part of the calling algorithm but were used to infer sufficient sampling.
Test results were compared using a 2 Â 2 table and standard measures of sensitivity, specificity, positive predictive value and negative predictive value were calculated using R. If results from either the RT-qPCR or LamPORE test were missing or indeterminate then no comparison was made and the sample was removed from the analysis. Standard analyses of variability in diagnostic precision were made, and modifications were made to the analysis to carry out clustered analyses of diagnostic precision using clustered logistic regression with a sandwich estimator (Stata 16.1; StataCorp, College Station, TX, USA), on the basis that each separate study represented a cluster within the whole and may bias estimates of sensitivity and specificity [16].

Participants
For the prospective asymptomatic study, a total of 1200 participants who were at work and reported to be well were recruited across the four sites (Birmingham n ¼ 600, Southampton n ¼ 200, Basingstoke/Winchester n ¼ 200, Manchester n ¼ 200). There were no adverse events. Sample flow is shown in Fig. 2.

Analytical performance
LamPORE reliably detected SARS-CoV-2 to the 20 copies/mL of sample. SARS-CoV-2 reads were detected in the 0.2 copies/mL sample but this was below the threshold for calling a positive sample in LamPORE and were not detected with RT-qPCR (Table 1, Fig. 3).
Intra-assay and inter-assay precision was calculated against the ORF1ab gene. For intra-assay precision on a single day, the standard deviation of ORF1ab was 50 reads with a coefficient of variation (CV) of ±2.3% (see Supplementary material, Table S1). For interassay precision across multiple days, the standard deviation of ORF1ab was 178 reads with a CV of ±7.8% (see Supplementary material, Table S2).
For reproducibility for 24 replicates the standard deviation for the ORF1ab gene was 128 reads with a CV of ±3.9% (see Supplementary material, Table S3).
For analytical specificity of the LamPORE assay, SARS-CoV-2 was not detected in any of the samples within the respiratory virus panel. In terms of quality control, the median number of reads reported aligned per sample to ACTB was 571 (range 76e7249 reads). The median ratio of mapped (to SARS-CoV-2 or ACTB) to unmapped reads was a median of 0.71 (interquartile range e0.63 to 0.79). There were generally lower levels of reads seen in extracted viral transport medium versus saliva in ACTB (mean 646 versus 206 reads, p < 0.001), ORF1ab (mean 146 versus 98 reads, p < 0.001) and E1 (14 versus 8 reads, p < 0.01) but not in N2 (22 versus 17 reads, p 0.06). In order to understand the context in which LamPORE operates, a comparative experiment was carried out examining the sensitivity of our reference RT-qPCR assay compared with fluorimetric LAMP (Optigene LAMP kit) and LamPORE (see Supplementary material, Table 4). This showed that LamPORE detected three additional positive samples detected by RT-qPCR that were not detected by RT-LAMP.

All participants
In total, 23 427 samples were obtained from all participants, of which 22 401 were from the asymptomatic study and 848 were from the retrospective symptomatic cohort. Both LamPORE (comparator assay) and RT-qPCR (reference assays) were performed on all 23 427 samples ( Table 2).
Of the 601 samples positive on the LamPORE assay, 477 were also positive and 124 were negative by RT-qPCR and in comparison with the reference assay 124 were false positives. Of the 22 826 negative samples, 22 824 were confirmed as negative by the RT-qPCR and there were two samples that were positive by RT-qPCR.
When modelled at 1% population prevalence the PPV dropped to 66.24% and at 0.1% population prevalence the PPV was 16.3%. NPV remained at >99.99% in all population scenarios.
If sensitivity and specificity were calculated with regard to the cohort (n ¼ 1200, see Supplementary material, Table S5) rather than individual test results, and either a positive or negative test was counted at any point in the 21 days of sampling, 66 were positive by LamPORE with an additional two participants detected by qPCR and 1031 testing negative by LamPORE with a further 101 testing negative by qPCR, giving a sensitivity of 97.06% (95% CI 89.78%e99.64%), a specificity of 91.08% (95% CI 89.26%e92.67%), a PPV of 39.52% (95% CI 35.07%e44.16%) and a negative predictive value of 99.81% (95% CI 99.25%e99.95%).

Asymptomatic cohort
For the asymptomatic cohort ( Table 2)

Symptomatic cohort
There was complete agreement between the RT-qPCR and LamPORE assays for 116 positive samples and 752 negative samples, for this cohort resulting in a diagnostic sensitivity of 100%, diagnostic specificity of 100%, PPV of 100% and NPV of 100%. The incidence of SARS-CoV-2 was 13.4% over the study period.

Variability across time course
To understand the utility of LamPORE across the time course of infection, a single participant who was identified as the beginning of their infection, early in the study with a long time course (5 days) of positivity was studied with daily saliva sampling as per protocol (Fig. 4). Initially a high viral load, indicative of a Ct value of 19.5 was Table 2 Diagnostic assay performance tables for RT-qPCR versus LamPORE assay for whole cohort, asymptomatic cohort (swabs), asymptomatic cohort (saliva) and retrospective symptomatic cohort  observed, which increased (indicating decreasing viral load) over the 5 days to 23.5 and then became undetectable at day 6. Only LamPORE N2 reads were detectable at day 1, but E1 reads became detectable at day 2 and ORF1ab reads at day 4.
To understand how the assay performed across the whole spectrum of viral loads, we plotted Ct against numbers of reads for the ORF1ab and N2 genes (see Supplementary material, Figs S1 and S2). There was no strong correlation between number of reads and Ct values, we hypothesized that tjhis was a result of the nature of the non-linear amplification techniques used.

Discussion
We carried out a very large asymptomatic cohort study of healthcare workers using a novel technology, LamPORE, comparing it with a reference RT-qPCR assay. We found that LamPORE has high sensitivity and specificity (>99%) in both the asymptomatic and symptomatic populations, directly comparable to RT-qPCR, and therefore has comparable predictive ability across a range of uses in varying levels of population prevalence. We studied a population with a wide range of viral loads as determined by cycle threshold, with LamPORE demonstrating good detection across the range.
LamPORE has the advantage that it is scalable [10] to allow testing of very large population levels because of the use of sample barcoding allowing pooling of up to 3500 samples on a single GridIon instrument. Due to the increased sensitivity of LAMP as part of the LamPORE system, it gives very high sensitivity for SARS-CoV-2. With combinatorial barcoding, as has been adopted in other population level assays [17] on larger flow cells (e.g. a Promethion flow cell), even greater sample multiplexing may be achievable. Another potential inherent advantage is the ability to multiplex gene targets allowing the detection of multiple respiratory pathogens [18] such as SARS-CoV-2, influenza and respiratory syncytial virus. It is not known what the upper limit of multiplexing of LAMP primers is, and they are considerably more complex to design than PCR primers [19]. Given the advantages of LAMP in terms of speed of amplification [8] and sensitivity of detection, an exploration of LAMP multiplexing is urgently required. Also, the assay chemistry uses different enzymes and methodologies to PCR, meaning a diversification of supplies and therefore potentially fewer reagent shortages in a pandemic setting. However, LAMP plus sequencing introduces several steps into the workflow, which means that LamPORE becomes inherently 'non-linear', i.e. the relationship between the genes amplified and sample viral load is not linear, so LamPORE may only be used to infer positivity rather than giving any measure of viral load.
A potential disadvantage of LamPORE is the differing workflow needed to prepare samples, including the LAMP step and library preparation, barcoding then sequencing. This requires more sample preparation steps than an RT-qPCR workflow.
During the testing of the asymptomatic cohort, we observed a number of false positives using LamPORE when compared with the RT-qPCR assay. There are a number of possible explanations for this observation. First, LAMP amplification is more sensitive than PCR amplification [20], so contamination risk is high; however, as the laboratories refined the technique, contamination issues seemed to resolve. Second, it is feasible that some of the samples are in fact, true positives as demonstrated by the ability of LamPORE to detect spiked, killed virus beyond the limit of detection of RT-qPCR. This may have useful implications for sample pooling [21], as greater sensitivity would allow more samples to be pooled and tested. Finally, the LamPORE protocol requires multiple manual liquid handling steps, which can lead to error and increases the number of opportunities for contamination to occur. We found two falsenegative samples within the saliva cohort. These had high Ct values (ORF1ab Ct 36.5 and 37.1) in the PCR, and we hypothesize that this may have occurred because of low viral loads as the extracted RNA went through a freeze thaw before it was run on LamPORE.
In conclusion, we have demonstrated the accuracy of LamPORE across a range of population use cases, maintaining a high specificity and sensitivity, reproducibility and limit of detection, as well as working well on saliva samples, making it suitable for the detection of symptomatic and asymptomatic SARS-CoV-2 infections.

Author contributions
Conceptualization was by ADB, AR, SH, AD and DP; data were curated by ADB and CW; formal analysis was by ADB, AR, AW, DB, SH and AD. ADB was responsible for the funding acquisition and software. ADB, AR, AW, VF, SK, SH, AD, ZD and DP contributed to the methodology. Project administration was by SH, AD, ZD and DP; supervision and validation were by ADB, AR, SK and VF; and visualization was by ADB and AR. All authors contributed to the investigation, to writing the original draft, and to reviewing and editing.

Transparency declaration
ADB has received travel funding from the Oxford Nanopore Community Meeting 2019 from Oxford Nanopore. The other authors declare no competing interests. 1348.e6