Evaluating Accuracy of Antigen, Molecular-Based Point-of-Care COVID-19 Tests

Rapid, point-of-care antigen tests have demonstrated utility in identifying cases of coronavirus disease 2019 (COVID-19) in symptomatic individuals, according to the results of an updated systematic review published in the Cochrane Database of Systematic Reviews.¹Test accuracy, however, varies widely by brand, and few meet World Health Organization (WHO) minimum acceptable performance standards.

Rapid diagnosis of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infections is necessary to ensure optimal public health and patient outcomes. Although rapid point-of-care tests do not measure disease severity, prompt determination of infection can help to guide clinical decision-making.

Led by Jac Dinnes, PhD, MSc, Ma, PGDip, Senior Test Evaluation Methodologist at the Institute of Applied Health Research at the University of Birmingham in the United Kingdom, a team of researchers have been working to create and maintain regular systematic reviews evaluating tests and patient characteristics in COVID-19 diagnosis. The most recent update¹ expands on a previously published analysis² of the accuracy of point-of-care rapid antigen and molecular tests, using data from published articles available from January 1, 2020, to September 30, 2020.  

Because public health interventions for COVID-19 focus on reducing disease transmission, it is especially important for clinicians to be able to quickly and accurately identify and isolate patients with SARS-CoV-2 infections. The current review focuses on SARS-CoV-2 infection as the target condition for both symptomatic and asymptomatic testing applications. Included tests were those that had the capacity to “be performed at the ‘point-of-care’ or in a ‘near-patient’ testing role,” according to the authors. Criteria for point-of-care testing include:  

• Portability or easy transport of the equipment for running or reading the assay;
• Minimal sample preparation requirements with no requirement for additional equipment or precise sample volume transfer unless a disposable automatic fill or graduated transfer device is used;
• Minimal biosafety requirements, such as personal protective equipment for the sample collector and operator, good ventilation, and a biohazard bag for waste disposal;
• No requirement for a temperature-controlled environment; and
• Test results available within 2 hours of sample collection

At the time of the review, 129 rapid antigen and 142 automated molecular tests were available for commercial use outside of a laboratory setting. The review included 64 reports, with data from 78 study cohorts tested via point-of-care COVID-19 testing; 48 of these evaluated antigen tests, and 29 evaluated molecular tests.

Antigen Test Results

Review results showed an average sensitivity and specificity of 68.9% and 99.6% (95% CI, 61.8-75.1 and 99.0-99.8), respectively, based on 51 evaluations and 21,614 samples (6136 samples with confirmed SARS-CoV-2 infection). Subgroup analyses by symptom status indicated that the average test sensitivity to detect infection was 13.8 percentage points lower in asymptomatic individuals compared with those who were symptomatic (58.1% vs 72%; 95% CI, 40.2-74.1 and 63.7-79).

When this comparison was restricted by symptom status to the 9 evaluations that reported data for both symptomatic and asymptomatic subgroups, similar differences in sensitivity were noted (14.4 percentage points lower in asymptomatic individuals; 95% CI, 38.8 lower to 10.0 percentage points higher).

Data for time from symptom onset were pooled separately for sensitivity and specificity. Sensitivity was 78.3% in the first 7 days following symptom onset (95% CI, 71.1-84.1) vs 51.0% in the second week of symptoms (95% CI, 40.8-61.0), representing a decrease of 27.3 percentage points (95% CI, -32.8 to -21.9). No differences in specificity were observed.

Per the researchers, antigen tests “do not appear to perform as well in asymptomatic populations compared [with] symptomatic populations for detecting infection,” although the data for asymptomatic populations are less prevalent and more limited.

Molecular Test Results

Among the rapid molecular test evaluations including samples both with and without SARS-CoV-2 infection, average sensitivity and specificity was 95.1% and 98.8% (95% CI, 90.5 and 97.6 and 98.3-99.2), respectively. Sensitivity data according to viral load were extracted from 10 evaluations of molecular tests. All sensitivity estimates for the subgroup with a higher viral load (24 samples) were 100%. Within the group with lower viral load, average sensitivity was 95.6% (95% CI, 55.7-99.7).

No clear differences in average sensitivity or specificity were noted when studies were separated by design. The average sensitivity was elevated in 2-group studies compared with single-group studies (97.2% vs 93.2%) with a difference of 4.0 percentage points. Specificity averages were almost identical at 99.4% and 99.3% for each study type.

In the face of an increase in testing strategies and situations in which antigen and rapid molecular assays may be considered and used, the researchers cautioned that test performance will likely need to be “considered separately for each of the use cases.”

“It is good to have found evidence that some test brands do meet the minimum ‘acceptable’ performance standards set by [the] WHO for testing people with symptoms,” said author Jon Deeks, PhD, CStat, professor of biostatistics at the Institute of Applied Health Research at the University of Birmingham, in a press release.³ “However, they represent only a very small proportion of the commercially available tests….We didn’t find any data or studies evaluating the accuracy of these tests when used in repeated screening of people with no known exposure to SARS-CoV-2. These testing policies have been implemented without any supporting real-world evidence.”

“Our review shows that some antigen tests may be useful in healthcare settings where COVID-19 is suspected in people with symptoms,” said Dr Dinnes in the same press release.³ “These tests do not appear to perform as well in people who don’t have symptoms of COVID-19. Confirming a positive result from a rapid test with a [reverse transcription polymerase chain reaction (RT-PCR)] test, particularly when cases of COVID-19 are low, may help avoid unnecessary quarantine.”

“All antigen tests will miss some people with infection, so it is important to inform people who receive a negative test result that they may still be infected,” she added.

Dr Dinnes provided further insight into the study results.

Of the 6 key findings described in the discussion, which are the most important in terms of implications for pandemic management going forward? What should clinicians keep in mind when using these point-of-care COVID tests?

Rapid antigen (lateral flow) tests are best used for a rapid diagnosis of COVID-19 in people with signs and symptoms of infection, especially when they are used during the first week after symptom onset. They are particularly useful for triggering immediate self-isolation and contract tracing for those with positive results and for allowing other diagnoses to be considered in those with negative results, especially those who are acutely ill.  The available tests are not equivalent, however, and based on pooled analyses, only one, the Standard™ Q COVID-19 Ag (SD Biosensor), meets WHO desirable performance targets of at least 80% sensitivity and 97% specificity. Depending on the prevailing prevalence of infection and the level of clinical of suspicion of COVID-19, confirmatory PCR may still be needed to mitigate the effects of false-negative or false-positive results.

Lateral flow tests are less sensitive when used for screening asymptomatic people, particularly if there is no clear epidemiologic basis for testing. For this iteration of the review, only 2 tests had been evaluated in more than 200 asymptomatic individuals: the Panbio™ COVID-19 Ag (Abbott Laboratories) and SD Biosensor Standard Q. Serial testing may compensate for poor test sensitivity, but we found no empirical evidence to support this approach.

At least 1 study published since our search cut-off suggests even daily testing may not avoid outbreaks of infection, possibly due to behavioral changes following negative lateral flow tests. At low prevalence of infection, false positives from lateral flow tests soon outweigh the number of true positives, potentially leading to significant numbers of individuals and their contacts self-isolating unnecessarily unless confirmatory PCR testing is used. Proper evaluation is needed to identify whether detecting asymptomatic cases leads to worthwhile reductions in disease spread.

Data for rapid molecular tests were limited, and although some tests show promisingly high accuracy, these tests need to be evaluated in more clinically relevant populations and at the point of care rather than in laboratory-based studies using samples left over from routine diagnostics.

Could you discuss the overall quality of the research available on these point-of-care COVID tests? What are your most significant methodological concerns, and how might these be mitigated in future research?

The methodological quality of studies has generally improved over time, especially for studies of lateral flow tests. A considerable proportion of studies recruit participants from COVID-19 testing clinics or emergency departments and implement procedures to minimize bias in results. Only approximately one-quarter of lateral flow evaluations clearly reported on-site point-of-care testing, however, and only 50% conducted tests according to the manufacturer’s instructions for use.

Evaluations of molecular tests were more likely to be at high or unclear risk of bias, again, because many were conducted in laboratories using anonymized remnant samples and providing limited details of study participants. Only slightly more than one-quarter were conducted according to the test manufacturer’s instructions for use, and in 50% testing was conducted by trained laboratory staff rather than at the point of care.

In order to understand how tests will perform in practice, the evaluations need to be conducted within their intended use settings, recruiting the full spectrum of patients who would be tested. Testing should be carried out by relevant personnel, at the same time minimizing bias in patient selection, test and reference standard conduct, and interpretation.

Future iterations of this review are underway, including data published through January 2021. How is that research different from the previous research that was used to create this review? Are these newer papers including evidence on rapid testing in asymptomatic individuals?

So far, the new searches take us through to March 1, 2021. We are certainly seeing more evaluations of lateral flow tests in asymptomatic cohorts, but it is too early to say much more about how these might affect our overall results and interpretation. 

Based on the available research, was there one specific study or test that stood out in terms of efficacy and accuracy?

I wouldn’t say there is one specific study, but so far there is only one test that on average performs at a sufficiently high standard for use in symptomatic people (the Standard Q COVID-19 Ag). The Panbio COVID-19 Ag assay is a close second but does not quite meet the WHO threshold. I am not certain, but I do not believe that this assay is quite the same as the BinaxNOW™ COVID-19 Ag Card (Abbott Laboratories) that is available in the United States. We have found at least one study of the BinaxNOW that will be included in the next review update.

Given the data presented here, what are your thoughts on the efficacy of the at-home COVID tests available to consumers? Might those tests have similar limitations?

In principle these use the same methodology as the lateral flow tests in our review and are likely to have the same limitations, although I haven’t checked to see which tests have been licensed for at-home use in the United States.

What are the implications of the results of this review in terms of pandemic management going forward?

As vaccination rates increase and the prevalence of infection hopefully drops, there will still be a place for rapid point-of-care tests. I would imagine they will be increasingly valuable for frontline clinicians looking to make a quick assessment of whether an individual presenting with possible symptoms of COVID-19 is likely to have the infection or not.

At a very low prevalence of infection, it is questionable whether mass asymptomatic testing — for example, in a test-to-enable context, to allow people to attend large events — will yield a sufficient volume of true cases of infection to be worth the trade-off in harms from false-positive results.

This interview has been lightly edited for length and clarity.

References

1. Dinnes J, Deeks JJ, Berhane S, et al; for the Cochrane COVID-19 Diagnostic Test Accuracy Group. Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev. 2021;3:CD013705. doi: 10.1002/14651858.CD013705
2. Dinnes J, Deeks JJ, Adriano A, et al; for the Cochrane COVID-19 Diagnostic Test Accuracy Group. Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev. 2020;8:CD013705. doi: 10.1002/14651858.CD013705
3. Updated Cochrane review assesses how accurate rapid tests are for detecting COVID-19 [press release]. Wiley Newsroom. March 24, 2021. Accessed April 2, 2021. https://newsroom.wiley.com/press-releases/press-release-details/2021/Updated-Cochrane-review-assesses-how-accurate-rapid-tests-are-for-detecting-COVID-19/default.aspx.

This article originally appeared on Infectious Disease Advisor