In a recent study published on the medRxiv * preprint server, a multinational group of scientists studied the performance characteristics of several point-of-care diagnostic (POC) kits for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2).
To study: Point-of-care point-of-care laboratory and clinical assessment of anti-SARS-CoV-2 devices. Image Credit: Jarun Ontakrai / Shutterstock.com
The unprecedented increase in SARS-CoV-2 cases has led to a rapid increase in demand for commercial diagnostic kits. However, such a rapid increase could potentially compromise the quality due to the lack of performance and clinical data on these test kits. Therefore, it is crucial to independently evaluate and validate these tests to ensure a minimum acceptable standard before widespread use in the population.
Currently, reverse transcriptase polymerase chain reaction (RT-PCR) is widely used to detect SARS-CoV-2 from nasal / oral samples; however, the limited access to RT-PCR for symptomatic cases, capacity constraints, and the short period of active infection may underestimate the burden of infection. Serological testing has therefore become key tools for monitoring the incidence of coronavirus disease 2019 (COVID-19).
About the study
In the current study, researchers evaluated 14 POC antibody tests, including 13 lateral flow immunoassays (LFA) and a microfluidic immunofluorescence assay (Lumira DX). The team tested the specificity and sensitivity of these tests to determine whether they meet the standards of the UK Medicines and Health Products Regulatory Agency (MHRA).
The MHRA determined that SARS-CoV-2 LFAs must exhibit sensitivity and specificity greater than 98% in samples collected up to 20 days after symptom onset. The group also analyzed changes in test sensitivity with increasing infection time and studied the status of neutralizing antibodies to correlate with immune status as well as the usefulness of these tests for patients. Hair and serum samples were collected in pairs to determine if the POC tests performed comparably on the hair samples.
The researchers found that only the LumiraDX POC assay could meet the MHRA standard in terms of sensitivity and specificity. Other LFAs, including those manufactured by Biomerica, Biozek, Fortress, Menarini, and Roche, have reported specificity of over 98%, even for high specificity serum samples.
Surprisingly, the researchers found variations in different lots of Menarini kits reporting a sensitivity of 76.3% with one lot and 94.9% with another lot for the same set of samples. All LFAs showed greater than 90% sensitivity for samples after more than 21 days of symptom onset.
Sensitivity versus specificity for the kits tested (IgG only, and positive or negative result for the LumiraDx test). For each kit, sensitivity ≥ 20 days after symptom onset is indicated. MHRA targets of 98% sensitivity and specificity are shown by dotted lines. Pharmact was not included in the graph due to its low specificity. The LFA which has not passed this stage is indicated in red.
The researchers also evaluated the performance of the POC kits over time and found that the LumiraDX maintained its constant sensitivity of over 98% for up to 224 days after the onset of symptoms. Neutralizing titers were available for some samples and were established by comparing POC results with semi-maximal neutralizing titers (NT50). A significant difference was observed in the NT50 level between positive and negative LFA results and a wide range of NT50 titer values were recorded for positive LFA results.
For paired serum and capillary samples, the serum sample was lab examined and interpreted by health workers (HCs), while capillary samples were read by the participant along with a second TS. Lower sensitivity was observed in hair samples compared to serum samples for five of the seven LFAs.
The magnitude of the difference in sensitivity ranged from 19.1 to 34.2% for LFAs from Biozek and Roche, regardless of who read the results. The Fortress LFA reported a slight increase in sensitivity for hair samples; however, it was not statistically significant. The LumiraDX test showed 100% sensitivity for both samples.
Sensitivity as a function of time after symptom onset for the selected kits (IgG for all LFAs except LumiraDX, where the overall antibody result was used).
About 75% of the participants said that it was very easy or easy to use the kit using the instructions on the package leaflet. Additionally, 89% found it very easy or easy to decipher the explanation for the results, while 20% said it was difficult or very difficult to collect the sample.
The results of the present study revealed variations between POC kits. In addition, several POC tests did not meet the performance characteristics indicated by the manufacturers, which underlies the need for an independent evaluation of the tests. Neutralizing antibody data revealed that these tests could not be used to confer immune status to SARS-CoV-2.
A significant difference was reported in the sensitivity between serum and hair samples. One of the main advantages of using POC tests is that even ordinary people can perform the test without the need of a qualified healthcare professional.
The observations discussed here highlight the need to optimize POC assays to generate comparable results on capillary and serum samples. The results of the current study could help countries identify people who may not need an imminent vaccination or adjust the vaccine dose to meet pan-national immunization status, given limited vaccine resources.
medRxiv publishes preliminary scientific reports that are not peer reviewed and, therefore, should not be considered conclusive, guide clinical practice / health-related behavior, or treated as established information