Aim To examine performance in the UK National External Quality Assessment Scheme (UKNEQAS) for toxoplasma serology for evidence of discrepant results as compared with the predistribution and postdistribution results supplied by the toxoplasma reference laboratories.
Methods Analysis of performance in the toxoplasma IgG and IgM schemes was made for the period 1994–2008 to look for trends in performance.
Results For the IgG scheme, a mean of 98% of participants obtained the correct result for detection of toxoplasma-specific antibody. The most common problem was failure to detect low levels of antibody. In some cases this was the result of participants deviating from the manufacturer's instructions and using higher cut-off levels. For the IgM scheme, an average of 95% of participants obtained the correct result for toxoplasma antibody detection. The most common problem was the failure of some enzyme immunoassay kits to detect specific toxoplasma IgM antibody, which was detected by the more sensitive immunosorbent agglutination assay.
Conclusions Performance standards in the UKNEQAS toxoplasma serology schemes were high. The problems encountered have highlighted the importance of detecting low levels of antibody, adhering to the kit manufacturer's instructions and selecting an appropriate assay for the clinical situation.
- quality assurance
- toxoplasma serology
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Participation of clinical laboratories in relevant external quality assessment (EQA) schemes is mandatory for clinical pathology accreditation and the following benefits have been described1:
Participating laboratories are able to assess whether their results are comparable with those of other laboratories
EQA can provide a valuable educational stimulus to laboratory staff
It provides credibility to the participating laboratory by providing evidence that it has a responsible attitude to quality issues (evidence of participation is required by some accrediting agencies)
EQA provides an insight to national performance levels
A UK National External Quality Assessment Scheme (UKNEQAS) for toxoplasma IgG serology was established in 1993, followed by a scheme for toxoplasma IgM serology in 1998. The schemes are operated from the Department of Clinical Parasitology, Hospital for Tropical Diseases, London, UK, and UKNEQAS for Microbiology, Health Protection Agency, London, UK. This longitudinal study examines the participants' performance in the schemes to assess the quality of toxoplasma testing in laboratories and identify problems and causes of poor performance.
Materials and methods
The toxoplasma IgG serology scheme consists of three distributions per year, each containing six specimens. The toxoplasma IgM scheme consists of two distributions per year, each containing four specimens. The specimens are single human serum samples and are provided by: the Scottish Toxoplasma Reference Laboratory, Inverness, Scotland, UK; the Toxoplasma Reference Laboratory, Swansea, Wales, UK; and the Department of Medical Microbiology, St George's Hospital, London, UK. These laboratories test the specimens for toxoplasma antibody prior to despatch. Specimens for the IgG scheme are tested for toxoplasma IgG/total antibodies by the Sabin–Feldman dye test (the accepted gold standard toxoplasma serology test), latex agglutination and IgG enzyme immunosorbent assays (ELISA, enzyme immunoassay (EIA)).4 Specimens for the IgM scheme are tested for total toxoplasma antibody by using the dye test, and for specific toxoplasma IgM by using the immunosorbent agglutination assay (ISAGA) (bioMérieux, Marcy l'Etoile, France) and a specific IgM EIA.4 Clinical information is supplied with each specimen.
Participants are required to select an appropriate assay for their defined clinical context in order to confirm or exclude the presence of toxoplasma-specific antibody, and to report their results within 3 weeks of specimen receipt. Results are collated and analysed, and a full report is provided to all participants. The report includes an analysis of the individual performance and a breakdown of the performance of individual toxoplasma assays. Specimens which do not give the same reference unit result as in predistribution testing are not scored. An important component of UKNEQAS is education, and the report also includes comments on the toxoplasma-specific antibody results for each specimen and its relevance to the clinical information supplied with the specimen. A teaching sheet, which contains information on a particular clinical aspect of toxoplasmosis, is also included in the report.
Toxoplasma IgG scheme
At the start of the IgG scheme in 1993, 141 UK laboratories participated in the scheme and nine different assays were used for the detection of toxoplasma specific IgG. In 1994, the scheme was offered to overseas laboratories and participation increased to 195 (153 from the UK, and 42 from overseas) participants using 14 different assays. Currently 329 laboratories from 28 countries participate (table 1) and 21 different assays are used (table 2). Over the 14-year period more than 95% of participants achieved the correct results, with a mean accuracy of 98% (figure 1). There were failures with latex agglutination and EIA tests, and problems detecting low levels of toxoplasma antibody.
Latex agglutination kits
In 2002, a significant number (ie, 45/83 and 55/87) of participants reported false-negative results for two specimens in two consecutive distributions using a commercial latex agglutination test kit. These specimens had dye test results of 16 IU/ml and were toxoplasma-antibody positive by EIA. The specimens were sent to the manufacturer for investigation and they confirmed low level toxoplasma-specific antibody. There was no evidence of batch-to-batch variation in the performance of the assay. It was noted that some of the participants who reported negative results gave a range of cut-off values for the test that deviated from the manufacturer's kit instructions. It was decided on advice from the UKNEQAS Toxoplasma Serology Advisory Panel that the scores allocated to the specimens would remain unchanged.
Enzyme immunosorbent assay
In 2003 and 2004, a significant number (35/64, 15/70 and 30/66) of participants reported false-positive results for three specimens in three separate distributions with one toxoplasma EIA kit. The specimens were sent to the manufacturer for further investigation. The manufacturer suggested that that the problem might relate to the storage conditions of the specimens predistribution and postdistribution, but there was no objective evidence to support this proposal and no explanations for why this particular kit produced results discordant with the other assays represented in the scheme. The manufacturers are continuing to monitor EQA results. It was decided on advice from the UKNEQAS Toxoplasma Serology Advisory Panel that these specimens would be scored.
In two distributions in 2006, and one distribution in 2008, a large number (ie, 136/311, 101/309 and 96/313) of participants reported false-negative results for three IgG specimens. This was reflected in the drop in performance (figure 1). All kits were affected by the inability to detect low levels of antibody. These specimens contained low levels of specific toxoplasma antibody (dye test 8 IU/ml) but such specimens are found in clinical practice and therefore should be represented in the specimens in the UKNEQAS distributions. It was decided on advice from the UKNEQAS Toxoplasma Serology Advisory Panel, that these samples would be scored.
Toxoplasma IgM scheme
At the start of the IgM scheme in 1999, 79 laboratories (36 from the UK and 43 from overseas) participated, and 12 different assays were used. This increased to 227 laboratories from 29 countries participating in 2007 (table 1) and 16 assays were used for the detection of toxoplasma-specific IgM antibodies (table 2). Over the 9-year period more than 90% of participants achieved the correct intended results with a mean accuracy of 95% (figure 2).
Performance fell in 2001, 2003 and 2004 (figure 2) when many EIA kits failed to detect toxoplasma IgM antibodies, which were detected by IgM ISAGA and other more sensitive kits, including kits that use fluorescence and chemiluminescence to detect IgM antibodies. This occurred in three specimens in three different distributions (ie, 51/187, 98/159 and 131/189 of participants achieved the correct results for these specimens). Although the results were included in the analyses, participants were not scored for these specimens, as not all laboratories used the IgM ISAGA. These specimens demonstrated the different sensitivities of the tests and highlighted the importance of choosing the correct assay for the clinical group being tested. The clinical details for two of the specimens were the investigation of lymphadenopathy and a neonate with failure to thrive.
The UKNEQAS toxoplasma serology scheme has been distributing serum specimens since 1993 in collaboration with the toxoplasma reference laboratories in Inverness and Swansea, and the department of Medical Microbiology, St George's Hospital, London, for the examination of toxoplasma antibodies. These specimens are designed to be assimilated into the routine workload of clinical laboratories. Both toxoplasma IgG and IgM schemes are designed to provide information allowing participants to gain an insight into their performance through the score that they receive, and to take individual action to investigate and remedy any incorrect results revealed when compared with the results obtained from the collaborative reference laboratories.
The schemes are unique in that they provide clinical histories with the sera to be examined. The clinical histories are selected by the toxoplasma reference laboratories and they reflect the signs and symptoms of cases of suspected toxoplasmosis encountered by clinical laboratories. Clinical histories include those from organ donors and recipients, those with ocular disease, immunocompromised patients, pregnant women, and patients with suspected congenital toxoplasmosis. The clinical histories are essential in helping the participants discern which tests to use, whether additional testing is required or whether to refer the sample to a toxoplasma reference laboratory for further analyses. In addition, the final report given to participants includes the results of several kits used to analyse the specimen predistribution by the toxoplasma reference laboratories, including the Sabin–Feldman dye test. This is accompanied by relevant interpretative comments for each specific specimen and a teaching sheet for an interesting specimen in each distribution. The teaching sheet may include the clinical features of a certain pathology associated with toxoplasmosis (eg, ocular toxoplasmosis), the management of patients groups with diagnosed toxoplasmosis (eg, immunocompromised patients), and the laboratory diagnosis of toxoplasmosis including non-routine rests for (eg, PCR). This gives participants a holistic perspective of toxoplasmosis. These aspects emphasise the strong educational feature of the schemes. A survey of participants in the UKNEQAS toxoplasma schemes showed that users found the wide variety of antibody titres distributed, and the additional interpretative clinical information supplied, beneficial in facilitating their choice of kit for routine clinical use.
An analysis of the specimens distributed since the start of the UKNEQAS toxoplasma serology scheme in 1993, and the toxoplasma IgM scheme in 1998, has shown that participants' performance was consistently high, with the mean percentage of participants achieving the correct result for each distribution being greater than 95% for the toxoplasma IgG scheme and greater than 90% for the toxoplasma IgM scheme. The analyses also showed that differences in performance characteristics between different assays were small, and that any difference reflected both the intrinsic precision of the assay as well as the technical procedures of the users. Consequently, detailed comparison of different commercial assays has not been attempted. Although it was noted that participant performance remained consistently high throughout the period of the study, there was no obvious trend towards improvement or deterioration in performance. Therefore, we have no direct evidence of the benefit of participation in the UKNEQAS scheme. However, user feedback consistently records high user satisfaction in the scheme, resulting in some laboratories being able to identify and correct systematic performance issues. These performance results are encouraging.
It has been noted that laboratories use a wide variety of toxoplasma tests; thus the sensitivity and specificity of toxoplasma-specific antibody tests must be considered in relation to the clinical context of the investigation. Some commercial screening tests may lack adequate sensitivity to identify patients who are truly immune to toxoplasma infection, while an assay that is too sensitive may be wrongly interpreted as confirming acute infection during pregnancy. Currently UKNEQAS does not include specimens for IgG avidity testing, although this development is under consideration.
In some instances laboratories failed to detect toxoplasma-specific antibody because participants deviated from the manufacturer's instructions, as evidenced by the range of titres and cut-off titres reported. The precision of testing and reporting could be improved by ensuring that all tests are performed strictly according to the instructions enclosed with the kit. In kits using doubling dilution assays, there is a normal distribution of results of ±1 dilution around the expected value. When the expected value is equal to the sensitivity limit of the assay, it is expected that a proportion of participants using that assay will obtain a negative result. Thus, further testing should be considered in cases where low concentrations of toxoplasma-specific antibody might be clinically relevant (eg, in immunosuppressed patients) and where early confirmation of toxoplasma infection is important in clinical management; this approach might include retesting the initial sample with a more sensitive assay.
There is a wide variety of assays available for the detection of toxoplasma-specific IgM.4 The different levels of sensitivity and specificity of these assays can make interpretation of the results difficult, and complicate patient management. Some ELISA assays typically detect IgM for 6–9 months after acute infection, others for up to 6 months, and ISAGA assays can remain positive for more than 12–15 months.4 Thus, different IgM tests are appropriate for use in different clinical situations and it is important that the results are interpreted with this information in mind. Highly sensitive IgM assays are required for detecting hyperacute infection or the specific immune response of a congenitally infected baby. Conversely, less-sensitive IgM assays are better suited to identifying infection acquired within a recent time frame (eg, 2–6 months). However, IgG avidity has been shown to be useful for further investigation when determining the likely clinical significance of an IgM-positive result.5
In general, non-specialist laboratories should screen samples for the presence of toxoplasma antibodies using a sensitive assay and then confirm any ‘positive’ reactions using a specific assay (or access the repertoire of assays performed in a reference unit). As with all laboratory investigations, toxoplasma-specific IgM tests produce some false-positive and false-negative results. Samples should be referred to a reference laboratory for confirmation when a screening IgM test produces a positive result, or when multiple complex investigations are required (eg, suspected congenital infection in a fetus or neonate).
In instances when it is noted that the discrepant results are caused by one test kit, UKNEQAS notifies the Medicines and Healthcare products Regulatory Agency, along with the kit manufacturers. This ensures that formal investigations are performed, kit manufacturers are encouraged to take corrective action, and participants are informed of the outcome. It should also be emphasised that the scheme organisers have never recommended that participants discontinue the use of any particular assay, but rather that they work collaboratively with the kit manufacturers.
We believe that the UKNEQAS toxoplasma serology schemes have been beneficial to clinical laboratories by ensuring the precision of routine testing, highlighting problem areas through the wide range of antibody titres distributed in the sera, and continuing professional education development associated with the teaching information.
Toxoplasma serology schemes schemes are unique in that they provide clinical histories with the sera to be examined, and these reflect the signs and symptoms of cases of suspected toxoplasmosis encountered by clinical laboratories.
Participants' feedback consistently records high user satisfaction in the scheme, resulting in some laboratories being able to identify and correct performance issues.
Participants in the schemes find the wide variety of antibody titres distributed and the additional interpretative clinical information supplied beneficial in facilitating their choice of kit for routine clinical use.
In instances when it is noted that the discrepant results are caused by one test kit, kit manufacturers are encouraged to take corrective action and participants are informed of the outcome.
Funding Professor Chiodini is supported by the UCL Hospitals Comprehensive Biomedical Research Centre Infection Theme.
Competing interests None to declare.
Provenance and peer review Not commissioned; externally peer reviewed.