Article Text
PDF
Abstract
Background There is growing interest in the potential utility of molecular diagnostics in improving the detection of life-threatening infection (sepsis). LightCycler® SeptiFast is a multipathogen probe-based real-time PCR system targeting DNA sequences of bacteria and fungi present in blood samples within a few hours. We report here the protocol of the first systematic review of published clinical diagnostic accuracy studies of this technology when compared with blood culture in the setting of suspected sepsis.
Methods/design Data sources: the Cochrane Database of Systematic Reviews, the Database of Abstracts of Reviews of Effects (DARE), the Health Technology Assessment Database (HTA), the NHS Economic Evaluation Database (NHSEED), The Cochrane Library, MEDLINE, EMBASE, ISI Web of Science, BIOSIS Previews, MEDION and the Aggressive Research Intelligence Facility Database (ARIF). Study selection: diagnostic accuracy studies that compare the real-time PCR technology with standard culture results performed on a patient's blood sample during the management of sepsis. Data extraction: three reviewers, working independently, will determine the level of evidence, methodological quality and a standard data set relating to demographics and diagnostic accuracy metrics for each study. Statistical analysis/data synthesis: heterogeneity of studies will be investigated using a coupled forest plot of sensitivity and specificity and a scatter plot in Receiver Operator Characteristic (ROC) space. Bivariate model method will be used to estimate summary sensitivity and specificity. The authors will investigate reporting biases using funnel plots based on effective sample size and regression tests of asymmetry. Subgroup analyses are planned for adults, children and infection setting (hospital vs community) if sufficient data are uncovered.
Dissemination Recommendations will be made to the Department of Health (as part of an open-access HTA report) as to whether the real-time PCR technology has sufficient clinical diagnostic accuracy potential to move forward to efficacy testing during the provision of routine clinical care.
Registration PROSPERO—NIHR Prospective Register of Systematic Reviews (CRD42011001289).
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.
Statistics from Altmetric.com
Article summary
Article focus
To describe the plans of a systematic review aimed at determining the diagnostic accuracy of a new real-time PCR technology (LightCycler® SeptiFast), designed to detect bloodborne pathogens in the setting of life-threatening infection (sepsis).
To highlight the unmet need for accurate and rapid infection diagnostics in the setting of life-threatening infection (sepsis).
Key messages
The study will provide the first independent systematic review of clinical validity studies of multiplex real-time PCR technology aimed at detecting circulating pathogen DNA straight from blood in the setting of suspected life-threatening infections (sepsis).
Based on the results of this study, independent recommendations will be made to the UK's Department of Health to help determine whether the real-time PCR technology has sufficient clinical diagnostic accuracy to move forward to efficacy testing during the provision of routine clinical care.
Strengths and limitations
The systematic review is focused on a single Conformité Européenne (CE)-marked real-time PCR technology designed for use in the setting of life-threatening infection (sepsis)
The systematic review is non-commercial and has been planned systematically by a multidisciplinary team of experts, working on behalf of the key stakeholders within a nationalised healthcare system.
Current clinical infection diagnostic reference standards may not have high diagnostic accuracy in all clinical settings and with all infections
Introduction
Sepsis is the clinical syndrome resulting from a host's systemic inflammatory response to infection.1 When severe, it is associated with considerable mortality and is a major international healthcare problem.2 Confirmation of sepsis requires objective evidence for infection,3 which should always include an attempt at microbiological identification of live pathogens from blood samples by culture techniques.3 4 However, culture routinely takes several days before a positive result is available and at least 5 days to determine that a specimen is culture negative.5 This temporal separation between initial clinical suspicion and confirmation of infection routinely results in the early and sustained delivery of potent broad spectrum antibiotics aimed at covering the most likely pathogens as a ‘safety first’ strategy because delay in appropriate antimicrobial therapy is associated with increased mortality.6 7 The inevitable consequence is unnecessary antibiotic prescription, which is associated with the development of antimicrobial resistance (eg, MRSA), Clostridium difficile infection as well as a range of avoidable adverse effects, and acquisition costs, of antimicrobial drug use.8 This is driven by a lack of access to time-critical high-specificity biomarkers of infection in critical care9 where overwhelming systemic inflammation of the body is a common occurrence and is often not caused by infection (it may, eg, be caused by trauma, blood transfusion, pancreatitis).10 There is therefore an urgent need to develop techniques that can provide accurate diagnostic information within hours of clinical signs appearing and so allow more informed use of antibiotic therapy at an early stage.
There is growing interest in the potential of real-time PCR technology to address this problem based on its ability to detect minute amounts of pathogen DNA in patient blood samples with results available within 4–6 h.11 Proof of concept studies have focused on two approaches using PCR for genomic amplification with either (1) broad range detection of bacterial or fungal DNA with universal primers, followed by species identification using a post-PCR technique such as gene sequencing or electrospray mass spectrometry or (2) using species-specific hybridisation probes that provide direct confirmation of the species present.12 Intuitively the latter approach would seem to have the greatest clinical utility assuming an appropriate pathogen panel can be established.
While the laboratory analytical sensitivity and specificity of these techniques for the detection of pathogen DNA in blood has been evaluated,12 there remains an acknowledged lack of clinical trial data to define the diagnostic reliability of such tests in patients who develop a systemic inflammatory response due to suspected infection. This has been due in part to the lack of standardised technology platforms that meet accepted regulatory standards for clinical diagnosis.
SeptiFast, manufactured by Roche Diagnostics, GmbH, Mannheim, Germany and run on their real-time PCR instrument (the LightCycler®), is the first real-time PCR-based system to be awarded a Conformité Européenne (CE) mark for pathogen detection and identification in suspected bloodstream infection12 13 and, to date, the most intensively investigated in clinical cohort studies. The system uses a multiplex approach, which allows detection of 25 of the most common pathogen species causing bloodstream infection in a single blood sample (table 1). Identification of the pathogens is based on the use of species-specific probes targeting the internal transcribed spacer region between the 16S and 23S areas of ribosomal DNA of bacteria and between the 18S and 5.8S ribosomal regions of the fungal genome. This real-time PCR technology has been extensively assessed at the laboratory level on clinical isolates and shown to have excellent analytical specificity and exclusivity, confirming its analytical validity.13 An EU registration study (unpublished), undertaken as part of the CE-marking process, investigated 278 critically ill patients with suspected sepsis from Denmark, Germany and Italy. Roche Diagnostics reported that the molecular test conferred a high diagnostic specificity and a 3- to 10-fold higher sensitivity when compared with conventional blood culture technology. Since this study, numerous commercial clinical diagnostic studies have been reported, predominantly focused on suspected sepsis. In addition, a large, independent, multicentre, level III clinical diagnostic accuracy study of this real-time PCR technology is currently recruiting14 as part of our detailed National Institute of Health Research (NIHR) Health Technology Assessment (HTA)-funded programme to assess real-time PCR technologies in sepsis diagnosis, treatment and outcome. A small number of newer emerging CE-marked PCR technologies are also currently available that can detect pathogen DNA rapidly in blood samples without the need for pre-culture, either using an all-pathogen approach without simultaneous speciation (eg, SepsiTest; Molzym GmbH, Bremen, Germany) or by using a multiplex approach based on a wide pathogen panel (eg, VYOO; SIRS-LAB GmbH, Jena, Germany). Compared with SeptiFast, the current number of clinical diagnostic studies using these technologies is very limited. Therefore, at this time, and as part of our independent HTA clinical studies,14 we describe here the protocol of a systematic review focused on the diagnostic test accuracy of LightCycler® SeptiFast for pathogen detection and identification in the blood of patients with suspected sepsis. This systematic review has been registered with PROSPERO—the NIHR International Prospective Register of Systematic Reviews (CRD42011001289).15
Pathogens detectable using the LightCycler® SeptiFast test
Methods
Inclusion criteria of studies
Participants
Patients suspected of developing sepsis, including adults and children, who require blood cultures irrespective of where their care is being delivered, and including suspected community or hospital acquired infection.
Target conditions
Sepsis, including severe sepsis and septic shock.16
Index test
LightCycler® SeptiFast as the index test on blood for the detection and identification of bacterial and fungal pathogens.13
Comparator test (reference standard)
Blood cultures used as the reference test and underpinning routine clinical practice.5 All diagnostic metrics will be reported using this reference standard. Clearly, there could be limitations to this standard, particularly in the setting of intercurrent antimicrobial therapy. However, we do not know the full extent of this problem or indeed whether studies have deliberately included or excluded such patients. In the absence of a straightforward and internationally agreed approach to an alternative reference standard at present, we believe that this is the most robust approach and is consistent with methods currently used in independent HTA clinical studies in this setting.14
Types of studies
We will include any clinical diagnostic accuracy study that compares the index real-time PCR test with standard culture results performed on a patient's blood sample during the management of sepsis.
Search methods for identifying studies
Electronic searches
We will search the Cochrane Database of Systematic Reviews, the Database of Abstracts of Reviews of Effects (DARE), the Health Technology Assessment Database (HTA), the NHS Economic Evaluation Database (NHSEED), The Cochrane Library, MEDLINE, EMBASE, ISI Web of Science, BIOSIS Previews, MEDION and the Aggressive Research Intelligence Facility database (ARIF). The CE mark for the index test was announced in January 2006; therefore, this systematic review will only consider publications from this date in humans. There will be no language restrictions in the electronic search for trials.
Search terms/search strategy
Specific search strategies will be developed for each electronic database, commencing with MEDLINE (table 2). The MEDLINE strategy will be adapted for each subsequent database and search yields reported and compared between databases.
MEDLINE search strategy
Other resources
Backward tracking will be performed by hand-searching the reference lists of all relevant articles uncovered from the electronic search and forward tracking using the keyword ‘SepiFast’ with ISI Citation Indices and Google Scholar and with a conference proceedings search using the Web of Science ISI Proceedings (2006 to present). We will request reference lists held by the only manufacturer of the index test (Roche Diagnostics) and include public-domain clinical diagnostic accuracy data collected by Roche Diagnostics to file for the CE mark. In addition, we will search for unpublished studies and ongoing trials involving the SeptiFast platform in the following online registers: http://www.nlm.nih.gov/hsrproj, http://www.controlled-trials.com/mrct/, http://portal.nihr.ac.uk/Pages/Portfolio.aspx and http://www.who.int/trialsearch, with identified corresponding authors of eligible trials and content experts contacted to identify potentially relevant studies and associated data.
Data collection and analysis
Selection of studies (Salford, UK)
The initial selection of titles and abstracts will be conduct by two review authors (CW and PD) independently using the inclusion criteria detailed above. The full papers of all abstracts deemed eligible (by any reviewer) will be obtained and read to determine their inclusion in the review. Disagreement at each step will be resolved with discussion between the two review authors (PD and CW) and a third author (GW).
Data extraction and management (Belfast and Warwick, UK)
A standard set of data will be extracted for each study using a tailored data extraction form which will include information regarding the inclusion criteria detailed above, an assessment of the level of evidence using the Oxford Centre for Evidence-based Medicine Levels of Evidence17 and additional information including: Study design; Clinical setting (ie, community, emergency department, in-hospital, critical care and general/specialist); Participant demographics; Clinical features of included population (illness aetiology); Intercurrent treatment (antimicrobial therapy); Reference standard methodology, including contamination rates; Supporting test results (culture of samples other than blood); Index test setting (point-of-care, near-patient, clinical or research laboratory, batched or individual analysis); Reported index test laboratory failures; Missing participant data; 2 by 2 table of results for primary outcome and reported diagnostic accuracy metrics; Follow-up (eg, survival and length of intensive care and hospital stay).
Three review authors (DFM, RM and GDP) will independently extract data, and any discrepancies will be resolved by discussion or if necessary by consultation with a fourth author (BB).
Assessment of methodological quality
Three independent authors (DFM, RM and GDP) will assess the quality of each individual study using the checklist (table 3) adapted from the QUADAS tool.18 Each question on the checklist will be answered with a yes/no response or noted as unclear if insufficient information is reported to enable a judgement to be made, and the reasons for the judgement made will be documented. Published standard operating procedures5 and interpretation of the reference standard (blood culture),5 including definitions of blood culture contamination, will be made available to the independent reviewers for reference. In addition, the 2006 CE-marked index test protocol will be provided to each reviewer as provided by Roche Diagnostics to purchasers.
Format of assessment of methodological quality adapted from QUADAS tool17
Statistical analysis and data synthesis
Data for 2×2 tables of index test against reference standard will be extracted from each study. Initially, these will be plotted as a coupled forest plot of sensitivity and specificity, and a scatter plot in ROC space (plotting sensitivity against 1 − specficity for each study). This will identify any issues of heterogeneity. Summary sensitivity and specificity will be estimated using the bivariate model method19 because the CE-marked index test is a semi-quantitative real-time PCR technique and reports results for the same threshold for positivity.
Investigations of heterogeneity
We will investigate the effects of patient characteristics (eg, aetiology of sepsis) and infection acquisition (community vs hospital) on test performance by incorporating covariates into the fitted models if sufficient individual studies are identified.
Sensitivity analysis
We will explore the potential effects of missing data using a range of assumptions. This will include the effects of both missing participant data from included studies and missing data from studies that did not provide data in a form that could be extracted and included in analysis.
Subgroup analysis
If adequate data are available, we will plan two subgroup analyses of adult versus paediatric populations and hospital versus community acquired infection.
Assessment of reporting bias
If there are sufficient studies included in analyses, we will investigate reporting biases using funnel plots based on effective sample size and regression tests of asymmetry, as recommended by Deeks and colleagues.20
Discussion
Blood culture technology is at the centre of evidence-based guidelines for the investigation and treatment of patients with sepsis. While culture has been refined over the last century, it remains insufficiently time critical and cannot assist with early management decisions, inevitably resulting in wasteful and potentially dangerous overtreatment with antimicrobial chemotherapy. PCR-based technologies have become standard laboratory techniques over the last two decades and could deliver real opportunity in terms of sensitivity and speed of pathogen detection in the clinical setting of life-threatening infection. LightCycler® SeptiFast is the first PCR-based system to be awarded a CE-mark for pathogen detection and identification in blood samples and, to date, is the most intensively investigated multiplex real-time PCR assay in the clinical setting of sepsis. The purpose of our planned systematic review is to determine, for the first time, the clinical diagnostic accuracy of this real-time PCR technology as part of an NIHR-funded HTA of this technology in the setting of potentially life-threatening infections. Based on the results of this non-commercial systematic review, independent recommendations will be made to the National Health Service providers as to whether LightCycler® SeptiFast has sufficient clinical diagnostic accuracy to move forward to efficacy and effectiveness testing during the provision of routine patient care.
Acknowledgments
We thank Mrs Helen McEvoy, formerly at the University of Manchester, School of Medicine Library, for her assistance with search strategy design and Mrs Valerie Haigh, Salford Royal NHS Foundation Trust Library and Information Resource Centre, for her advice on the use of filters in diagnostic search strategies.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data Supplement - Manuscript file of format pdf
Footnotes
To cite: Dark P, Wilson C, Blackwood B, et al. Accuracy of LightCycler® SeptiFast for the detection and identification of pathogens in the blood of patients with suspected sepsis: a systematic review protocol. BMJ Open 2012;2:e000392. doi:10.1136/bmjopen-2011-000392
Funding PD and GW are part-funded by the UK Health Technology Assessment (HTA) programme of the National Institute of Health Research grant number NIHR HTA 08/13/16. CW is funded by Greater Manchester Comprehensive Local Research Network in support of this programme of work. DFM and GDP receive funding from The Intensive Care Foundation (UK) in support of their roles as National Research Directors. None of these funding organisations or any commercial organisation have contributed to the study design; collection, management, analysis and interpretation of data; writing of the report or the decision to submit the report for publication. Only the named authors have ultimate authority over each of these activities.
Competing interests None.
Contributors PD and GW initiated the project, CW and BB worked together on the initial architecture for the review with specialist molecular diagnostic input from GW, critical care and clinical trial input from PD, DM and GDP, microbiological input from RM and statistical input from SG. PD drafted the protocol. All authors critically reviewed the first draft and contributed to the production of the final manuscript and its subsequent revision.
Provenance and peer review Not commissioned; externally peer reviewed.