Special reportFrom biomarkers to medical tests: The changing landscape of test evaluation
Introduction
There is an increasing awareness that the introduction of costly new medical interventions, including medical tests, can only be justified if they deliver proportionate benefits to patients. Increased public, media and political awareness of the harms from medical tests has come from debates about the potential for over-diagnosis in asymptomatic patients [1] and concerns about the harms of direct-to-consumer testing [2]. Weaknesses of the current systems to assure the quality and clinical utility of in vitro medical devices (IVDs) have been pointed out [3]. Amidst all this, the regulatory environment for therapeutic and diagnostic technologies is changing rapidly. Revisions of the European directives on medical [4] and in vitro diagnostic devices [5] are being prepared in parallel. The ability of novel medical tests to improve health outcomes is also becoming more central in discussions about their market entry and reimbursement. The increasing requirements for clinical benefits and patient safety mirror public and political pressures for more transparency. These processes are affecting the way novel medical tests and biomarkers are being developed, and are likely to reshape the landscape of medical test evaluation. Laboratory professionals need to be familiar with these evidence requirements to better engage in biomarker research and in clinical and policy decisions about the appropriate use of laboratory tests.
Over the past decade landmark advances have been made to define the types of evidence required to evaluate medical tests and distinguish between the different phases of test evaluation from discovery to assessment of cost-effectiveness [6], [7], [8], [9], [10]. There is less guidance, however, about the most efficient approaches to produce this evidence and judging whether it is adequate for proving the clinical effectiveness of biomarkers.
The evaluation of medical tests differs from comparable processes for therapeutic interventions. One of the most important differences is that medical testing rarely improves health outcomes directly. Testing is usually part of a more complex clinical pathway where test results guide treatment decisions, which include a variety of medical actions and processes. All of these shape the final health outcomes for the patients tested. Test evaluation therefore requires the consideration of all the consequences of clinical management decisions that are guided by the test results. An understanding of these more complex concepts for test evaluation is becoming essential for informed decision making by all potential stakeholders.
To help address these issues, the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) has formed a Working Group on Test Evaluation which consists of laboratory professionals, clinical epidemiologists, health technology assessment experts and representatives of the in vitro diagnostics industry. The primary purpose of this working group is to provide key stakeholders, i.e. laboratory professionals, clinicians, researchers, manufacturers, policy makers and purchasers, with guidance and practical tools for assessing the clinical benefits of in vitro medical tests. In this introductory paper, the working group outlines the key principles and defines some of the key components of contemporary approaches to test evaluation, such as analytical performance, clinical performance, clinical effectiveness, cost-effectiveness, and the broader impact of testing on social, psychological, legal, ethical, societal, organizational and other consequences. We additionally present a framework for the evaluation of medical tests that integrates these components into a dynamic process. We illustrate the key principles and components with examples from the literature on Hemoglobin A1c (HbA1c) and cardiac markers, including cardiac Troponins (cTn) and B-type natriuretic peptides (BNP).
Section snippets
Key definitions and principles
There is no international consensus on the terminology related to test evaluation and numerous definitions exist in the literature. Table 1 lists a number of alternative terms and illustrates the proposed definitions with examples.
Under the general umbrella term of medical tests, which encompasses tests from all clinical disciplines, specialties, or types (laboratory, histopathology, imaging, and others), we define and focus this paper on in vitro medical tests only; yet the key messages
Key components of test evaluation
Within the evaluation process of medical tests, we can distinguish between five essential components. Laboratory professionals are familiar with the assessment of analytical performance (Table 1). This refers to the ability of a laboratory assay to conform to predefined technical specifications [7], [20]. Specific elements of analytical performance include analytical sensitivity and specificity, limit of detection and quantitation, measurement range, linearity, metrological traceability,
The test evaluation framework
The sequence of activities that describe the pathway of newly discovered biomarkers from the “bench to the bedside” has been described in the literature [8], [9], [10], [21], [24]. A systematic review identified at least 19 different test evaluation frameworks [10]. The evaluation process for medical tests has been described in these frameworks in a linear fashion, similarly to the staged evaluation of drugs.
Only certain elements of test evaluation are addressed before in vitro medical tests
Conclusions
There is an increasing awareness that the introduction of new technology can lead to increased costs which are often not in direct proportion to the benefits for patients. This is particularly the case where new technologies offer only marginally better alternatives to existing processes. Healthcare funders and regulators are indicating that such marginal improvements are less likely to be funded or reimbursed in the future. Professional, public and government organizations have criticized the
Acknowledgments
The Working Group wishes to thank the European Federation of Clinical Chemistry and Laboratory Medicine for supporting its work. Roche Diagnostics is thanked for supporting the Working Group with an independent educational grant to EFLM.
Role of sponsor
Neither the funding organization (EFLM) nor the independent educational support by Roche Diagnostics influenced the content of this paper. This publication reflects the collective view of the working group. The authors include representatives from
References (43)
- et al.
The Evaluation of Genomic Applications in Practice and Prevention (EGAPP) Initiative: methods of the EGAPP Working Group
Genet Med
(2009) - et al.
Randomised comparisons of medical tests: sometimes invalid, not always efficient
Lancet
(2000) Offline: the scandal of device regulation in the UK
Lancet
(2012)- et al.
Preventing overdiagnosis: how to stop harming the healthy
BMJ
(2012) - et al.
Direct-to-consumer disease screening with finger-stick testing: online patient safety risks
Clin Chem
(2012) Assuring the quality of diagnostic tests. How do we know that they do what we think they do?
BMJ
(2013)Directive 2007/47/EC of the European Parliament and of the Council of 5 September 2007 amending Council Directive 90/385/EEC on the approximation of the laws of the Member States relating to active implantable medical devices, Council Directive 93/42/EEC concerning medical devices and Directive 98/8/EC concerning the placing of biocidal products on the market
Off J Eur Union
(21.9.2007)Directive 98/79/EC of the European Parliament and of the Council of 27 October 1998 on in vitro diagnostic medical devices
Off J Eur Communities
(December 12, 1998)- et al.
Comparative accuracy: assessing new tests against existing diagnostic pathways
BMJ
(2006) Methods guide for medical test reviews
AHRQ publication no. 12–EC017
(June 2012)
ACCE: a model process for evaluating data on emerging genetic tests
Proposals for a phased evaluation of medical tests
Med Decis Making
International vocabulary of metrology — basic and general concepts and associated terms (VIM) 3rd edition; 2008 version with minor corrections
JCGM
Biomarkers and surrogate endpoints: preferred definitions and conceptual framework
Clin Pharmacol Ther
Characterization of circulating endothelial cells in acute myocardial infarction
Sci Transl Med
What is value in health care?
N Engl J Med
What is a clinical pathway? Development of a definition to inform the debate
BMC Med
Guidelines and recommendations for laboratory analysis in the diagnosis and management of diabetes mellitus
Clin Chem
Third universal definition of myocardial infarction
Eur Heart J
High sensitivity troponin outperforms contemporary assays in predicting major adverse cardiac events up to two years in patients with chest pain
Ann Clin Biochem
Reflective testing: how useful is the practice of adding on tests by laboratory clinicians?
J Clin Pathol
Cited by (144)
Methods for determining clinical utility
2023, Clinical BiochemistryClinical utility – Information about the usefulness of tests
2023, Clinical BiochemistryEffects of R1-Rg1 on myocardial ischemia-reperfusion injury in mice through mitochondrial autophagy mediated by FUNDC1
2023, Current Research in BiotechnologyThe promise of omics in the precision medicine era
2022, Revue de Medecine InterneResearch on point-of-care tests in outpatient care in Germany: A scoping review and definition of relevant endpoints in evaluation studies
2022, Zeitschrift fur Evidenz, Fortbildung und Qualitat im GesundheitswesenMultidisciplinary perspectives on the regulation of diagnostic technologies
2022, Social Science and Medicine