Article Text

Download PDFPDF

Improving validation methods for molecular diagnostics: application of Bland-Altman, Deming and simple linear regression analyses in assay comparison and evaluation for next-generation sequencing
  1. Maksym Misyura1,
  2. Mahadeo A Sukhai1,
  3. Vathany Kulasignam2,3,
  4. Tong Zhang1,
  5. Suzanne Kamel-Reid1,3,4,5,
  6. Tracy L Stockley1,3,5
  1. 1 Advanced Molecular Diagnostics Laboratory, Princess Margaret Cancer Centre, University Health Network, Toronto, Ontario, Canada
  2. 2 Department of Clinical Biochemistry, Laboratory Medicine Program, University Health Network, Toronto, Ontario, Canada
  3. 3 Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Ontario, Canada
  4. 4 Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada
  5. 5 Department of Clinical Laboratory Genetics, Laboratory Medicine Program, University Health Network, Toronto, Ontario, Canada
  1. Correspondence to Dr Tracy L Stockley, Department of Clinical Laboratory Genetics, University Health Network, 11-454 Eaton Wing, Toronto General Hospital, Toronto, Ontario, Canada; Tracy.stockley{at}uhn.ca

Abstract

Aims A standard approach in test evaluation is to compare results of the assay in validation to results from previously validated methods. For quantitative molecular diagnostic assays, comparison of test values is often performed using simple linear regression and the coefficient of determination (R2), using R2 as the primary metric of assay agreement. However, the use of R2 alone does not adequately quantify constant or proportional errors required for optimal test evaluation. More extensive statistical approaches, such as Bland-Altman and expanded interpretation of linear regression methods, can be used to more thoroughly compare data from quantitative molecular assays.

Methods We present the application of Bland-Altman and linear regression statistical methods to evaluate quantitative outputs from next-generation sequencing assays (NGS). NGS-derived data sets from assay validation experiments were used to demonstrate the utility of the statistical methods.

Results Both Bland-Altman and linear regression were able to detect the presence and magnitude of constant and proportional error in quantitative values of NGS data. Deming linear regression was used in the context of assay comparison studies, while simple linear regression was used to analyse serial dilution data. Bland-Altman statistical approach was also adapted to quantify assay accuracy, including constant and proportional errors, and precision where theoretical and empirical values were known.

Conclusions The complementary application of the statistical methods described in this manuscript enables more extensive evaluation of performance characteristics of quantitative molecular assays, prior to implementation in the clinical molecular laboratory.

  • molecular pathology
  • cancer genetics
  • diagnostics
  • statistics
  • quantitation

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Footnotes

  • Handling editor Runjan Chetty.

  • Contributors MM designed experiments, performed analyses and wrote the manuscript. MAS designed experiments, performed analysis and wrote the manuscript. VK performed analyses and reviewed the manuscript. TZ designed experiments, collected data and reviewed the manuscript. SK-R designed experiments and reviewed the manuscript. TLS designed experiments and wrote the manuscript.

  • Funding The Princess Margaret Cancer Foundation and Genome Canada (Genomic Applications Partnership Program).

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.