Background The absence of high-quality next-generation sequencing (NGS) reference material (RM) has impeded the clinical use of liquid biopsies with plasma cell-free DNA (cfDNA) in China.
Objective This study aimed to develop a national RM panel for external quality assessment and performance evaluation during kit registration of non-small-cell lung cancer (NSCLC)-related Kirsten rat sarcoma viral oncogene (KRAS)/neuroblastoma ras oncogene (NRAS)/epidermal growth factor receptor (EGFR)/B-type Raf kinase (BRAF)/mesenchymal–epithelial transition factor (MET) genetic assays using plasma circulating tumor DNA (ctDNA).
Methods Mutation cell lines detected by NGS and validated by Sanger sequencing were selected to establish the RM. Cell line genomic DNA was sheared and used to spike basal plasma cfDNA at 10% concentration. Then, the calibration accuracy was determined by four sequencing platforms. Average values were adopted and diluted to 0.1%, 0.3%, 1% and 3% concentrations with basal plasma as the RM panel. Then, five manufacturers were invited to evaluate the performance of the RM panel.
Results 20 cell lines with 23 clinically important mutations were selected, including six mutations in KRAS, two mutations in NRAS, three in BRAF, four in phosphatidylinositol-4,5-bisphosphate 3-kinase catalytic subunit alpha (PIK3CA), six in EGFR, one EGFR Gain (4-5 copy) and one MET Gain (2-5 copy). The RM panel consisted of 87 samples, including these 21 mutations at four concentrations (0.1%, 0.3%, 1% and 3%), one MET gain, one EGFR gain and one wild type. The detection rate was 100% for the 3%, 1% and 0.3% samples at all five companies. For the 0.1% concentration, 15 samples had inconsistent results, but at least three companies had correct results for each mutation.
Conclusion RM for a KRAS/NRAS/EGFR/BRAF/MET mutation panel for plasma ctDNA was developed, which will be essential for quality control of the performance of independent laboratories.
- molecular biology
- pathology, molecular
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Lung cancer is the most common incident cancer and is the leading cause of cancer death in China.1 There were 520 000 new lung cancer cases and 631 000 lung cancer deaths in China in 2015. First-line platinum doublet chemotherapy is the backbone of treatment for patients with newly diagnosed metastatic non-small-cell lung cancer (NSCLC).2 However, treatment of an unselected population leads to low response rates and modest survival effects, highlighting the need for target therapy, which significantly improves survival and reduces side effects, especially in NSCLC.3 Targeted therapies are currently approved for EGFR mutations, anaplastic lymphoma kinase (ALK) and ROS proto-oncogene 1 (ROS1) gene rearrangements and BRAF mutations,4 with the list of emerging ‘actionable’ targets growing.
The molecular basis of lung cancer is complex and heterogeneous. In lung cancer, tumourigenesis involves the activation of growth-promoting genes such as EGFR, KRAS, BRAF, meiosis-specific serine/threonine protein kinase 1 (MEK1), hairy-related 2 (HER2), MET, ALK and ret proto-oncogene (RET), and the inactivation of tumour suppressor genes such as tumour suppressor p53 (TP53), phosphatase and tensin homolog deleted on chromosome 10 (PTEN) and liver kinase B1 (LKB1).5 The list of mutations with approved targeted therapies includes EGFR mutations (in 30%–40% of Asians and 10%–20% of Caucasians),6 7 ALK rearrangements (in up to 6%),8 ROS1 rearrangements (1%)9 and BRAF mutations (in 2%).10 Other targets with therapies that are not approved for use in NSCLC include HER2, RET, MET and neurotrophin receptor kinase (NTRK). Thus, the precise detection of gene mutations at an early stage is of great benefit in patient treatment. With many approved targeted therapies, performing multiplex testing is extremely important in NSCLC. Virtually all patients who respond to frontline EGFR tyrosine kinase inhibitor therapy develop resistance. The most common mechanism of acquired resistance is the EGFR exon 20 p.T790M mutation (≈50% of patients). Osimertinib is a US Food and Drug Administration (FDA)-approved drug for treating acquired EGFR exon 20 p.T790M resistance.11
Tumour biopsies are the standard for obtaining tumour DNA; however, biopsies cannot always be obtained, and their interpretation may be confounded by intratumour heterogeneity,12 13 which could lead to false-negative results and suboptimal therapy selection. Furthermore, tissue-based tumour profiles provide only a snapshot of tumour heterogeneity and cannot be obtained repeatedly, are not suitable for monitor response to treatment, and assess the emergence of drug resistance.14 Graph-based methods usually require comparison of images before and after cancer therapy, a technique that cannot be used for selecting patient treatment. Therefore, it is necessary to develop a novel non-invasive method to predict the effectiveness of cancer therapy. The presence of cell-free nucleic acids (cfDNA) in human blood was first described in 1948.15 Elevated cfDNA was first found in patients with cancer in 1977.16 In 1989, Stroun et al reported that at least some cfDNA in the plasma of patients with cancer originates from cancer cells.17 This circulating tumour DNA (ctDNA) is mainly released through apoptosis, with some from secretion or necrosis.18 The analysis of ctDNA is likely to play a major role in personalised cancer therapy by enabling the non-invasive genotyping of clinically actionable tumour-derived variants to guide therapy selection19 20 and to detect emergent, actionable variants associated with resistance during targeted therapy.21–23 Personalised cancer therapy based on plasma ctDNA assessment as companion diagnostics have obtained FDA approval for patients with advanced stage NSCLC (EGFR mutational assessment in treatment naïve patients when tissue is not available or inadequate for molecular analysis, and in resistance setting to a first or second generation tyrosine kinase inhibitor in order to detect the EGFR exon 20 p.T790M). Analysis of ctDNA is also helpful when tumour material is unavailable and for monitoring disease progression and relapse.
Allele-specific PCR kits have been approved for clinical use in detecting hot-spot mutations in serum and plasma,24 but these have limited analytical sensitivity. Digital PCR (dPCR) assays using microfluidic platforms are quantitative and very sensitive, but it is difficult to develop a multiplex assay that avoids the need to split samples for multiple reactions. They are nonetheless suited in investigating a small number of mutations and are often applied to the analysis of cancer hot-spot mutations. Based on targeted sequencing using PCR amplicons or hybrid capture, next-generation sequencing (NGS) is used to detect more loci. NGS can also detect amplifications, deletions and fusion in a single test,25 and promises to improve diagnostic yield.
Plasma ctDNA genotyping based on NGS can enable convenient detection of clinically actionable mutations in patients with NSCLC. Many independent laboratories have developed in-house NGS panels and offer services to patients, and dozens of companies have submitted registration requirements. However, the interpretation of NGS data remains challenging owing to the low concentrations, complexity of the genome and technical errors that are introduced during sample preparation, sequencing and analysis. These errors can be mitigated by using reference standards. However, the absence of high-quality reference material (RM) has impeded the detection of KRAS/NRAS/EGFR/BRAF/MET gene variants in plasma cfDNA based on NGS in China. In this study, we established a set of RMs to evaluate the performance of companion diagnostics to target therapy for NSCLC. These RMs are suitable for external quality assessment and performance tests during kit registration procedures for oncogene mutation tests using plasma cfDNA based on NGS, dPCR and real-time fluorescence qualitative PCR.
Material and methods
In principle, NGS-based plasma ctDNA test can be used for targeted therapies in patients with NSCLC on different sequencing platforms. We designed the RM set which includes different concentrations of common variants in EGFR, KRAS, NRAS, BRAF and PIK3CA, and a MET amplification (figure 1). All of these mutations have been frequently targeted clinically and reported in the literature, and play roles in anticancer therapeutics.
Sample collection and DNA extraction
Cancer genomes can easily be obtained from cell line culture and can be mixed with matched cell lines to simulate blood plasma. Here, 20 cell lines containing target mutations were selected for RM preparation and then purchased from the American Type Culture Collection (ATCC) (Manassas, Virginia, USA). The mutations were determined by NGS and validated by Sanger sequencing and fluorescence in situ hybridisation (FISH).
Genomic DNA was extracted using the QIAamp DNA FFPE Tissue Kit (QIAGEN, Valencia, California, USA) according to the manufacturer’s instructions. The quality of all samples was checked using a Thermo NanoDrop 2000c (Life Technologies, Carlsbad, California, USA). The concentrations were recorded with Qubit V.3.0 (Life Technologies, Carlsbad, California, USA).
Preparation of 10% mutation samples and accuracy calibration
First, 998 healthy controls aged 23–45 years were recruited to prepare basal plasma. Informed written consent was obtained from each control who agreed to participate in this project. Then, 50–200 mL of peripheral blood was collected in EDTA tubes from each of healthy control. Plasma was separated with a two-step centrifugation protocol and pooled as mixed plasma samples. In detail, peripheral blood was collected into EDTA-containing blood tubes and centrifuged at 1600×g for 10 min at 4°C. The plasma was transferred and centrifuged again at 16 000×g for 10 min at 4°C to remove residual cells. The plasma aliquots were then carefully transferred to clean tubes. Then, cfDNA was extracted from 200 µL of plasma using the QIAamp DSP Circulating Nucleic Acid Kit (Qiagen, Hilden, Germany) and quantified. Only plasma not including the reference mutations was selected; the remaining plasma was mixed as the basal plasma preparation and stored at −80°C before using.
We added mutation DNA fragments to the mixed plasma samples in a DNA ratio of 1:9 and called these the 10% mutation plasma samples. Other reference samples were prepared using the same protocol.
Five companies (BGI Genomics, Geneplus-Beijing, Berry Genomics, Annoroad Gene Technology and CapitalBio) were invited to conduct calibration accuracy tests of the 10% mutation samples with four sequencing platforms (BGISEQ-500, Ion Proton Torrent, NextSeq 500 and MGISEQ-2000). DNA libraries were constructed according to individual modified protocols. The average concentration was set as the defined concentration.
RM preparation and performance tests
The analysis of tumour DNA in a patient’s blood offers a non-invasive way to detect cancer and monitor responses to therapy. Unfortunately, the sensitivity of circulating DNA analysis is limited by the amount of tumour DNA in the blood and by the methods of detection.26 Target therapy would benefit from improvements in analytical sensitivity as long as high specificity was maintained. The samples were diluted in basal plasma to prepare 3%, 1%, 0.3% and 0.1% RMs. Then, 4.2 mL aliquots of the RM plasma samples was put in 5 mL tubes and stored at –80°C. Basal plasma was used as a negative control. Each plasma aliquot was frozen and thawed only once.
Five participating centres (BGI-Shenzhen, Shenzhen, China; Geneplus Technology, Beijing, China; Nanjing Geneseeq Technology; Amoy Diagnostics; and Genetron Health, Beijing) using four NGS platforms (MGISEQ-2000, NextSeq 550AR, NextSeq 500 and Ion Proton Torrent S5) were invited to assess the accuracy and specificity of detecting the oncogene variants in the RMs. To test homogeneity, BGI-Shenzhen tested three cfDNA RMs; the other participants tested one each. The detection methods followed standard operating procedures for the relevant oncogene variant detection kits of each laboratory.
The χ2 test was used at a significance level of 5% to compare the detection rate among NextSeq 550AR, NextSeq 500, MGISEQ-2000 and Ion Proton Torrent.
Preparation, validation and quantification of mutation cell lines and basal plasma
We prepared 23 mutations in 20 cell lines to produce ctDNA RM that included six mutations in KRAS (exon 2 p.G12D, exon 2 p.G12C, exon 2 p.G12A, exon 2 p.G13D, exon 2 p.G12S and exon 2 p.G13C), two in NRAS (exon 3 p.Q61K and exon 3 p.Q61R), three in BRAF (exon 15 p.V600E, exon 11 p.G469A and exon 15 p.L597V), four in PIK3CA (exon 10 p.E542K, exon 10 E545K, exon 21 p.1047R and exon 21 p.1047L), seven in EGFR (exon 20 p.T790M, exon 18 p.G719S, exon 21 p.L858R, exon 19 p.E746_A750del, exon 19 p.L747_E749del, exon 19 p.E746_S752>I and 4-5 copy EGFR Gain) and one 2-5 copy MET Gain. Three cell lines had two mutations: NCI-H1975 had exon 20 p.T790M and exon 21 p.L858R in EGFR; NCI-H2087 had exon 3 p.Q61K in NRAS and exon 15 p.L597V in BRAF; and HCC287 had exon 19 p.E746_A750del in EGFR and the 45-copy EGFR Gain. The details of the mutations are presented in table 1. All mutations were detected by NGS and validated by Sanger sequencing or FISH. Genomic DNA was prepared, digested and recycled. Most of the sheared DNA was around 160 bp. The DNA concentration (1.82–12.16 µg) and recovery rate (2.8%–53.7%) differed greatly among the 20 cell lines, which might have been caused by differences in the total numbers and morphology of cells.
Quality control of the performance of independent laboratory NGS panels and performance testing in the kit registration process have a lot of demand for the RMs in China. In the present study, 150 L of basal plasma was collected and pooled. cfDNA was extracted and tested by NGS and dPCR, plasma without RM mutations was selected for RM preparation. The cfDNA concentration of the prepared basal plasma was 6.5 ng/mL.
Preparation and quantification of the 10% mutation standards
To prepare the 10% mutation standards, sheared genomic DNA from 20 cell lines was diluted with the basal plasma cfDNA for the 23 mutations. Online supplementary table 1 gives the details of the amount of DNA and conformity degree of preparation for each cell line. To quantify the actual allelic frequency of the 23 mutations in the prepared 10% standards, a NGS assay was performed. The variant frequency ranged from 7.78% (MET Gain) to 14.12% (exon 21 p.L858R in EGFR). The actual allelic frequencies differed slightly from the expected frequencies, but were stable.
To adjust the RM fits with different NGS platforms, the 10% mutation standards were distributed to five manufacturers to evaluate the allelic frequencies using their in-house NGS panels on four platforms: NextSeq 500 (Illumina), BGISEQ-500 (BGI), Ion Proton Torrent (Thermo Fisher) and MGISEQ-2000 (MGI). The allelic frequencies were obtained from the five participants, and the average was used as the frequency for the series of concentrations of prepared RM. While the allelic frequencies differed significantly among the participants, the average showed higher conformity (5.4%~12.96%). The detailed results are shown in table 2.
Performance of the RMs
The RMs consisted of 87 samples, including 21 mutations at four concentrations (0.1%, 0.3%, 1% and 3%), one MET gain, one EGFR gain and one wild type (online supplementary table 2). The RMs were distributed to five manufacturers to evaluate the performance of their in-house NGS-based cancer kits. Four participants received one set of 87 RMs, and one received three sets of RMs to test their uniformity. No significant differences were observed among the replicated measurements of each mutation result, except for inconsistency due to the failure of NGS testing among the 0.1% RMs (online supplementary table 3). These results indicate that the RMs have good consistency.
The results for the five participants with the 3%, 1% and 0.3% allelic frequencies were consistent with the theoretical results (figure 2), except for one 0.3% RM sample for which one participant could not construct a library. These results indicate that 0.3% RMs can be determined stably.
At 0.1% concentration, the five participants failed to detect 15 RMs. Participant 1 failed to detected three RMs (KRAS exon 2 p.G12C, NRAS exon 3 p.Q61R and EGFR exon 19 p.E746_S752>I). Participant 2 failed to detect 11 RMs (KRAS exon 2 p.G13C, BRAF exon 15 p.V600E, BRAF exon 11 p.G469A, BRAF exon 15 p.L597V, PIK3CA exon 10 E545K, PIK3CA exon 21 p.1047R, EGFR exon 18 p.G719S, EGFR exon 21 p.L858R, EGFR exon 19 p.E746_A750del, EGFR exon 19 p.L747_E749del and EGFR exon 19 p.E746_S752>I). Participant 3 failed to detected seven RMs (KRAS exon 2 p.G12C, NRAS exon 3 p.Q61K, NRAS exon 3 p.Q61R, BRAF exon 11 p.G469A, PIK3CA exon 10 E545K, EGFR exon 18 p.G719S and EGFR exon 19 p.L747_750>P). Participant 4 failed to detect six RMs (KRAS exon 2 p.G13C, NRAS exon 3 p.Q61K, NRAS exon 3 p.Q61R, BRAF exon 15 p.L597V, PIK3CA exon 10 E545K and PIK3CA exon 21 p.H1047L). Participant 5 detected all of the mutations except one 0.3% RM sample for which no library was constructed. Each 0.1% concentration RM was detected correctly by at least three participants. For EGFR and MET amplification, one participant’s kit did not contain this analysis, and the other four participants gave correct results. Therefore, except for failure to construct one sample library, all mutations were reported correctly for the 3%, 1% and 0.3% concentration RMs (online supplementary table 4). No mandatory requirements were set for the 0.1% concentration RMs.
Increasingly, NGS is being applied to clinical diagnosis because it can analyse DNA or RNA sequences in a single test and provide qualitative and quantitative information, thereby promising improved diagnostic yield for determined mutations, insertions/deletions and fusions in a single assay.27 The NGS process includes sample preparation, library construction, sequencing and bioinformatic analysis, which increase the risk of both false-positive and false-negative diagnoses. Many of these errors can be mitigated with the use of reference standards. However, unlike hereditary disease caused by germline mutations, it is difficult to establish stable, renewable, well-characterised RM for cancers. Tumour samples are typically small and finite, and do not provide a ready source of RM. Heterogenetic tumour samples also make it difficult to develop homogeneous RM.28 Cell line cultures can provide cancer genomes and can be mixed with matched cell lines to simulate blood plasma.29 Here, we use selected tumour-derived cell lines to prepare RM. The cell lines selected were (1) readily acquired and (2) included common cancer mutations that were (3) included in commercial registration kits. Based on these principals, 20 cell lines that included 23 mutations in six genes (EGFR, KRAS, NRAS, BRAF, PIK3CA and MET) were selected.
NGS of ctDNA promises to facilitate personalised cancer therapy.30 Two different liquid biopsy diagnostic tests for EGFR mutations in plasma ctDNA have been approved by regulatory agencies in Europe and the USA for selecting patients with NSCLC for anti-EGFR treatment. The important factor in ctDNA detection is the low amount of cfDNA and ctDNA in blood.31 Circulating cfDNA is primarily germline DNA that originated from normal cells, with a relatively small, highly variable fraction of ctDNA originating from tumour cells. The ctDNA levels are usually positively related to the cancer stage and tumour volume and can be very low in some patients. This requires high-sensitivity detection methods. Newman et al have developed an ultrasensitive method (0.02%) for quantifying ctDNA in patient’s plasma.32 33 In the present study, four concentrations of RM were assessed (0.1%, 0.3%, 1% and 3%). Two important factors underlying the detection limits of all ctDNA profiling methods are the number of cfDNA molecules that are recovered and the number of mutations being examined in a patient’s tumour. High sequencing coverage can overcome the errors caused by DNA damage and sequencing errors.34 However, systematic sequencing errors cannot be overcome by high coverage sequencing.
Although it is a national plasma cfDNA reference materials develop study carried in multicentre, multisequencing platform for NGS panel testing, there are still several weaknesses that need to improve in the future. First, due to the limitation of real positive samples, we have to simulate the positive samples using fragmented genomic DNA with health control plasma. Second, it is reported that a reassessment of the raw data in a second-look analysis can increase detection rate for the low allele frequency mutations.35 36 However, our RMs are designed for the quality control of the performance of independent laboratory NGS panels and performance testing in the kit registration process which cannot permit a reassessment process. Third, since each laboratory uses its own sequencing platform and mutation bioinformatic analysis pipeline, the algorithms for mapping and variants interpretation would affect the results of detection rate.
Increasingly, NGS is being applied in clinical diagnosis. Reference standards can improve the accuracy, reliability and standardisation of clinical diagnosis based on NGS technologies. However, most reference standards have been used to benchmark workflow. Here, we developed RM for detecting KRAS/NRAS/EGFR/BRAF/MET mutations in plasma ctDNA. These are NSCLC-related mutations that are suitable for target therapy. The RM can also be used for quality control of the performance of independent laboratory NGS panels and will facilitate performance testing in the kit registration process in China.
Take home messages
Present study developed a national RM panel for external quality assessment and performance evaluation of non-small-cell lung cancer related genetic assay using plasma ctDNA.
The present method of preparing plasma ctDNA reference materials are helpful to the preparation of other reference materials for liquid biopsy.
Present study can provide specifications for the quality and performance of liquid biopsy kits based on different NGS platforms.
Handling editor Runjan Chetty.
JX, SQ and NS contributed equally.
Contributors JH, XY and XC designed and supervised the study. JX, SQ and NS analysed and interpreted the data and prepared the manuscript. WZ, QS, ML, WG, QZ, MH, CN and RX carried the test, collected and analysed the data. All authors reviewed and approved the final version and agreed to publish the manuscript.
Funding This study was supported by the National Key Research and Development Program (2017YFC0906500 and 2016YFC1000301) and the Science and Technology Program of Guangdong (Grant Nos. 2019B020208009 and 2017A040405043).
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval The study was approved by the institutional review board of National Institutes for Food and Drug Control (NIFDC).
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available from the corresponding author by reasonable request.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.