Design and validation of a next generation sequencing assay for hereditary BRCA1 and BRCA2 mutation testing

DNA samples

The Counsyl Inherited Cancer Screen validation study was conducted by testing three classes of samples: (a) deidentified blood samples (N = 57), (b) deidentified paired blood and saliva samples (7 pairs), and (c) genomic DNA reference materials obtained from Coriell (N = 57), including the well-characterized NA12878 sample from HapMap/1000 Genomes and 15 samples from the BIC BRCA1/BRCA2 Mutation Panel (Table 2 and Table S1). The protocol for this study was approved by Western Institutional Review Board (IRB number 1145639) and complied with the Health Insurance Portability and Accountability Act (HIPAA). The information associated with patient samples was de-identified in accordance with the HIPAA Privacy Rule. A waiver of informed consent was requested and approved by the IRB.

Test design

The reportable range of the test is all coding exons of BRCA1 (NM_007294.3) and BRCA2 (NM_000059.3), 20 bp into the introns from intron/exon junctions, and selected intronic and untranslated regions where pathogenic variants have been reported in the literature (Table S2).

Next generation sequencing

DNA from a patient’s blood or saliva sample is isolated, quantified by a Picogreen fluorescence assay and then fragmented to 200–1,000 bp by sonication. The fragmented DNA is converted to an adapter-ligated sequencing library by end repair, A tailing, and barcoded adapter ligation; samples are multiplexed and identified by molecular barcodes. Hybrid capture-based enrichment with 40-mer oligonucleotides (Integrated DNA Technologies, Coral, IL, USA) complementary to BRCA1/2 targeted regions is performed on these multiplexed samples. Next generation sequencing of the selected targets is performed with sequencing-by-synthesis on the Illumina HiSeq 2500 instrument to mean sequencing depth of ∼500×. All target nucleotides were required to be covered with a minimum depth of 20 reads.

Bioinformatics processing

Generated sequence reads are aligned to the hg19 human reference genome using the BWA-MEM algorithm (Li, 2013), which also trims sequencing adapters. Automated statistical analysis is used to identify and genotype single-nucleotide variants (SNVs) and short insertions and deletions (indels) following methods in GATK 1.6 and FreeBayes (Garrison & Marth, 2012; McKenna et al., 2010). The calling algorithm for copy number variants (insertions or deletions longer than 100 bp) is described below. All SNVs, indels, and large deletions/duplications within the reportable range are analyzed and classified by the method described in the section “Variant classification.” All reportable calls are reviewed by licensed clinical laboratory personnel.

CNV calling algorithm

Our method for CNV calling is based on high-resolution depth of coverage analysis and performed in a manner similar to that successfully used by other groups (Nord et al., 2011; Judkins et al., 2015; Lincoln et al., 2015).

Analysis is performed on a per-lane basis. The region of interest for the assay is grouped into a number of regions for which copy number is counted (e.g., exons); each exon is considered independently and with no smoothing (e.g., HMM). Define matrix d_i,j to be the matrix containing the number of reads from sample i overlapping with region j. This matrix must be normalized. To protect against normalization issues due to individual samples with very large CNVs (such as a whole-gene deletion), we generate a normalization matrix n_i,j by removing the highest variance probes from the total data set D via the invariant set method described in Li & Hung Wong (2001). The data matrix d is then normalized in two steps: ${\displaystyle \begin{array}{c}{d}_{i,j}^{\prime }=d_{i,j}∕\text{mean}\left(n_{i,j}\text{for all}j\right)\hfill \\ d_{i,j}^{″}=d_{i,j}^{\prime }∕\text{mean}\left(n_{i,j}\text{for all}i\right).\hfill & \hfill & \hfill & \hfill \end{array}}$

For each putative CNV j in sample i, a hypothetical copy number and corresponding Z-score is computed: ${\displaystyle \begin{array}{c}{c}_{i,j}=2\ast d{″}_{i,j}\hfill \\ z_{i,j}=\left(d{″}_{i,j}-\text{mean}\left(d{″}_{i,j}\text{for all}i\right)\right)∕\text{stdev}\left(d{″}_{i,j}\text{for all}i\right).\hfill & \hfill & \hfill & \hfill \end{array}}$ A CNV call is considered confidently non-reference if abs (z) ≥ 4 and the estimate c is <1.2 or >2.8.

Assay quality metrics

To ensure the quality of the results obtained from the Counsyl Inherited Cancer Screen, documentation and QC systems (Table S3) were developed in the Counsyl CLIA (Clinical Laboratory Improvement Amendments)-certified laboratory. Ancillary quality-control metrics, including amount of DNA recovered from a specimen (≥18 ng/ul), fraction of sample contamination (<5%), unreliable GC bias, read qualities (percent Q30 bases per Illumina specifications), depth of coverage (per base target coverage >20×), are computed on the final output and used to exclude and re-run failed samples.

Variant classification

Variants are classified according to the ACMG Standards and Guidelines for the Interpretation of Sequence Variants (American College of Medical Genetics and Genomics, 2015). All variants that are known or predicted to be pathogenic are reported; patients and providers have an option to have variants of uncertain significance reported as well. Final variant classifications are regularly uploaded to ClinVar.

Statistical analysis

Validation metrics were defined as: Accuracy = (TP + TN)/(TP + FP + TN + FN); Sensitivity = TP/(TP + FN); Specificity = TN/(TN+FP); FDR = FP/(TP + FP), where TP = true positives, TN = true negatives, FP = false positives, FN = false negatives, and FDR = false discovery rate. The confidence intervals (CIs) were calculated by the method of Wilson (1927).

Assay development

The Counsyl Inherited Cancer Screen we have developed employs next-generation sequencing and includes comprehensive analysis of all coding exons of BRCA1 and BRCA2, 20 bp of flanking intronic sequences, and selected intronic and untranslated regions with known pathogenic variants (Table S2). The test was designed and optimized to detect single-nucleotide variants, indels, and copy-number variants. A proprietary bioinformatics pipeline was developed for sequence data alignment and variant detection as described above. To test the analytical sensitivity, specificity and accuracy of the assay, we sequenced peripheral blood and cell line DNA from 114 samples with an extremely high mean read depth (more than 500×) across all samples. Every targeted position was covered with a minimum of 20 reads.

Analytical validation

To establish analytical accuracy for detecting single-nucleotide variants and indels, we compared Counsyl BRCA1/2 sequence data of 41 Coriell samples (listed in Table S1) to reference data obtained from the 1,000 Genomes project and Counsyl BRCA1/2 sequence data for NA12878 to high-quality reference data published by Illumina, Inc. (http://www.illumina.com/platinumgenomes/) (Table 2). The results presented in Table 3 demonstrate that 536 true positive calls, 12,920 true negative calls and no false positive or false negative calls were observed. In addition, to confirm the detection of documented variants in BRCA1/BRCA2, 15 samples from the BIC BRCA1/BRCA2 Mutation Panel (available from Coriell) were included in the validation (Table 2 and Tables S1 and S4). The concordance between the BRCA1/2 mutations detected by the Counsyl ICS and the BIC reference data was 100%.

Furthermore, to demonstrate the accurate detection of variants that are technically challenging for NGS, 10 pathogenic indels discovered in patient blood samples were subjected to orthogonal confirmation by Sanger sequencing (Tables 2 and 4; Table S4). All the indels detected by the Counsyl Inherited Cancer Screen were concordant with the Sanger results.

To establish analytical accuracy for detecting CNVs, we compared Counsyl copy number calls to CNV calls provided by reference labs, when available, and MLPA assays otherwise, on 63 samples: 13 samples from reference labs; 15 samples from the BIC BRCA1/BRCA2 Mutation Panel; 25 random blood samples; 9 patient samples positive for CNVs; and NA12878 (Table 2). As shown in Table 5, 79 true positive calls, 3,067 true negative calls and no false positive or false negative calls were observed from the analysis of 63 analyzed samples (Table 5). Among the 63 tested samples, 10 had a deletion or duplication of a single exon, which can be technically challenging for NGS-based analysis.

The accuracy, sensitivity and specificity are therefore all 100% for SNPs, indels, and copy number variants. Only samples with reference data for the entire region of interest were used to calculate these metrics in order to avoid overestimating sensitivity or the confidence interval (McAdam, 2000). We also calculated false discovery rate (FDR) and the associated confidence interval (Tables 3 and 5). For SNPs and indels, the FDR is 0 of 536 positives, or 0% (95% CI [0–0.7%]). For CNVs, the FDR is 0 of 79, or 0% (95% CI [0–4.6%]).

Our validation samples represent a diversity of variant subtypes (Table 4 and Table S4), including 28 samples with variants technically challenging to detect by NGS, such as large indels and CNVs.

Inter-run and intra-run reproducibility

In addition to establishing the test analytical sensitivity, specificity and accuracy, the Counsyl BRCA1/2 test was validated for intra- and inter-run reproducibility. For indel detection reproducibility, each BIC sample (n = 15) was run three times each on three flow cells, for a total of nine replicates (Table S5). For SNV detection reproducibility, 11 deidentified blood samples were rerun on 2–3 different flow cells (Table S5). For CNV detection reproducibility, 15 Coriell cell line DNA and 11 patient samples were analyzed in replicates (Table S5). Concordance between replicates was 100% (Table S5), with no differences between inter- and intra-run replicates observed.

Test compatibility with different input materials

Finally, to demonstrate compatibility with different sample types, deidentified paired blood and saliva samples (seven pairs) were tested. The results from paired blood and saliva samples were 100% concordant (Table S5).