Article Text

Download PDFPDF

External quality assessment of BRAF molecular analysis in melanoma
  1. Zandra C Deans1,
  2. Andrew Wallace2,
  3. Brendan O'Sullivan3,
  4. Andrew Purvis1,
  5. Suzanne Camus4,
  6. Jennifer A Fairley4,
  7. David Gonzalez5
  1. 1Department of Laboratory Medicine, UK NEQAS for Molecular Genetics, The Royal Infirmary of Edinburgh, Edinburgh, UK
  2. 2Department of Genetic Medicine, Saint Mary's Hospital, Manchester, UK
  3. 3Department of Cellular Pathology, Queen Elizabeth Hospital, Birmingham, UK
  4. 4Department of Laboratory Medicine, Molecular Pathology, The Royal Infirmary of Edinburgh, Edinburgh, UK
  5. 5Molecular Diagnostics, The Centre for Molecular Pathology, The Royal Marsden NHS Foundation Trust, Surrey, UK
  1. Correspondence to Dr Zandra Deans, Department of Laboratory Medicine, UK NEQAS for Molecular Genetics, The Royal Infirmary of Edinburgh, Little France Crescent, Edinburgh EH16 4SA, UK; Sandi.Deans{at}ed.ac.uk

Abstract

The availability of BRAF inhibitors has given metastatic melanoma patients an effective new treatment choice and molecular testing to determine the presence or absence of a BRAF codon 600 mutation is pivotal in the clinical management of these patients. This molecular test must be performed accurately and appropriately to ensure that the patient receives the most suitable treatment in a timely manner. Laboratories have introduced such testing; however, some experience low sample throughput making it critical that an external quality assurance programme is available to help promote a high standard of testing, reporting and provide an educational aspect for BRAF molecular testing. Laboratories took part in three rounds of external quality assessment (EQA) during a 12-month period giving participants a measure of the accuracy of genotyping, clinical interpretation of the result and experience in testing a range of different samples. Formalin fixed paraffin embedded tissue sections from malignant melanoma patients were distributed to participants for BRAF molecular testing. The standard of testing was generally high but distribution of a mutation other than the most common, p.(Val600Glu), highlighted concerns with detection or reporting of the presence of rarer mutations. The main issues raised in the interpretation of the results were the importance of clear unambiguous interpretation of the result tailored to the patient and the understanding that the treatment is different from that given to other stratified medicine programmes. The variability in reporting and wide range of methodologies used indicate a continuing need for EQA in this field.

  • Melanoma
  • Molecular Pathology
  • Quality Assurance

Statistics from Altmetric.com

Introduction

Malignant melanoma is the fifth most common cancer type in the developed world, and the third most common in Australasia. The incidence of melanoma is rapidly increasing mostly due to environmental factors such as sun exposure, with a predicted increase of 50% in the next 20 years in the UK.1 Although the majority of malignant melanoma cases can be radically excised with a very low risk of recurrence, approximately 20% of patients will develop metastatic disease, requiring systemic treatment.1 Until very recently, the treatment options for malignant melanoma patients were very limited but with the introduction of targeted therapy the options have expanded significantly. The initial success of the phase I study with vemurafenib in patients with BRAF V600 mutant melanoma was further confirmed in the randomised control phase III trial (BRIM3), showing a fourfold increase in progression-free survival with BRAF-inhibitor therapy compared with dacarbazine.2 These data led to The Food and Drug Administration, European Medicines Agency and National Institute for Health and Care Excellence approval in 2011–2012 of vemurafenib for first-line treatment of advanced metastatic malignant melanoma with BRAF V600 mutations. More recently, new trials have shown the activity of a combination of BRAF and MEK inhibition as a very successful therapeutic choice for BRAF V600 mutant patients, increasing the choice of treatment for metastatic patients.3

The inception of molecular testing determining patient treatment, commonly referred to as molecular pathology, has resulted in demand from laboratories for appropriate external quality assessment (EQA).4–6 Since 2008, the United Kingdom National External Quality Assessment Service (UK NEQAS) has provided molecular pathology exercises and from 2010 the collaborative group UK NEQAS for Molecular Pathology (comprised of UK NEQAS for Molecular Genetics and UK NEQAS for Immunocytochemistry and In situ hybridisation) has focused on personalised medicine molecular testing EQA schemes.4–6 The widespread adoption of molecular testing for BRAF codon 600 mutations in melanoma samples instigated the provision of a pilot EQA scheme which assessed and monitored the genotyping accuracy of testing laboratories over a period of 12 months. Three assessments were performed, each requiring the testing of three clinical cases. The scheme aimed to improve and standardise the interpretative content of the laboratory reports through feedback comments and educational scheme reports. The samples distributed for the scheme were included to deal with issues particular to melanoma testing, for example, potential inhibition of the PCR assay by melanin present in the sample,7 which was addressed by distributing malignant melanoma tissue sections to participants for testing.

Methods

Validation

The EQA material was independently validated by two laboratories prior to use in the scheme by a range of technologies. The first and last sections from each tumour block were tested by both validating laboratories and furthermore these laboratories also tested internal sections as part of the EQA runs. Validating Laboratory 1 tested for the presence of BRAF codon 600 mutations by cobas BRAF V600 mutation test (Roche Molecular Systems, Pleasanton, California, USA) and capillary-electrophoresis single-strand conformation analysis which also detects non-V600 mutations in exon 15 of BRAF. Validating Laboratory 2 tested the specimens using Sanger sequencing, the cobas BRAF V600 mutation test and an inhouse pyrosequencing method. The tumour content was estimated by both laboratories. Table 1 summarises the validated results for the EQA samples distributed.

Table 1

Validated results for EQA samples distributed

The EQA was administered using the UK NEQAS for Molecular Genetics website and participation was open to all interested laboratories. Each participating centre was assigned a unique laboratory identifier in order to encrypt laboratory identity and ensure impartiality of the assessment process. Only scheme staff had access to laboratory codes. The UK NEQAS Molecular Pathology EQA Specialist Advisory Group was responsible for the delivery of the EQA service and was advised by the UK NEQAS Steering Committees for Molecular Genetics and Cellular Pathology Techniques to ensure that both the molecular genetics and histopathology aspects of the testing were addressed and needs met.

For each EQA run, three clinical case scenarios with corresponding formalin fixed paraffin embedded (FFPE) tumour sections were distributed for routine testing. Participants requested their sample preference: 2×5 µm rolled FFPE tumour sections, 2×5 µm rolled FFPE tumour sections plus 1×5 µm slide mounted FFPE tumour section or 3×5 µm slide mounted tumour sections. A 4-week timescale was allowed for testing. For the first EQA run, submissions were assessed for genotyping accuracy only and laboratories were provided with assessor feedback comments for report interpretative content. For the two subsequent EQA runs, scores were assigned for genotyping accuracy, interpretation of the result and clerical accuracy of the report. In line with other molecular pathology EQAs each category was scored out of a total of 2.00 marks. All EQA returns were reviewed independently by two assessors and scored against peer ratified marking criteria tailored to the clinical case scenarios. The assessment teams comprised senior clinical scientists/biomedical scientists from both histopathology and molecular genetics disciplines with extensive experience in BRAF mutation testing. A scheme report was published for each EQA run summarising the data gathered and detailing issues arising from that run.

Results

In all, 54 laboratories participated in run 1 (one did not submit any results as they were in the process of setting up the assay but submitted results for run 2), a further six laboratories joined for the second EQA run (a different laboratory did not submit results for run 2 and has since failed to communicate with the scheme) and 64 laboratories participated in run 3.

Figure 1 summarises the sample type requested by participants demonstrating that in the first two runs the most requested sample type was slide mounted FFPE tumour sections and slightly more laboratories requested rolled sections plus one slide for testing for run 3.

Figure 1

Graph summarising the sample type in the three external quality assessment runs.

Table 1 summarises the BRAF genotypes distributed. There were no BRAF genotyping errors in run 1 although one laboratory reported a KRAS test result for case 1. For run 2, 14 (14/59) laboratories did not correctly report the presence of the c.1789_1799delinsAA mutation p.(Val600Lys). Table 2 summarises the errors and methods used. Of the 14 laboratories, three did not report the presence of a BRAF codon 600 mutation; two of these used pyrosequencing and one used the Roche cobas assay. Overall, 10 of these 14 laboratories incorrectly reported the mutation as c.1799T>A p.(Val600Glu): of these, three used pyrosequencing, two real time PCR, one Sanger sequencing and one mass spectrometry. The remaining three laboratories used a combination of methods (see table 2). One laboratory performed the test using the Qiagen Rotor-Gene Q (RGQ) PCR kit and although this does detect the c.1789_1799delinsAA mutation p.(Val600Lys), this laboratory only reported that c.1799T>A p.(Val600Glu) was not present. The assessors agreed that this was an incomplete test and the options for treatment of the patient would have been affected and consequently the result was classed as a genotyping error. All laboratories reported the correct genotype result for EQA cases 5, 6, 7, 8 and 9, although deductions of a 0.5 mark were taken from four laboratories in case 7 due to the use of non-Human Genome Variation Society (HGVS) compliant mutation nomenclature. One of these laboratories also received a further 0.5 mark deduction for not stating all of the mutations covered by the kit used. They only stated the kit would detect the p.Val600Glu mutation even though other codon 600 mutations would have been detected.

Table 2

Case 4 genotyping errors and methods used

There was a range of methodologies used by participating laboratories (figure 2), many using a combination of two or more. This may be for the purposes of the EQA scheme or for internal test validation/verification and not reflective of routine practice. The most commonly used method was direct Sanger sequencing with pyrosequencing and Roche cobas assay the second and third most frequently used methods (the cobas test being more frequently used in the most recent EQA run). Following feedback from run 1, all laboratories stated their testing methodology on the submitted reports.

Figure 2

Graph displaying the different methodologies used by participating laboratories in the three external quality assessment runs.

There was considerable variability in the terminology used to report a mutation. For case 4, the description of the mutation was analysed (see table 3 for summary). Of the 45 laboratories correctly reporting the result, 14 laboratories described the mutation correctly using HGVS guidelines,8 11 using the c.1798_1799delinsAA nomenclature and three laboratories using c.1798_1799delGTinsAA version. A further 14 laboratories used c.1798_1799GT>AA which is a mutation name frequently used in the literature but does not conform to HGVS guidelines and should not be used. One laboratory reported the mutation as two separate mutations, c.1798G>A and c.1799T>A. One laboratory reported the mutation correctly at the amino acid level but described it incorrectly at the nucleic acid level as c.1797T>A. No marks were deducted from the EQA scores for these participants but feedback was given. Fifteen laboratories did not use mutation nomenclature but described the sequence with statements such as ‘presence of a BRAF codon 600 mutation’ (or similar) and did not provide any further information as to which mutation had been detected. Some methods performed do not characterise the mutation and if such methods are used then this should be clearly stated in the report. Eight of the laboratories included a clear statement that the method performed could not characterise the mutation (six used the Roche cobas assay and two performed real time PCR assay). The remaining seven laboratories also used the Roche cobas assay but did not report clearly that the mutation could not be fully characterised. No EQA marks were deducted but feedback comments were given.

Table 3

BRAF mutation nomenclature used for case 4 when reporting correct result

There was a wide variation in the level of interpretation provided on the EQA reports. Run 1 did not assign EQA scores for interpretation but the assessors provided feedback to individual laboratories. Scores were assigned for runs 2 and 3 according to the peer reviewed criteria which required statements regarding the likelihood of the patient's response to BRAF-inhibitor therapies, and for cases where no codon 600 mutation was detected then the test sensitivity and an indication of the percentage of BRAF mutations that were covered by the assay performed. Many laboratories did not include this information and were marked down accordingly for runs 2 and 3.

Many reports included errors in the patient identifiers, for example, name and date of birth and these were deducted marks in the clerical accuracy category. Other reports contained numerous typographical errors in the main text and although marks were not deducted the reports were unprofessional in appearance and could lead to a loss of confidence by the reader in the result being reported. Table 4 summarises the scheme mean of the scores for all three categories.

Table 4

Summary of scheme mean scores for genotyping accuracy, interpretation of the results and clerical accuracy of the report

Discussion

To give laboratories a measure of the standard of service provided to service users it is important to assess the whole process involved in the testing of the sample, from sample receipt in the laboratory, tumour content assessment (if performed), molecular testing and the reporting of the result in the context of the clinical case. The way our scheme has been designed examines the whole testing process. The provision of three distributions of a pilot external quality assurance scheme by UK NEQAS for BRAF molecular testing in metastatic melanoma has demonstrated a high level of genotyping accuracy by laboratories, in particular for the common p.(Val600Glu) mutation. However, the content and interpretive aspects of the reports were highly variable between laboratories.

It is important that both the sensitivity of the assay performed and the coverage of mutations tested are provided enabling the reader of the report to interpret the result appropriately and schedule further tests if required. This is particularly critical when no mutation is detected. Also the estimated percentage of tumour content or a statement that the tumour content had not been assessed should be included in the report. Some laboratories used the terminology ‘negative’ or ‘wild type’; however, the assessors recommend the use of the term ‘no mutations detected’ to avoid misinterpretation or misunderstanding of the result. Similarly, the assessors recommend avoidance of the terminology ‘positive’ for a case when a mutation has been detected. Some laboratories failed to give any clinical interpretation of the test result for the patient and although the assessors are aware of differences in local and country specific practices, the assessors felt that some interpretation should be provided in the report.

When a report states the presence of a mutation and/or the mutation is named, then it is important that an appropriate reference sequence and version number are provided. Reference sequences do change and can result in a change in the numbering of codons. For example, the BRAF reference sequence has changed over recent years with p.(Val600Glu) previously being called p.(Val599Glu). Consequently, it is important that a reference sequence is quoted in the report so the details of the mutation can be interpreted in the correct context.

It is recommended to avoid using drug names in clinical reports as licencing of drugs can vary between countries and more drugs may be available in the future. Furthermore, the patient report should not recommend the use of a particular drug. Depending on the decision made by the clinical teams, patients may be enrolled into experimental clinical trials where the BRAF mutational status may be very useful, for example, other BRAF inhibitors or MEK inhibitors. It is important to remember that several clinical and biological factors in addition to BRAF mutational status may alter treatment and laboratories are not aware of the full clinical details of every patient; therefore, statements such as ‘the patient is eligible for BRAF inhibitors’ are discouraged in favour of comments such as ‘the patient may benefit from BRAF-inhibition therapies’ or ‘specific BRAF inhibitors could be considered for treatment in this patient’.

Approximately 20% of malignant melanoma samples with a BRAF mutation carry a ‘non-p.(Val600Glu)’ mutation; in particular, p.(Val600Lys) may be present in 10%–15% of cases. Tumours with these rarer codon 600 mutations still have the capacity to respond to BRAF inhibitor therapies; therefore, laboratories should be aware of the risks of potentially denying therapy to patients who may otherwise benefit from it by the failure to detect or report on the presence of other BRAF codon 600 mutations.

Encouragingly, many laboratories had amended their reports based on feedback from the first EQA run, in particular the fact that BRAF is not a tyrosine kinase but a serine/threonine-protein kinase. Comments on the therapeutic use of TKIs had been replaced in run 2 reports with more relevant statements along the lines of ‘This patient may benefit from BRAF-inhibitor therapies’ or ‘This patient is unlikely to benefit from BRAF-inhibitor therapies’.

By the very nature of EQA, measurement of the laboratory service at a single point of time is only obtained but continual participation allows a laboratory assurance of the level of service provided in both the analytical and clinical reporting elements of BRAF testing. Through UK NEQAS, participating laboratories are assessed three times per year, therefore allowing rapid identification of any service problems, providing an educational aspect, help, support and giving participants a source of validated material (and less common mutations) which they may not otherwise obtain through routine testing. Given this, there is a continuing need for external assessment in this field for all laboratories testing patient samples.

Conclusions

EQA of molecular testing for BRAF codon 600 mutations over a 12-month period has indicated a high standard of genotyping accuracy in the participating laboratories; however, the testing and identification of rarer codon 600 mutations can be an issue in some centres. Clear reporting and interpretation of the results are critical in delivering a high standard of testing and ultimately a high standard of patient care. External assessment of the full testing pathway gives an impartial measure of the level of service being delivered.

Take-home messages

  • The United Kingdom National External Quality Assessment Service (UK NEQAS) scheme for BRAF molecular testing in melanoma has shown a high standard of testing by laboratories.

  • There is variation in the detection and reporting of non-p.(Val600Glu) BRAF codon 600 mutations.

  • Participation in external quality assessment schemes provides laboratories with a measure of the standard of their testing and is a mechanism by which educational advice and reference material can be supplied to laboratories.

Acknowledgments

The authors would like to thank all the laboratories for their participation in the UK NEQAS scheme for BRAF testing in metastatic melanoma and the UK NEQAS for Molecular Pathology Specialist Advisory Group for their support and guidance. We thank the scheme assessors for all their work and the Molecular Pathology Laboratory, The Royal Infirmary of Edinburgh, the Molecular Diagnostics Laboratory, The Royal Marsden NHS Foundation Trust and the Department of Cellular Pathology, Queen Elizabeth Hospital, Birmingham, for help with sample preparation and validation.

References

Footnotes

  • Contributors ZCD had the idea for the article, planned, conducted and reported the work. ZCD, AW and DG prepared the manuscript. ZCD, AW, BOS, AP, SC, JAF and DG were involved in sample sourcing, validation, distribution and assessment.

  • Funding The sourcing of some of the material used in the EQA runs was funded by an educational grant from Roche Products. No funding parties had any input into scheme design, assessment or access to any specific participant data.

  • Competing interests ZCD and DG report grants from Roche Products during the conduct of the study; ZCD, AW, BOS and DG have received personal fees from Roche Products during the conduct of the study. ZCD reports personal fees from Astra Zeneca, grants from Qiagen and grants from Astra Zeneca outside the submitted work. AW reports personal fees from Astra Zeneca and Boehringer Ingelheim outside the submitted work. BOS reports personal fees from Astra Zeneca outside the submitted work. DG reports personal fees from Roche Products, GSK and Roche Molecular Systems/Roche Diagnostics outside the submitted work.

  • Patient consent No.

  • Provenance and peer review Not commissioned; internally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.