Targeted therapy with trastuzumab (Herceptin) has had major impact on the adjuvant therapy of breast cancer. Trastuzumab is effective only in those breast cancers with amplified HER2 oncogene or showing overexpression of the HER2 protein; it has therefore recently become mandatory to assess the HER2 status of all newly diagnosed breast cancers to assess eligibility for the treatment. The commonly used algorithm for HER2 assessment is testing by immunohistochemistry (IHC) for all breast cancer samples. Fluorescent in situ hybridisation (FISH) is then used in the relatively small number of cases in which the IHC result is inconclusive (ie, IHC score 2+).

The diagnostic accuracy of HER2 assays has been a continuous concern. Recent data from the American Society of Clinical Oncology/College of American Pathologists (ASCO/CAP) indicates that as many as 20% of HER2 assessments performed by IHC (supplemented by FISH analysis of IHC 2+ cases) may be inaccurate.1 It has been reported several times that IHC is prone to producing a significant number of false-positive results, as well as a smaller number of false-negative results.2^–5

Although FISH is currently regarded as the gold standard HER2 test, the results of a pilot external quality assurance scheme in the UK6 have again indicated only modest reproducibility between the participating laboratories. While the reference laboratories performed FISH well, only 25% of the other participants were able to consistently achieve acceptable results. The sources of variability of FISH have not been clearly identified. While the FISH reagent kits, protocols and interpretation are well standardised, the microscopic evaluation of FISH is inherently difficult due to the fact that carcinoma cells can be difficult to identify in the tissue section with 4,6-diamidino-2-phenylindole (DAPI) counterstaining, which shows the cell nuclei but no tissue architecture. Tissue sections submitted to FISH analysis are often large and contain significant areas of non-malignant tissue, inflammation and ductal carcinoma in situ, all of which need to be distinguished from invasive carcinoma. This distinction can be made easily by trained pathologists, however, when a non-pathologist is involved in assessing the DAPI-stained FISH slides inaccuracies may occur. Difficulties in histopathological assessment have recently been cited as a possible cause for false-negative findings with FISH.7 Additionally there is the problem of non-specific fluorescence, as well as that caused by chromosomes duplicating during cell division and creating additional signals.

An alternative method for HER2 oncogene analysis is chromogenic in situ hybridisation (CISH). The principles of CISH and FISH are essentially the same, with the exception that the gene copies are detected using a peroxidase reaction instead of fluorochromes. The resulting slides are examined with brightfield microscopy, which allows the use of a haematoxylin counterstain and enables the HER2 signals to be interpreted within the histological context. In FISH, the probe is directly labelled with a fluorescent tag; however, CISH uses an amplification technique to enhance the diaminobenzidine signal, and this has the potential to produce artefacts. However, a number of studies have revealed 90–100% concordance between CISH and FISH in surveys of whole tissue sections and tissue microarrays.8^–19 A positive test result by CISH was used as a criterion for patient selection in one of the adjuvant trastuzumab trials.20

As a newer test CISH, is not yet widely utilised in diagnostic pathology laboratories in the UK, but has gained wide acceptance, for instance, in Greece, Portugal, Finland and Australia.21 22

Currently there are no established quality assurance programmes for CISH. Therefore in order to assess how well we and others were performing, as well as investigating how robust and reproducible the CISH analysis was, we decided to perform an inter-laboratory comparison. Seven participating laboratories from six different countries (UK, Italy, Portugal, Australia, Greece and Finland) all of whom use CISH on a routine basis, were asked to analyse the test slides using their CISH protocol exactly as they would in routine diagnostic use. The results were compared with each other and with HER2 IHC and FISH data where available.

METHODS

Samples

Tissue samples, from 30 breast cancer cases, that had been routinely processed, formalin-fixed and paraffin-embedded in the Department of Histopathology at the Royal Surrey County Hospital (RSCH) were used for the study. All specimens were surgical specimens of invasive ductal carcinoma of the breast.

Two cases were excluded from the subsequent analysis because there was insufficient tumour tissue for analysis by the majority of the centres. Twenty-five of the remaining 28 samples had had their HER2 status assessed previously at a reference laboratory (ie, a laboratory performing >1000 IHC tests/year and >100 FISH tests/year) following the UK recommendations for HER2 testing in the UK.23

CISH analysis was performed on all the tissues in the first instance at RSCH during a validation study to assess the performance of CISH against IHC and FISH.19 The IHC and FISH data were obtained from reference laboratories that had been responsible for the routine HER2 testing at RSCH prior to CISH being available.

For this survey we selected those cases from our previous study that had shown IHC/CISH or FISH/CISH discrepant data (n = 10),19 additional cases were chosen on the basis that they fell into the non-amplified/equivocal/low-level-amplified category (n = 5) as this was known to be the most difficult category to assess, other cases were chosen because we had FISH data for them (n = 8), while the rest were a selection of CISH-amplified and non-amplified cases (n = 5). Twelve of the cases had been scored as IHC 2+ and 11 as 3+ (table 1).

Concordance study

Sections (4 μm) were cut from each block onto charged slides. Three slides were sent to each of the participating laboratories. Each laboratory was asked to perform CISH analysis according to their routine protocols. Reagent kits from Invitrogen/Zymed (San Francisco, California, USA) or ZytoVision (Bremerhaven, Germany) were used. One laboratory (University of Tampere) used an in-house digoxigenin-labelled BAC clone for HER2 and the Powervision+ kit for probe visualisation. Chromosome 17 (C17) analysis was performed, as recommended, on all samples in which 5–10 HER2 signals were seen.

The results from each of the centres were sent to RSCH and the data were collated and analysed.

Interpretation

Each laboratory used their routine interpretation criteria which followed these basic principles: any invasive tumour with <5 HER2 copies per cell in >50% cancer cells was classified as not amplified (“No amp” in table 1). Those with 5–10 copies of HER2 were called amplified (“Amp” in table 1) only if the C17 analysis showed no C17 polysomy. Any tumour with >10 copies of HER2 per cell or a mixture of dots or small or large clusters of signals in >50% cancer cells was recorded as amplified (Amp) for HER2.

Amplified and not amplified cases are considered “positive” and “negative” respectively for Herceptin therapy eligibility. Amp in table 1 includes low- and high-level amplification, both of which were specifically reported by several of the centres.

At least three separate areas of the tumour were assessed to overcome problems of heterogeneity.

Statistical methods

In order to assess the concordance of the results between the centres we used two methods: a crude agreement measure and the Fleiss’ κ analysis, which is a statistical measure of inter-rater reliability for multiple observers.24

We interpreted the κ value according to the categories proposed by Landis and Koch25: <0, poor agreement; 0–0.20, slight agreement; 0.21–0.40, fair agreement; 0.41–0.60, moderate agreement; 0.61–0.80, substantial agreement; 0.81–1.00, almost perfect agreement.

RESULTS

In the 28 cases tested in this study 192/195 (98.5%) of the results obtained were concordant. Of the 12 IHC 2+ cases, six were found to be amplified and six not amplified. Only three discrepant results were recorded (tables 1 and 2), indicating a 1.5% incidence of possible misdiagnoses. It should be noted that two centres were unable to perform C17 centromere analysis on all cases because insufficient numbers of slides were available.

Case 22 was reported as amplified by five centres but not amplified by centre 3 and not assessable (NA) by centre 5. Case 13 was reported as not amplified by six centres but amplified by centre 3, and case 17 was reported as not amplified by six centres but amplified by centre 4.

All three discrepant cases had previously given an IHC score of 2+. Case 13 was subsequently given a dual-colour FISH ratio of 1.16 (ie, not amplified) (although the FISH result a year previously had been reported as 2.55), and case 22 had a FISH ratio of 13.8 (ie, amplified), both of which concur with the majority view in this study. However, case 17 was given a ratio of 2.6 by FISH (ie, amplified), while six out of seven of the centres in this study scored this case as not amplified.

The overall agreement among the seven centres was 98.5%. Using Fleiss’ statistical analysis, we obtained a κ coefficient 0.91, indicating almost perfect agreement among the seven participating centres.

DISCUSSION

This study shows that HER2 CISH, as it is currently used in routine diagnostics, has very high reproducibility between testing laboratories. Only three results were found to be discordant, indicating a misdiagnosis rate of 1.5%. This is less than has been reported in a study using FISH (4.2%).6

One of the discrepancies was due to the invasive carcinoma having been cut out in the sections received by two of the centres (case 22); one centre identified this problem while the other did not. The second discrepant case, case 13, was reported as non-amplified by six centres but amplified by centre 3 (with remarks of low level amplification). This centre presumably mis-counted the HER2 signals in the tumour cells and interpreted this as low-level amplification with normal C17. However, the true nature of this case must remain doubtful since there were also two discordant FISH results for this sample (from two experienced FISH testing laboratories), with gene copy ratios of 1.16 and 2.55.

The final discrepant case, case17, was reported by the majority of centres to have up to five HER2 signals in the nuclei of the tumour cells and this would qualify the tumour as not amplified. Centre 3 mis-counted the HER2 signals as >5 and therefore recorded the case as HER2 amplified. Earlier FISH analysis of case 17 had reported a HER2:C17 ratio of 2.6, indicating gene amplification. However, as already noted above and reported by others,6 discrepancies between laboratories in cases that fall within the equivocal gene copy number also occur in FISH analysis. The IHC result in this case was 2+, which favours the view that this patient was unlikely to benefit from trastuzumab-based therapy. The recent report from ASCO/CAP suggests that the cut-off level for HER2 signals be raised to 6 before calling the case unequivocally amplified.1

Case 26 in this study was specifically chosen as it had proved to be a very difficult case when included in our original validation study.19 It had been assessed at an experienced HER2 testing centre and classified as IHC 2+ and subsequently analysed by FISH and given a HER2:C17 ratio of 7.69. CISH analysis indicated that this case was not amplified using the CISH cut-off value of <5 signals. The same case was subsequently re-analysed by FISH in another laboratory and given an equivocal result of HER2:C17 ratio 2.2. This case was subsequently assessed by CISH in seven different laboratories, with all of them reporting it as not amplified. It must also be noted that this case proved technically challenging, requiring more than one attempt for in situ testing to be successful, when assessed with chromogenic or fluorescent analysis.

The inter-laboratory reproducibility of CISH was nearly perfect in this study, even though the case selection favoured difficult, equivocal cases. This is in stark contrast to similar studies of the reproducibility of IHC in which up to 50% of IHC results were found to be discordant.3 5 Therefore it would seem that the objective method of counting signals is less prone to error than the subjective assessment of depth of colour. However even signal-counting assays can produce errors, but it would appear from this study that CISH may be less prone to these than the “gold standard” FISH.6

Suboptimal performance of the various tests for HER2 have led not only to doubt and uncertainty in the minds of clinicians but also to the judgment from ASCO/CAP that there is no ideal test for HER2 assessment in breast cancer.1 However, one advantage of CISH over FISH is that it provides a permanent histological record allowing retrospective re-examination of the slide if necessary (eg, if the recommendation for trastuzumab eligibility changes). For example, if the recent data that suggest that cases with an increased number of HER2 gene copies due to an increase in the number of copies of C17 (polysomy) may partially respond to Herceptin therapy is found to be important.26

According to current knowledge, response to trastuzumab is best predicted by the presence of a gene amplification27 28 and very recently it has been reported that the level of amplification significantly affects this response to Herceptin therapy.29 As accurate determination of the gene copy number is vital, any role played by the C17 status can be addressed by the up-coming dual-colour CISH. This being the case, the widespread use of IHC can only be explained by its relatively cheap cost and its easy incorporation into the laboratory routine as compared with the technical complexity and skills required for FISH analysis.

Recently an automated version of CISH using silver as the chromogen (termed SISH) has become commercially available. While automated protocols may provide faster turnaround and easier standardisation, it is unlikely that they will improve the quality of the assay since near-perfect concordance has been found between laboratories using manual CISH protocols routinely.

The results presented in this study indicate a high level of concordance between laboratories using CISH routinely for diagnostics. However, the need for external quality control schemes cannot be over-emphasised.22 While the CAP has just added such a test to its portfolio of assessments offered, there are currently no specialised external quality control schemes for brightfield HER2 CISH available in Europe.