Materials and methods

TUMOURS CIRCULATED BY THE EQA SCHEME

Laboratories participating in the UK NEQAS-ICC programme for steroid hormonal receptors were sent, at each assessment, two unstained slides containing histological tissue sections of formalin fixed and paraffin wax processed breast tumours showing different degrees of hormonal expression. Each participant was asked to demonstrate ER and/or PR and to return the stained slide(s) to the UK NEQAS coordinating centre for assessment of staining quality. Table 1 shows details of the tumours circulated at each assessment from April 1994 to June 1998. Although tested in the organisers laboratory, most of these tumours were fixed and processed in the laboratories of participants, from where they were kindly donated. Whenever possible, tumours that also had their receptor status determined biochemically by the ligand binding assay (LBA) were used.

IN HOUSE TUMOURS

In parallel with the immunostaining of slides circulated by the scheme, participants were asked to demonstrate the same receptor on their own in house tumour, and to submit this stained slide for assessment of staining quality, along with two additional unstained tissue sections from the same tissue block. Tables 2 and 3 give examples of the types of tumours submitted by participants and the various fixatives used. These unstained slides were then stained by the UK NEQAS organising laboratory along with its routine workload, utilising its routine methodology and reagents, for the demonstration of hormonal receptors (table 4). They were then coded and filed alongside the participants own immunostaining before routine assessment. The UK NEQAS-ICC organising centre also stained the participants' in house tumours so that a comparison could be made between this staining and that produced by the participant on the same case.

ASSESSMENT OF SLIDES

An expert panel of four, comprising consultant pathologists and biomedical and clinical scientists, assessed the quality of the IHC independently on a single blind basis, with each assessor awarding marks out of 5 for each of the coded slides. The four individual marks were then added together to give a total mark out of 20. Marks were awarded by comparing the proportion and intensity of tumour nuclei staining in the participant's slide, to that achieved on duplicate sections of the same cases by the UK NEQAS-ICC organising centre. A total mark > 12 out of 20 indicates acceptable immunostaining and a pass at assessment, a mark of 10–12 out of 20 is considered to be suboptimal and borderline, whereas a total mark < 10 out of 20 is given for staining that is of unacceptable quality and represents a failure at assessment. One of the main criteria by which staining is deemed unacceptable is when < 10% of receptor positive tumour nuclei are clearly demonstrated in a tumour that has been shown by the UK NEQAS-ICC organising centre to express > 10% ER or PR positive nuclei. To ensure assessor concordance, within ± 1 mark, the slides were marked in batches of 20. On the marking of the 20th slide, all scores were read out, when there was a difference of greater than 1 mark between any of the assessors' individual marks, the respective slide was reviewed until a consensus was reached—that is, all the assessors gave the same mark, within ± 1 mark.

For a selected assessment (run 41), the participants in house staining and that achieved by the organising centre on sections from the same tumours was assessed using the semiquantitative quick score method of evaluation.^12–14 With this method, the intensity of the immunohistochemical reaction as viewed under the light microscope was recorded as either: 0, negative (no staining of any nuclei even at high magnification); 1, weak (only visible at high magnification); 2, moderate (readily visible at low magnification); or 3, strong (strikingly positive even at low power magnification). The proportion of tumour nuclei showing positive staining was also recorded as either: zero (0), approximately 1–25% (1), 26–50% (2), 51–75% (3), or 76–100% (4). The score for intensity was then added to the score for proportion, giving the quick score with a range of 0–7. In the case of composite blocks (n = 17), the tumour showing the lowest amount of expression was assessed using this method, and where participants had submitted normal breast tissue, these were not submitted for quick score evaluation. Before the assessors' (AR, BJ) evaluation of these slides, all were randomised using numbers generated by a Microsoft Excel program. Determination of the degree of concordance between the quick scores of the participants' immunostaining of their own in house tumour and the quick scores of the UK NEQAS laboratory's immunostaining of the same tumour was established using Cohen's κ coefficient. The proportion of cases that showed the same or a higher degree of expression when stained by the organising laboratory was analysed by means of the χ² test, as was the proportion of cases that showed the same or a higher degree of expression when stained by the participant's laboratory. The degree of assessor concordance when using the quick score method to evaluate stained slides was measured using Goodman and Kruskal's γ statistic.

VALIDATION OF ASSESSMENT THRESHOLDS AND OPTIMAL SENSITIVITY

Of the participating laboratories, six were identified as having published studies clinically validating their technique. These studies are not referred to here because this would identify the laboratories concerned, and in so doing transgress the UK NEQAS code of practice, which confers anonymity to all participants.¹⁵ The proportion of these laboratories achieving staining comparable with that of the UK NEQAS organising centre on the tumours circulated at assessment by the scheme and on their own in house tumours was determined for all the assessments between April 1994 and June 1998. For the in house tumours, the Wilcoxon two sample matched pairs signed ranks test was also used to test for differences in distributions between the marks awarded to the expert centres and those awarded to the UK NEQAS organising laboratory for staining of the same tumours.

COMPARISON OF THE SCORES AWARDED TO PARTICIPANTS FOR STAINING OF IN HOUSE TUMOURS AND THE STAINING OF UK NEQAS-ICC TUMOURS

The UK NEQAS scores for all the assessments conducted for ER and/or PR between April 1994 and June 1998 were compared by the use of box plots and by establishing the proportion of participants achieving acceptable staining at each assessment on the two sets of slides. The Wilcoxon two sample matched pairs test was used to test for differences in distributions. Spearman's rank coefficient was used to test for correlations between the results achieved by participants on the UK NEQAS tumours circulated at assessment and on their own in house tumours.

Results

VALIDATION OF ASSESSMENT THRESHOLDS AND OPTIMAL SENSITIVITY

Comparison of the scores of expert centres and those of the UK NEQAS organising centre on UK NEQAS tumours

For all the assessments (but one) conducted between April 1994—June 1998, all of the six expert centres demonstrated > 10% nuclei in tumours deemed by the organising centre to be hormonal receptor positive. The exception was assessment run 42, where two of the expert laboratories stained < 10% of invasive nuclei in a low expressing infiltrating ductal carcinoma, considered by the UK NEQAS organising centre to be ER positive.

Correlation with biochemical values of UK NEQAS tumours circulated at assessment

Of the tumours used for assessments by the scheme between April 1994 and June 1998, 17 of 21 had been initially tested before assessment using both the LBA and IHC. Of these, all were similarly receptor positive or negative with either assay, using an arbitrary threshold value of 10% or greater of invasive tumour nuclei stained by IHC or 10 fmol/mg protein or greater with the LBA, as designating receptor positive status.

Comparison of the scores of expert centres and those of the UK NEQAS organising centre on the same in house tumours

The Wilcoxon two sample matched pairs signed ranks test was used to test for differences in distributions between the scores awarded to the expert centres for the quality of immunostaining of their own in house tumour and the marks awarded to the UK NEQAS organising laboratory for staining of the same tumours. This test revealed no significant difference, either in the routine scores (Z = −0.170; p = 0.865) or the quick scores generated for run 41 (Z = −0.647; p = 0.518).

COMPARISON OF THE UK NEQAS SCORES AWARDED AT ASSESSMENT FOR THE QUALITY OF IMMUNOSTAINING ON THE SLIDES CIRCULATED AND THOSE ACHIEVED BY THE SAME PARTICIPANTS ON IN HOUSE SLIDES FROM APRIL 1994 TO JUNE 1998

For nine out of the 10 assessments analysed, the median for the scores that participants achieved on in house tumours was higher than the median for the scores achieved on UK NEQAS tumours (fig 1). The Wilcoxon signed ranks test showed a highly significant difference in the distribution of marks for these nine runs (p < 0.0001; two tailed; table 5). The interquartile range was also frequently smaller for the scores achieved on in house sections, indicating less spread in the results. The proportion of participants achieving acceptable staining was always greater on the in house tissues than on the tumours circulated at assessment. These differences ranged from just 8% for run 26 to 44% for run 42 (table 5). However, Spearman's test showed a significant positive correlation between the routine scores awarded for the staining of UK NEQAS tumours and the staining of in house tumours. This relation is seen for all the assessments for ER/PR conducted between April 1994 and June 1998 (table 5; figs 2–9).

COMPARISON OF THE QUICK SCORES (RUN 41) ON PARTICIPANTS' STAINING OF THEIR OWN IN HOUSE TUMOURS AND THE QUICK SCORES OF THE SAME TUMOURS WHEN STAINED BY THE UK NEQAS-ICC ORGANISING CENTRE

The number of participants who submitted two unstained slides containing sections of breast tumour, along with their own laboratories immunostaining of that tumour, for run 41 was 152 (85% of the total returns). The remaining 26 participants (15%) did not provide unstained slides, or only ones of normal breast tissue. Table 2 details the types of tumours submitted, as described in the returned questionnaires. The initial Wilcoxon test indicated a highly significant difference between the quick scores for the in house tumours when stained by the participant and when stained by the organising laboratory (Z = −6.814; p < 0.0001; two tailed). Table 6 shows the degree of expression of the 152 tumours as evaluated using the quick score method on both the slides stained by the participants and duplicate slides stained by the organising laboratory. The proportion of cases designated as high expressers was 51.3% (n = 78) by the participants' staining and 80.9% (n = 123) by the UK NEQAS organising laboratory, with concordance on 72 cases (55.8%; κ coefficient, −0.091; p = 0.043). The proportion of cases designated as medium expressers was 27.6% (n = 42) by the participants' staining and 12.5% (n = 19) by the organising laboratory, with concordance on seven cases (13.0%; κ coefficient, −0.495; p < 0.0001). The proportion of cases designated as low expressers was 15.8% (n = 24) by the participants' staining and 4.6% (n = 7) by the organising laboratory, with concordance on just three cases (10.7%; κ coefficient, −0.316; p < 0.0001). Lastly, the proportion of cases designated as negative was 5.3% (n = 8) by the participants' staining and 2.0% (n = 3) by the organising laboratory, with concordance on three cases (37.5%; κ coefficient, not applicable). Overall, there was agreement on the degree of expression, as defined by the participants' staining and that of the organising laboratory, in 96 of the 152 cases (63.2%; κ coefficient, −0.026; p = 0.291).

Table 7 details the analysis of the tumours showing less than high ER expression by the participants' IHC. Of the 42 cases classified as medium ER expressers and obtaining quick scores of 4 and 5, 69% (p = 0.014) were shown to have higher expression when stained by the UK NEQAS organising laboratory and achieved quick scores that were higher by 2 marks or more. Of the 24 cases initially classified as low expressers and obtaining quick scores of 3 or 2, 83% (p = 0.001) were shown to have higher expression when stained by the UK NEQAS organising laboratory, with quick scores that were higher by 2 marks or more (table 7). Lastly, five of the eight cases classified by the participants' IHC as being ER negative and having quick scores of zero were shown to be ER positive when stained by the UK NEQAS reference laboratory, with one having a quick score of 2, two quick scores of 3, and two quick scores of 6.

EVALUATION OF THE EFFICIENCY OF THE UK NEQAS ORGANISING LABORATORY'S ROUTINE METHOD IN STAINING TUMOURS SUBMITTED FROM PARTICIPATING LABORATORIES

Using the standard UK NEQAS scoring system, for six of the seven assessment runs at which the UK NEQAS organising laboratory stained participants' in house slides, the median of the scores awarded to the organising laboratory was either equal or greater to the median of the scores awarded to the participants on these same in house slides (fig 10). The interquartile range for the scores achieved by the organising laboratory was smaller than the interquartile range of the participants scores on all seven occasions, indicating less spread in the results. The Wilcoxon signed rank test showed that the distribution of scores awarded to the UK NEQAS organising laboratory was significantly higher overall (Z = −6.190; p < 0.0001; two tailed), and individually in four of the seven assessments. For the remaining three runs, there was no significant difference between the two sets of scores (table 8).

Using the quick score method of evaluation, the technique used by the UK NEQAS organising laboratory was 99% efficient (p < 0.0001) in demonstrating the 152 different tumours submitted by participants for run 41, at either the first or second attempt. The overall efficiency achieved by participants using various different methods was 65% (p < 0.0001; table 9).

MEASURES OF ASSESSOR CONCORDANCE WHEN EVALUATING IHC SENSITIVITY BY THE QUICK SCORE METHOD FOR RUN 41

Goodman and Kruskal's γ statistic showed highly significant observer concordance between the assessors (AR, BJ) when using the quick score method to evaluate slides, with values of 0.949 (p < 0.0001) and 0.960 (p < 0.0001) for the staining of in house tumours by the participants and by the UK NEQAS organising laboratory, respectively.

Discussion

For the accurate assessment of the results achieved by different laboratories participating in EQA it is essential to validate the standards against which optimal sensitivity is defined. In our study, we have sought to validate these standards in various ways. Comparison of the results deemed to be optimal by the organising centre with those achieved by participants of the scheme who are known to have clinically validated their results has revealed many similarities, both in the proportion of the UK NEQAS tumours confirmed to be receptor positive and in the quick scores generated on the in house tumours for run 41. Of the tumours used for assessment by the scheme between April 1994 and June 1998, 81% had been initially tested using both the LBA and IHC. Of these, all were similarly receptor positive or negative with either assay, using a threshold value of 10% or greater of invasive tumour nuclei stained by IHC and 10 fmol/mg protein or greater with the LBA, as designating receptor positive status. Although the use of any threshold value is arbitrary, we have used this cut off point because of its use in several studies that correlate IHC receptor assay results with clinical and biochemical values.^16–20 We have also shown previously that this is the threshold most commonly used by the laboratories participating in UK NEQAS-ICC.³ Also imperative to our study is the reproducibility of the methods of evaluation used to assess the quality of IHC. The reproducibility of the routine UK NEQAS scoring system was ensured at assessment by the checking of assessor concordance after every 20 slides. A highly significant degree of concordance with the quick score evaluations was confirmed by Goodman and Kruskal's γ statistic.

Between April 1994 and June 1998, UK NEQAS-ICC conducted 10 assessment runs for ER or PR. During this period, the pass rate on in house tumours remained high (81–97%), whereas that on the distributed UK NEQAS slides fell, particularly for the later runs (runs 40–42). The reasons for these differences are twofold. First, many in house slides submitted for assessment contain just one tumour with high ER/PR expression, and thus are easier to stain than the UK NEQAS tumours. For example, for the one assessment (run 41) subjected to quick score evaluation, at least 47% of the in house tumours submitted were judged to be high expressing tumours by both the participant and the UK NEQAS organising laboratory (table 6). In contrast, the scheme has circulated slides from composite blocks comprising ER positive/PR positive tumours with progressively lower amounts of expression, particularly for runs 40, 41, and 42 (table 1).

The second reason for the difference does not appear to be possible biological differences in ER expression of the tumours used, but more probably differences in the way the tissues have been prepared. In cellular pathology, a multitude of variables affect a specimen the moment it is removed from the patient—for example, delay in fixation, type of fixation, duration of fixation, fixation temperature, paraffin wax processing schedule, and so on.^21–23 Individually, or in combination, these variables might have an effect on the efficiency and reliability of the immunohistochemical assay to demonstrate various antigens.²³ To minimise their effect, the technologist will have adjusted the methodology (over a period of time) to achieve consistently optimum results, according to his or her laboratory's expectations of the desired standard. When presented with tissue subjected to a different set of fixation and processing variables, as is the case with EQA and referred material, the efficiency of the in house method might fail to achieve the optimal result to varying degrees. This probably is the most likely reason for the differences seen between the scores that participants achieved for in house tumours and for the UK NEQAS slides, over the range of antigen expression included in the tumours examined.

Although the results achieved by participants on the UK NEQAS tumours were significantly different to those obtained on the in house tumours, Spearman's coefficient revealed that there was a significant positive correlation between the two sets of scores (table 5). To understand why there is a significant difference, and yet still a significant correlation, it is necessary to consider the relation between IHC assay sensitivity and the degree of receptor expression by the tumours under investigation. Suboptimal IHC assay sensitivity when used to stain ER or PR in a low expressing receptor positive tumour (for example, a UK NEQAS tumour) usually results in < 10% of invasive nuclei being stained and a failure at assessment. When applied to a high expressing in house tumour, this same degree of IHC sensitivity is also suboptimal because some invasive receptor positive nuclei that should be demonstrated are not. However, this is unlikely to result in < 10% of the tumour nuclei being stained, purely on the basis of the large number of receptor epitopes available. Consequently, participants who fail on the low expressing receptor positive UK NEQAS tumour tend to achieve lower scores than they should on their high expressing in house tumour. However, they do not usually fail (a score < 10 out of 20) if the proportion of nuclei demonstrated is equal to, or greater than, the designated 10% threshold. Conversely participants with high IHC assay sensitivity, who achieve a relatively high score on the low expressing receptor positive UK NEQAS tumour, tend to score very high marks on their in house tumour. This relation between assay sensitivity and the proportion of nuclei demonstrated in low expressing ER positive tumours and high expressing ER positive tumours is illustrated in figs 2–9.

The main implication of this correlation is that the IHC sensitivity achieved by laboratories on tumours circulated by UK NEQAS-ICC at assessment is a reflection of the sensitivity that the same laboratories achieve on tumours fixed and processed in their own laboratory (in house tumours). This is the first time evidence has been obtained in support of the view that the IHC results achieved on EQA material are accurate indicators of in house laboratory performance.

It has been shown previously that there is a significant positive correlation between the sensitivity achieved by the same laboratories on tumours of differing expression when these tumours are stained as a composite block.³ In our study, we show that there is a similar correlation between suboptimal demonstration of in house tumours and suboptimal demonstration (< 10% nuclei staining) of relatively low expressing ER positive tumours circulated by an EQA scheme.

At present, there is a tendency to overlook the implications of suboptimal staining of tumours with relatively high amounts of ER or PR expression. Reiner and colleagues¹² and MacGrogan and colleagues²⁴ showed that patients whose carcinomas contained high numbers of hormone receptor positive cells (> 30%, > 50%, > 70% ) had a better overall survival than those patients whose tumours had fewer receptor expressing cells. This was in general agreement with the results of Barnes and colleagues^13,14 and Walker et al, who showed a high rate of recurrence occurring in patients whose tumours contained high proportions of ER negative cells.²⁵ Hawkins has suggested that it is possible for ER IHC results to be divided into a minimum of four categories (negative, low, medium, and high) and still provide prognostic/predictive information similar to that provided, as a continuum, by a sensitive and quantitative biochemical assay.²⁶ Our present study found that 69% (p = 0.014) of tumours initially classified as medium expressers by participants' staining were subsequently shown to be high expressers when stained by the organising centre, and that 83% (p = 0.001) of those classified as low expressers were shown to be medium or high expressers (table 7). In addition, five of eight completely ER negative tumours were found to be ER positive. Two of these tumours were subsequently classified as high expressers, with quick scores of 6, and three low expressers, with quick scores of 3 or 2, although all with 10% or greater of the tumour nuclei staining. Obviously, the clinical importance of producing false negative staining is greater than that of suboptimally staining relatively high expressing ER positive tumours.¹ However, all these tumours were from patients who, according to the criteria of Reiner et al,¹² Barnes et al,^13,14 Walker et al,²⁵ and Hawkins,²⁶ would have a better overall survival than that predicted by the initial in house IHC assays.

This raises the important question as to whether anything short of optimal immunostaining is acceptable for ER testing, the results of which are likely to influence overall clinical management. A number of other immunocytochemical markers also appear to fall into this category, c-erb-2 being one of the best examples.^27–30

Evaluation of the efficiency of the UK NEQAS reference laboratory's routine technique in analysing the in house tumours submitted between February 1995 and June 1998 using the standard UK NEQAS scoring system gives an efficiency ranging from 90% to 100% (table 8). The in depth analysis of run 41, using the quick score method to evaluate staining of 152 in house tumours, shows the organising laboratory's routine method to be 99% efficient in achieving an equivalent or greater sensitivity to that of the participating laboratory from where the tumours were submitted (table 9). This degree of sensitivity was achieved on the first, or second, attempt.

These results clearly indicate that the variations in fixation and paraffin wax processing that have been used by participating laboratories on the tumours submitted for assessment to date are not limiting factors preventing a different laboratory achieving a similar or greater degree of sensitivity for hormonal receptors. This is supported by a detailed analysis conducted by Williams et al, which showed that variations in immunostaining as a result of variations in fixation and processing regimens could be overcome by heat mediated antigen retrieval.²³

The organising laboratory's standard technique uses routine commercial antibodies and reagents—that is, the same antibodies or reagents used by numerous participants at assessment and by some of the laboratories that have clinically validated their results (table 4). Although some participants use different clones to the ones used by the UK NEQAS organising laboratory, the technical comparisons performed after recent assessments do not show one clone to ER or PR to be significantly superior to another.³¹ The same applies for the differing secondary detection systems. This leaves the efficiency of the heat mediated antigen retrieval step as the most likely factor preventing some participants from achieving optimal demonstration of hormonal receptors. Because all the clones to ER currently used at assessment, and most of those used to PR, necessitate heat mediated antigen retrieval for use on routinely processed tissues,^32–36 the degree of sensitivity ultimately achieved with these clones is directly dependant on how well the heat mediated antigen retrieval step has been performed. A multicentre study, involving 15 French laboratories, found the duration of the antigen retrieval step to be the crucial factor preventing some of the participating laboratories producing adequate results for ER on tissues fixed in a different laboratory.³⁷ Subsequent extension of the heat mediated antigen retrieval time allowed these laboratories to achieve optimal results. This supports the findings of our study, which suggest inefficiencies in the heat mediated antigen retrieval step might be the most important factor responsible for poor IHC demonstration of hormonal receptors. It is beyond the remit of our present investigation to provide an in depth technical analysis of the different variables that affect the efficiency of heat mediated antigen retrieval. To date, the few publications on this subject have mainly restricted their investigations to the efficiencies of the buffers used.^38–41 Other papers have compared the efficiency of the various heating methods available—for example, microwave ovens versus pressure cookers.^42,43 However, a comprehensive study is clearly required to compare the relative merits of all the different systems, particularly with respect to their efficiencies in the demonstration of low ER/PR positive breast tumours that have been fixed and processed under differing conditions. This would provide valuable information in the formulation of recommended technical guidelines for optimal IHC demonstration of ER and PR. In turn, this should help in the standardisation of the technique, increase the sensitivity of detection for some laboratories, and ultimately help ensure that hormonal receptor positive cases are not erroneously reported as hormonal receptor negative.