Article Text
PDF
Abstract
Aims Diagnostic/interpretative accuracy can be challenging in anatomical pathology due to the subjective element of the diagnostic process. This can lead to false-negative or false-positive diagnoses of malignancy, variations in grading and diagnostic misclassification of a condition.
It is imperative that an accurate diagnosis is achieved so that an appropriate and timely treatment is administered to the patient, for example, the success of targeted molecular therapeutic options for treatment of cancer is dependent on accurate anatomical pathology diagnoses being issued.
Methods A literature review of diagnostic accuracy in selected specimen categories was undertaken and was compared with data on metropolitan and regional pathologist diagnostic proficiency performance in an external quality assurance programme from surveys provided 2015–2017. For each specimen category, cases having attracted a diagnostic inaccuracy (ie, major discordance) of ≥20% and cases attracting a combined error rate (ie, major and minor discordance) of ≥30% are reviewed and discussed.
Results The rate of inaccurate diagnoses (assessed as a major discordance) ranged from 3% to 9% among the different specimen groups, with highest mean percentage of inaccurate diagnoses in gynaecology, dermatopathology and gastrointestinal specimens.
Conclusions It was possible to ascertain that gynaecology, dermatopathology and gastrointestinal specimens had presented the greatest diagnostic challenge to the participant pathologists, determined as highest rate of diagnostic inaccuracy, that is, major discordance with respective case target diagnoses.
Through a combination of routine second opinions, directed retrospective peer review and participation in appropriate external quality assurance schemes, the risk associated with these diagnoses can be minimised.
- diagnostic error
- diagnostic imprecision
- interpretative error
- diagnostic accuracy
- quality assurance
- second opinion
- anatomical pathology
- case review.
Statistics from Altmetric.com
- diagnostic error
- diagnostic imprecision
- interpretative error
- diagnostic accuracy
- quality assurance
- second opinion
- anatomical pathology
- case review.
Introduction
A lack of standardised quality metrics has hindered the ability of anatomical pathology laboratories and individual pathologists to objectively measure diagnostic accuracy, irrespective of intuitively gauging what distinguishes acceptable performance from poor performance.1–4 Laboratory policies and protocols for quality assurance and quality control processes may have idiosyncratic limitations for evaluating diagnostic quality metrics peculiar to institutional internal processes, specimen mix and staff experience.5–7
Variation in diagnostic interpretation and reporting practice in anatomical pathology is not uncommon. In most instances, such differences in diagnosis are minor and have little or no clinical consequence. A major error, however, may necessitate a clinically significant diagnostic amendment, for example, changing a diagnosis from benign to malignant or failure to identify and report treatable infection. With a focus on either general or specialist pathologist case mix, various studies have identified instances of major and minor diagnostic discordances between internal and referral diagnoses. The magnitude of diagnostic disagreement varied considerably between these reviews with a range of 16%,8 5.8%9 and 1.3%.10 This may reflect the variations in type of review undertaken, error detection method and definition of error in these reviews.
Second opinions reduce the incidence and necessities for amendments of original pathology reports11–13 and are an effective quality metric in anatomical pathology. They constitute an integral component of a broader quality management policy. Second opinions increase the likelihood of securing an accurate diagnosis from which an appropriate course of treatment for a patient can be undertaken. For this reason, prospective case review is now commonly undertaken and is especially frequent in cases that require a specialist diagnostic opinion.14 In this context, diagnostic accuracy is based on the specialist opinion being the ‘gold standard’; however, this can be confounded and influenced by potential diagnostic bias of the specialist. This may depend on expertise, relevant experience and appropriate application of contemporary recommendations, for example, diagnostic criteria, arising from their continuing professional development.
While second opinions in challenging cases may be referred internally or externally, the latter has been particularly beneficial in detecting, and therefore preventing, major diagnostic errors. External referrals have been shown to have a five times greater sensitivity for detecting major discrepancies compared with internal reviews by peers.15 External second opinions (ie, referrals) afford laboratories the distinct advantages of securing additional and specialist peer input, while removing any potential internal diagnostic bias or conflict of opinion between colleagues.
Internal case reviews may be undertaken either prospectively or retrospectively to a case’s final validation. Targeted (ie, directed) peer review involves the selection and methodical review of targeted case types or specific areas of anatomical pathology. Random review, on the other hand, comprises review of a specific percentage of all cases previously received and reported in the laboratory. This format affords peers the opportunity to review a broad spectrum of specimen types and pathologies.16
A literature review of diagnostic accuracy in selected specimen categories was undertaken and compared with data collated on the performance of pathologists in an external quality assurance diagnostic module. The module was designed for testing diagnostic proficiency. The performance results from a few selected case exercises were compared with results from published studies on similar specimens.
The risks associated with erroneous reporting of anatomical pathology are discussed, and the key role and merit of second opinion (referral) diagnoses and directed peer review in the prevention of misdiagnosis are discussed.
Method
The Royal College of Pathologists of Australasia Quality Assurance Programme (RCPAQAP) is a provider of multiple modules of external diagnostic proficiency testing. Participants may also enrol solely for educational purposes. The purpose of enrolment (ie, either for proficiency testing or educational purposes) is specified by each participant.
An advisory committee, comprising highly experienced anatomical pathologists from public and private laboratories, is provided identical case information and diagnostic virtual images from a pool of archived prospective cases previously donated by participants and patient de-identified for EQA purposes. All diagnoses are omitted. Each committee member independently reviews each case according to stringent EQA selection criteria. The committee subsequently convenes to discuss each prospective case for diagnostic suitability, that is, each case must demonstrate either classical or common histological features of a disease encountered in routine histopathology laboratories or a critical disease process of clinical importance. Slide image quality and relevance is also stringently reviewed. A documented consensus is reached by the advisory committee on the case selection, including target diagnosis and assessment rationale. Each online survey of 10 case exercises comprises assorted specimens and pathologies.
Each case exercise comprises a clinical history, macroscopic description and virtual H&E slide. In a small number of cases, additional images are provided, for example, macroscopic pictures, special stains, immunofluorescence or immunohistochemistry. The aim of the programme is to provide the participant enough information to render a definitive diagnosis on the material provided. Participants are required to submit a free-text preferred diagnosis for each case online. The virtual slides are available online and (optionally) on CDROM. The introduction of virtual slides has enabled the RCPAQAP to include a broader range of pathologies and specimen types including small biopsies. It also ensures that all participants view exactly the same material.
The objective of this review is to compare documented issues affecting accuracy in anatomical pathology diagnosis and reporting using the comprehensive information acquired from the RCPAQAP General Diagnostic module from surveys provided 2015–2017. The archived records of the RCPAQAP General module from 2015 to the present were analysed, amounting to 80 cases (8 surveys). The data evaluated included the participant numbers, survey and case designations, targeted diagnoses, purpose of enrolment (ie, for diagnostic proficiency assessment or education only), type of laboratory where the participant was employed and performance assessments (table 1).
Quality assurance programme assessment categories and criteria
The data from eight surveys were first separated to distinguish participants having enrolled for diagnostic proficiency testing purposes from those having enrolled solely for education. For the purpose of this publication, only data pertaining to diagnostic proficiency testing were analysed. Using tick boxes in the online survey, the participants were also required to specify the type of laboratory they are employed in, that is, metropolitan (a city laboratory) or regional (non-city laboratory).
The 80 cases provided participants a diversity of specimen types and pathologies. The cases are selected by the General Module advisory committee whose membership has representation from public and private laboratories in both metropolitan and regional locations from Australia and New Zealand. The committee aims to reflect the full range of contemporary general diagnostic practice.
Data arising from the 80 RCPAQAP General Module cases, including participant diagnostic accuracy results, were collated and sorted into nine distinct anatomical specimen categories, as listed below. The quantity of cases is denoted in brackets.
Bone and soft tissue (9).
Breast pathology (2).
Gynaecology (8).
Gastrointestinal system (18).
Head, neck, oral and maxillofacial pathology (11).
Lymphoid system (3).
Respiratory system (8).
Skin (11).
Urology system (9).
For each specimen category, the case count mean percentage of major discordant diagnoses and mean percentage of minor discordant diagnoses, the latter including one or more minor differences from a target diagnosis and considered of no serious clinical consequence, were determined. Instances of diagnostic bias were identified in cases having received significantly high levels of stark diagnostic inaccuracy.
Results
One of the 80 cases was excluded from the analysis because of a failure to reach a consensus concordant diagnosis. The rates of diagnostic inaccuracy according to specimen category are shown in figure 1 and the differences between metropolitan and regional laboratories are shown in figure 2.
Participant diagnostic performance in Royal College of Pathologists of Australasia Quality Assurance Programme General Module cases, by specimen category.
Metropolitan and regional participant major discordance in Royal College of Pathologists of Australasia Quality Assurance Programme General Module cases, by specimen category.
Cases having attracted a diagnostic inaccuracy (major discordance) of ≥20% and cases attracting a combined error rate (ie, major and minor discordance) of ≥30% were reviewed and discussed. These cases are shown in table 2.
Royal College of Pathologists of Australasia Quality Assurance Programme cases attracting highest diagnostic major and minor discordance rates(2015–2017)
Discussion
By categorising and grouping the 80 cases used in the RCPAQAP anatomical pathology surveys, it was possible to ascertain which group of the EQA specimen types had presented the greatest diagnostic challenge to the participant pathologists, determined as highest rate of diagnostic inaccuracy, that is, major discordance with respective case target diagnoses.
The specimen groups attracting the highest mean percentage of incorrect diagnoses (7%–9%) also attracted a mean percentage accuracy rate of over 80% as assessed by concordance with the respective case target diagnoses. The high concordance achievement in each of these specimen groups confirms the validity of the target diagnoses established by the RCPAQAP General Module advisory committee during its stringent case selection process, especially as the results for these four specimen groups (gynaecology, gastrointestinal, head and neck and skin) pertain to independent diagnoses from approximately 270 pathologists worldwide. It should be reiterated that all data analysed pertained to participants having participated for diagnostic proficiency testing purposes; therefore, based on the review of cases used between 2015 and early 2017, our findings appear to indicate a risk of approximately 7%–9% misdiagnosis of cases falling within these specified specimen groups (table 2).
The case of necrotising sialometaplasia was misdiagnosed by 63/270 (23%) of participants enrolled for the purpose of proficiency testing. This case was deemed to have proven valid for its intended purpose, on account of having been correctly diagnosed by the majority (189) of participants. The majority (48/63) of discordant responses received stated squamous cell carcinoma (SCC). This concurs with the report by Woolgar et al 17 who, in their experience, list ‘mistaking necrotizing sialometaplasia for SCC’ as an example of potential misdiagnosis. The presence of squamous islands in necrotising sialometaplasia is considered a classic example of a benign lesion having the misleading appearance of SCC.17
The discordant diagnoses submitted for the case of mixed metastatic SCC and metastatic medullary thyroid carcinoma critically failed to identify two morphologically different tumours in the slide or interpreted them as one tumour with two different components. This significant oversight constituted a major lapse in diagnostic accuracy. Medullary thyroid carcinoma with squamous differentiation has been reported;however,it is very rare, and rendering this diagnosis overlooks the more likely and important diagnosis of coexisting metastatic SCC. It was also noted that despite having incorrectly diagnosed the dual pathology case, 41 of these participants correctly interpreted the virtual immunohistochemistry slides provided. The immunohistochemistry should have aided in ascertaining the presence and identity of the coexisting tumours.
Our data showed that the case with the highest rate of major discordance was the diffuse signet ring cell adenocarcinoma of the stomach with 73% of participants failing to diagnose this correctly. Participants failed to recognise the subtle but critical changes present, highlighting an important area of diagnostic error especially when it is already known to have a high rate of litigation.18
Interestingly, the case of normal secretory endometrium attracted a high rate of major discordance (22%). This is likely due to the artificial nature of EQA where pathologists have a bias to diagnosing a pathological condition for each case. Other areas of potential diagnostic error that have been highlighted by our module are the failure to detect infectious organisms (spirochaetosis and viral inclusions) and diseases with subtle histological features (malakoplakia and leucocytoclastic vasculitis). It is through the progressive accumulation of such interpretative error data that appropriate risk management measures can be introduced.
The routine use of second opinions, also known as ‘preventive medicine for pathologists’,3 provides a means of reviewing and recording peer performance in reaching an accurate final case diagnosis.3 The review of data initially received from participants for the selected RCPAQAP cases indicated a high incidence of case referral not ordinarily being performed by those participants having misdiagnosed those cases; however, indication of referral was not mandatory and therefore may not accurately reflect actual practice. Accessibility to colleagues (internally or externally) seeking a second opinion prior to reporting is prudent; however, there has historically been an absence of standardisation between laboratories in how and when this is undertaken.1 19
Typical instances when referral may be considered advisable include unfamiliar specimen types with high-risk diagnoses. Prospective second opinions reduce the incidence and necessity for amendments of original pathology reports11–13; an effective quality metric in anatomical pathology constituting an integral component of a broader quality management policy. Regional or smaller institutions may have an increased requirement for referral externally to a metropolitan or specialist laboratory for specialist review and/or special stains to be performed prior to final diagnosis compared with a metropolitan laboratory where internal second opinion and access to special stains is readily available. Our EQA data analysed indicated pathologists located in regional laboratories generally had a slightly higher rate of diagnostic inaccuracy compared with those located in metropolitan laboratories (figure 2). For example, 18 gastrointestinal specimens attracted an overall rate of inaccuracy of 9% from 187 regional participants, compared with 8% from 224 metropolitan participants. This may in part reflect the greater likelihood of available specialist expertise and experience with these specimen types in a metropolitan laboratory.
Review of the RCPAQAP General Module data showed significant levels of diagnostic error across a diversity of pathologies in each of these specimen groups.
Gastrointestinal and gynaecology specimens each attracted a9%error rate, whereas bone and soft-tissue specimens attracted a4%–5%errorrate. The high level of error observed concurs with a review performed over a5-yearperiodcomprising in excess of71 000referral cases20in which it was deduced that 80/457 cases (17.5%) of major diagnostic disagreement between primary and secondary opinions were those of gastrointestinal cases, and47/457(10.3%)were found to be from bone/soft-tissue cases.
Nakhleh et al 21 developed a comprehensive set of five evidence-based recommendations for the review of anatomical pathology and cytology cases in detecting or preventing diagnostic errors. The recommendations for case reviews were proposed for anatomical pathologists.
That they be developed for selected cases to detect disagreements and diagnostic errors.
They are performed promptly to prevent compromising patient case management.
Reviews must be documented and they must reflect the laboratory routine case mix.
Results from reviews must be routinely monitored and documented.
Remedial action must be taken in instances of significant diagnostic disagreement or interpretive errors for a defined case type, that is, ‘closing the loop’.
Anatomical pathology diagnostic error has been defined by the Institute of Medicine as being the failure to correctly diagnose a disease process in a patient.1 Quality metrics have been established in the discipline; however, there is a lack of reproducible related performance standards.4 Diagnostic/interpretative accuracy is a classic example of a quality metric applied in anatomical pathology. Similar to other areas of medicine,22 variation in anatomical pathology interpretation and reporting practice is not uncommon between peers.22 Diagnostic accuracy will facilitate a patient being administered an appropriate and timely course of treatment.
Summary
The literature review of diagnostic accuracy in anatomical pathology showed that a range of challenges continue to be experienced in routine practice. There appears to be a greater universal comprehension and appreciation of the merits of seeking second opinions. Standardised operational practice in diagnostic anatomical pathology encompasses all aspects of case management in laboratories. The critical responsibility and contribution of pathologists, together with these practices, enables beneficial operational quality metrics and improved diagnostic proficiency for patients.
The merits of second opinions are well documented and appear to have been substantiated in the data analysed from the EQA cases; however, indication of referral by participants had not been mandatory. Second opinions are highly recommended for consideration in routine diagnostic practice, especially if specialist interpretative skill and experience is necessary. External second opinions afford laboratories the distinct advantages of securing additional and specialist peer resources, while removing any potential internal diagnostic bias or conflict of opinion between colleagues. External prospective (ie, pre-reporting) diagnostic review has a role as it has proven an effective measure in preventing major interpretative errors.5 The EQA data analysed were insufficient to correlate second opinions with diagnostic accuracy and therefore substantiate their merit. The primary objective for the data was to identify those EQA cases which the authors believed a second opinion would be most beneficial. The impact of second opinions will be further reviewed by the authors. Nakhleh et al 21 recommended case reviews be developed for selected disagreements and diagnostic errors.
The findings indicate a relatively higher incidence of diagnostic inaccuracy or ambiguity in regional laboratories. Laboratories are encouraged to undertake prospective case reviews in their routine practice prior to reporting. In addition to including case referral for second opinion in standard operational quality policy, laboratories are encouraged to engage in regular departmental case review meetings, configured and operated to address particularly challenging specimen types, for example, brain biopsies and breast biopsies. To their credit, such meetings are already undertaken by some laboratories.14
Annual participation in an external quality assurance scheme, an external peer review activity, is recommended as an activity for measuring diagnostic proficiency. Selection of diagnostic modules should be pertinent to a laboratory’s and pathologists’ scope of practice. This will more effectively address the maintenance of diagnostic competence and confidence. This is especially important in the current era of rapidly proliferating immunohistochemistry panels/stains, molecular assays and biomarkers.
To conclude, our experience with the General Diagnostic Module of RCPAQAP has indicated areas of potentially serious diagnostic inaccuracy in histopathology; however, it is acknowledged that under real case circumstances, a reporting pathologist would ordinarily have access to additional case material and clinical resources. It is also acknowledged that the number of EQA cases analysed was relatively modest in comparison with contemporary laboratory workloads and may therefore not reflect actual overall rate of error in diagnostic pathology practice.
Through a combination of routine second opinions, directed retrospective peer review and participation in appropriate external quality assurance schemes, the risk associated with these diagnoses can be minimised with subsequent benefits to patient safety.
Take home messages
Gastrointestinal and gynaecology specimens each attracted a 9% error rate, whereas bone and soft-tissue specimens attracted a 4%–5% error rate.
17.5% of major diagnostic disagreement between primary and secondary opinions were those of gastrointestinal cases, and 47/457 (10.3%) were found to be from bone/soft-tissue cases.
External second opinion (ie, case referral) is considered an effective quality metric in anatomical pathology and should constitute an integral component of laboratory quality management policy, especially regional laboratories that may not have specialist resources available.
When reporting a challenging case, accessibility to additional case-pertinent information, for example, previous histology/cytology and radiology results, may be critical for making an accurate anatomical pathology diagnosis for the purpose of case correlation.
References
Footnotes
Handling editor Dhirendra Govender.
Correction notice This article has been corrected since it was published Online First. The second author's name has been corrected.
Contributor All four authors were involved with the drafting, interpretation of the data, critical revision and final approval of the article. MP performed the data collection and analysis.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interest None declared.
Patient consent Not required.
Provenance and peer review Not commissioned; externally peer reviewed.