Article Text

Download PDFPDF

A human factor event-based learning assessment tool for assessment of errors and diagnostic accuracy in histopathology and cytopathology
  1. David Nigel Poller1,
  2. Massimo Bongiovanni2,
  3. Beatrix Cochand-Priollet3,
  4. Sarah J Johnson4,
  5. Miguel Perez-Machado5
  1. 1Department of Pathology, Queen Alexandra Hospital, Portsmouth, UK
  2. 2Department of Pathology, Synlab Suisse SA Lausanne, Lausanne, Switzerland
  3. 3Department of Pathology, Cochin Hospital, University of Paris, Paris, Île-de-France, France
  4. 4Department of Cellular Pathology, Royal Victoria Hospital, Newcastle upon Tyne, UK
  5. 5Department of Pathology & Cytology, Royal Free Hospital, London, UK
  1. Correspondence to Dr David Nigel Poller, Queen Alexandra Hospital, Portsmouth, UK; mail{at}poller.com

Abstract

This review article summarises systems for categorisation of diagnostic errors in pathology and cytology with regard to diagnostic accuracy and the published information on human factors (HFs) in pathology to date. A 12-point event-based checklist for errors of diagnostic accuracy in histopathology and cytopathology is proposed derived from Dupont’s ‘Dirty Dozen’ HF checklist, as used in the aerospace industry for aircraft maintenance. This HF checklist comprises 12 HFs; (1) Failure of communication. (2) Complacency. (3) Lack of knowledge. (4) Distractions. (5) Lack of teamwork. (6) Fatigue. (7) Lack of resources. (8) Pressure. (9) Lack of assertiveness. (10) Stress. (11) Norms. (12) Lack of awareness. The accompanying article explains practical examples of how each of these 12 HFs may cause errors in diagnostic accuracy in pathology. This checklist could be used as a template for analysis of accuracy and risk of diagnostic error in pathology either retrospectively ‘after the event’ or prospectively at the time of diagnosis. There is a need for further evaluation and validation of this proposed 12-point HF checklist and similar systems for categorisation of diagnostic errors and diagnostic accuracy in pathology based on HF principles.

  • histopathology
  • diagnostics
  • cytopathology
  • laboratory tests
View Full Text

Statistics from Altmetric.com

The role of human factors (HFs) in clinical medicine is undisputed. A definition of HF that is widely accepted is as follows: ‘environmental, organisational and job factors in human and individual characteristics which influence behaviour at work in a way which can affect health and safety’.1 HF analysis has been widely applied in aviation and aerospace industries but has been increasingly seen as important in medicine, particularly in critical care and acute medical settings to explain and reduce rates of clinical error and improve decision making in areas that are of high clinical risk.2–4 There has been little discussion of the role of HFs in diagnostic specialties such as pathology, cytology, or radiology. Published HF studies in diagnostic specialties have primarily addressed the HF aspects of care involving either handovers from clinical teams to pathology5 or aspects of HF involved in multidisciplinary case discussion which are common to other clinical specialties and not just pathology and radiology6 or fatigue7. One of the authors (DNP) initially sought to devise a more in-depth tool using HF principles for assessment of accuracy of individual pathologists’ diagnoses, applicable to the study of diagnostic errors. The definition of a diagnostic error in the literature varies. In 2015 the US Institute of Medicine redefined diagnostic error in a wide-ranging way as ‘the failure to (a) establish an accurate and timely explanation of the patient's health problem(s) or (b) communicate that explanation to the patient’, encompassing three aspects—accuracy, timeliness and communication.8 This HF tool assesses accuracy of diagnosis alone. It does not address issues of timeliness of explanation of the patient’s health problem or of communication to the patient. It is also important to define what constitutes accuracy. Variation in diagnostic interpretation and reporting practice is not uncommon, and in many cases differences in diagnosis may have little or no clinical impact or consequences. Diagnostic errors may be classified or graded according to various different systems; three differing systems are described below.9–11 The UK Royal College of Pathologists (UK RCPath) categorises diagnostic discrepancies in five categories9.

Category A. Inadequate dissection, sampling or macroscopic description. The RCPath states that where relevant this should be assessed against guidance such as RCPath data sets and tissue pathways. The UK RCPath also comments that it should be remembered that the pathologist issuing the final report may not have dissected, described and sampled the specimen. This category also includes failure to request further work (eg, histological levels, immunostains) where these are clearly required to make a diagnosis.

Category B. Discrepancy in microscopy

  1. A diagnosis that one is surprised to see from any pathologist (eg, an obvious cancer reported as benign).

  2. A diagnosis that is fairly clearly incorrect, but which one is not surprised to see a small percentage of pathologists suggesting (eg, a moderately difficult diagnosis, or missing a small clump of malignant cells in an otherwise benign biopsy).

  3. A diagnosis where interobserver variation is known to be large (eg, disagreements between two adjacent tumour grades, or any very difficult diagnosis).

Category C. Discrepancy in clinical correlation. This would represent a failure to answer the clinical question (if clearly expressed on the request form) despite the answer being evident from the material available; or a failure to indicate that a specimen is clearly inadequate to answer the clinical question

Category D. Failure to seek a second opinion in an obviously difficult case. This could imply overconfidence or may be indicative of dysfunctional relationships with a department. It is important that any second opinion is clearly evidenced within the report.

Category E. Discrepancy in Report. This would imply included typographical errors and internal inconsistencies or ambiguities in the report which should have been corrected before authorisation. It would also include cases where there is suspicion that reports may have been allocated to the wrong patient, case mix-ups and so on.

The Royal College of Pathologists of Australasia Quality Assurance Program classifies diagnostic discrepancies as follows: 10

Concordance. The preferred diagnosis is essentially/substantially identical with the target diagnosis, that is, accurate diagnosis

Minor discordance. The preferred diagnosis has one or more minor differences from the target diagnosis, that is, minor diagnostic inaccuracy.

Major discordance. The preferred diagnosis is substantially different from the target diagnosis, that is, inaccurate diagnosis

The French Quality Assurance Association in Pathology, Association Francaise d’Assurance Qualite en Anatomie et Cytologie Pathologique, (AFAQAP), in 2016 established an online national register of second opinions, for example, for when a pathologist sends a case to a colleague for a second opinion which defines levels of discrepancy as follows:11

  • Impact of the diagnostic discrepancy on patient management (from the pathologist’s perspective): none, minimal, major or not known.

  • Impact of the diagnostic discrepancy on patient management (after clinical case review with name of clinician, specialty and date of discussion): none, minimal, major or not known.

  • The AFAQAP also categorises the discrepancy based on the stain(s) or laboratory technique(s) and asks the respondent to specify which stain(s), antibody(ies) or probe(s) were associated with the discrepancy.

____YES____NO Standard stain

____YES____NO Special stain(s)

____YES____NO Immunocytochemistry (cytology)

____YES___ NO Immunohistochemistry (histology)

___ YES___ NO In situ hybridisation

____YES____NO Molecular technique

A joint working group of the College of American Pathologists Pathology and Laboratory Quality Centre and the Association of Directors of Anatomic and Surgical Pathology, convened an expert panel to develop an evidence-based guideline to help define the role of case review in surgical pathology and cytology.12 While this does not provide specific definitions for interpretative diagnostic error, it is a useful consensus-based guideline. The panel assessed the published evidence for the effect of targeted case review undertaken at either the analytical or the postanalytical phase of surgical pathology or cytology cases (slides and/or reports) to either reduce or increase the rate of interpretive error detection (often measured as amended reports) compared with no review, random review or usual review procedures. The five recommendations were as follows:12 (1) Anatomic pathologists should develop procedures for review of selected pathology cases to detect disagreements and potential interpretive errors. (2) Anatomic pathologists should perform case reviews in a timely manner to avoid having a negative impact on patient care. (3) Anatomic pathologists should have documented case review procedures that are relevant to their practice setting. (4) Anatomic pathologists should continuously monitor and document the results of case reviews. (5) If pathology case reviews show poor agreement with a defined case type, anatomic pathologists should take steps to improve agreement.

Hence as the examples above show it can be seen that there are differences in how diagnostic accuracy is categorised and in recommendations for case reviews to address diagnostic errors. It is not the purpose of this tool to try and address these, rather to identify HF-related issues that may impact diagnostic accuracy. This event-based learning tool uses a 12-point framework widely used in aerospace, specifically in the process of aircraft maintenance and referred to colloquially as Dupont’s ‘Dirty Dozen’.13 The ‘Dirty Dozen’ is broadly accepted as a useful tool for error management in aircraft maintenance worldwide, and is endorsed or quoted in publications from the US Federal Aviation Authority14 and the European Aviation Safety Agency among others. It has also been applied in ophthalmology.15 A 12-point questionnaire applicable to cytology and pathology was formulated by DNP with advice from the remaining authors using the same generic 12 factors as applied to HF errors in the ‘Dirty Dozen’ process of aircraft maintenance.14 DNP with the assistance of a writing group then devised a computer-based questionnaire template using the 12-point multifactorial HF framework. The factors in any individual case may be single or may be multiple and may in some cases overlap. This list could be used either retrospectively as a root cause analysis tool ‘after the event’ and also prospectively at the time of reporting a case as self-assessment quality assurance tool.

The 12 factors applied to histopathology and cytopathology are as follows:

  1. Errors due to failure of communication—not receiving or seeking sufficient information (including clinical information, or laboratory information to make the diagnosis correctly). In pathology or cytology this would normally be because of lack of adequate clinical information provided to properly analyse a specimen.

  2. Errors due to complacency. Diagnostic errors typically made often by senior and experienced practitioners of cytology or histopathology who fail to appreciate that a case is more complex or challenging than initially appears to be the case. These cases are often reported in reflex mode without sufficient consideration because of a failure to appreciate the complexity and nature of the diagnostic problem and the relevant differential diagnoses. This may also be a manifestation of overconfidence, in what might seem superficially to be a straightforward diagnosis.

  3. Errors due to lack of knowledge. Diagnostic errors made because of lack of training, knowledge, or where personal practice is deficient or was not up to date. In diagnostic histopathology or cytology this is usually due to lack of knowledge of the relevant differential diagnoses or the relevant diagnostic pitfalls and requirements of a particular case. While these diagnostic errors tend to be made by less experienced pathologists, experienced pathologists can also fall into this trap. Another example might be a specialised pathologist/cytopathologist covering colleagues’ absence or leave in a specialty with which the specialised pathologist/cytopathologist was not recently experienced.

  4. Distractions. This could be for many reasons, typically interrupting emails, texts phone calls, or requirements to multitask which may cause lack of sensory attention and hence increase the risk of diagnostic error.

  5. Lack of teamwork. This rather generic overarching point that relates to any institutional or team failure. An example might be a diagnostic error made because laboratory colleagues, or a multidisciplinary team or a departmental organisation did not support or did not assist the pathologist appropriately. Pathology samples received in the laboratory in inappropriate fixative, delayed in transit or compromised in quality for whatever reason; if this situation could have been avoided by better institutional or teamworking these would fall into this category.

  6. Fatigue. This is fairly self-explanatory but pathologists are frequently required to work long hours. There is evidence of the value of regular rest periods in enhanced diagnostic performance. Diagnostic errors made because a pathologist or cytologist was tired, either physically or mentally exhausted, would fall into this category

  7. Lack of resources. Diagnostic errors made because the laboratory facilities or laboratory staffing or equipment or reagents were not adequate or appropriate for the diagnostic tasks required. This would also include consequences of lack of resources, for example, a lack of quality in routine laboratory stains, immunohistochemistry or other ancillary techniques including molecular techniques.

  8. Pressure. Errors made because of real or perceived forces including either feeling time pressured in general or in relation to a specific case to complete a diagnosis, including being pressured to make a diagnosis by a clinician or by a patient.

  9. Lack of assertiveness. Errors made because of failure to speak up or document concerns about instructions, information or other aspects of a case dealt with by somebody else.

  10. Stress. Errors attributable to personal stress, or anxiety, so that the pathologists’ skills in diagnostic decision making were not normal at the time of the diagnostic error

  11. Norms. Errors caused by acting according to the expected yet unwritten rules of behaviour, for example, by acting in a reflex fashion and intuitively rather than by fully appreciating the diagnostic issues or challenges surrounding the case.

  12. Lack of awareness. An error that occurred because of failure to recognise the importance or significance of a diagnosis or the differential diagnosis (even though the pathologist knew about the diagnosis and/or relevant differential diagnosis). This would overlap with point 3 above.

The text version of this survey is shown in box 1. An online version of this survey can be viewed on the website www.pathlab.org. While an HF approach to diagnostic error categorisation in cytology or histopathology has not as yet been validated we think it might be a very useful way of analysing the complex reasons for diagnostic errors in cytology or histopathology, and that this approach could be used either prospectively at the time of reporting or retrospectively ‘after the event’ for confidential surveys or root cause analyses when diagnostic errors arise. We are submitting this suggestion to the journal for publication in the hope that it can be more widely disseminated and as a basis for further discussion and development.

Box 1

Human Factors Checklist for Diagnostic Accuracy in Histopathology and Cytopathology based on Dupont’s ‘dirty dozen’

HOW WAS THE ERROR DISCOVERED?

____Review by departmental pathology colleague(s) (general)

____Review of FNA cytology after histology assessment (eg, after surgical resection)

____Review after I remembered making the error

____Review by pathology trainee(s)

____Review of cytology or histology at a multidisciplinary meeting

____Review by outside pathology consultation or expert

____Review after query or a complaint by clinician

____Review after a patient complaint or legal action

____Other

____I have not made any diagnostic error(s)

____I cannot remember the details of any error(s) I have made

____Time interval between error and date of completion of checklist

DID THE ERROR DELAY OR AFFECT A PATIENT'S CARE OR TREATMENT?

____YES

____NO

____NOT SURE

DID THE ERROR CAUSE HARM TO A PATIENT?

____YES

____NO

____NOT SURE

DO YOU THINK YOU WILL MAKE THE SAME OR A SIMILAR ERROR (S) IN THE FUTURE ?

____YES

____NO

____NOT SURE

In relation to the THIS DIAGNOSTIC ERROR THAT I PERSONALLY MADE IN MY OWN PRACTICE

I BELIEVE THAT THE REASON(S) FOR ME PERSONALLY MAKING THE ERROR WERE AS FOLLOWS

1. FAILURE OF COMMUNICATION—I did not receive or seek sufficient or correct information (including clinical information, or laboratory information to make the diagnosis correctly)

____YES

____NO

____NOT SURE

2. COMPLACENCY—I was overconfident because I had made what seemed to me at the time to be a straightforward diagnosis which I had made many times before

____YES

____NO

____NOT SURE

3. LACK OF KNOWLEDGE—I made the error because my training, knowledge, or personal practice was deficient or was not up to date

____YES

____NO

____NOT SURE

4. DISTRACTIONS—I made the error because I was disturbed, distracted, or interrupted or not fully concentrating on the diagnostic task

____YES

____NO

____NOT SURE

5. LACK OF TEAMWORK—I made the error because my laboratory colleagues, my multidisciplinary team, or my departmental organisation let me down or did not assist me appropriately

____YES

____NO

____NOT SURE

6. FATIGUE—I made the error because I was tired, either physically, or was mentally exhausted

____YES

____NO

____NOT SURE

7. LACK OF RESOURCES—I made the error because my laboratory facilities or laboratory staffing or my equipment or my reagents were not adequate or appropriate

____YES

____NO

____NOT SURE

8. PRESSURE—I made the error because of real or perceived forces including either feeling time pressured in general or specifically in relation to a case to complete a diagnosis, including being pressured to make a diagnosis by a clinician or by a patient

____YES

____NO

____NOT SURE

9. LACK OF ASSERTIVENESS—I made the error because of my failure to speak up or document my concerns about instructions, information or other aspects of a case dealt with by somebody else

____YES

____NO

____NOT SURE

10. STRESS—I made the error because of personal stress, or because I was overanxious so that my skills in diagnostic decision making were not normal at the time I made the error

____YES

____NO

____NOT SURE

11. NORMS—I made the error because I did what I would normally do according to the expected yet unwritten rules of behaviour, by acting in a reflex fashion and intuitively rather than by fully appreciating the diagnostic issues surrounding the case

____YES

____NO

____NOT SURE

12. LACK OF AWARENESS—I made the error because I failed to recognise the importance or significance of the diagnosis or the differential diagnosis (even though I knew about the diagnosis and/or relevant differential diagnosis)

____YES

____NO

____NOT SURE

References

View Abstract

Footnotes

  • Handling editor Dhirendra Govender.

  • Contributors DNP: Conception and principal major contribution to the design and conduct of study MB, BC-P, SJJ, MP-M: Assisted with the design and conduct of study. DNP is the guarantor.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient consent for publication Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.