Abstract
Issue. There are signs of inappropriate application of medical imaging to diagnosis. Inappropriate imaging is a threat to effective diagnosis and effective allocation of resources. The development and deployment of knowledge-based clinical decision support systems is one strategy to reduce inappropriate imaging.
Development of imaging pathways. A suite of 78 imaging pathways was conceived both as a decision support and educational tool. The pathways were drafted by imaging specialists, but further developed and modified, based on graded evidence and input from requesting clinicians. An electronic environment was developed to contain and deliver the pathways.
Dissemination and implementation. Imaging pathways were distributed via a hospital local area network and on compact disk. A multifaceted approach was used to raise general awareness of the pathways, followed by intensive ‘marketing’ activities. Two groups of clinicians were targeted; hospital-based clinicians and general practitioners.
Evaluation. There was increased awareness of imaging pathways. Clinicians judged them to be useful for education and decision support. The method of electronic delivery was adequate. Knowledge of diagnostic imaging and requesting behaviour tended to become more aligned with the pathways. The central objective to reduce inappropriate medical imaging seems to be achievable.
Lessons learned. There is scope to improve the content and the electronic environment, achieve better integration into decision-making processes, and achieve better compliance. A linkage between imaging pathways and electronic requesting could provide alerts to ‘non-compliant’ requesting. The assignment of a higher cost, or a lower remuneration, to non-authorized and non-compliant imaging would provide tangible incentive to comply, unless there are compelling clinical contraindications.
Identification of the issue
Diagnostic imaging in Australia accounted for 15.5% of Medicare benefits paid in 2003/2004 and has become a major driver of rising health care costs [1]. Although the principle of ‘accountability’ requires government to ensure that public funds are expended appropriately, there are signs of inappropriate expenditure stemming from inappropriate application of medical imaging technology to diagnosis. It is common for Imaging Specialists to give examples of unnecessary examinations being provided for particular clinical conditions and for desirable examinations not to be provided. A recent article cites previous research indicating that up to a third of radiological examinations are totally or partially inappropriate [2].
Inappropriate imaging has a number of potential adverse outcomes. Unnecessary or excessive examinations may expose patients to risk without benefit [2], including the risk of false–positive results. On the other hand, failure to provide the most appropriate examination may result in inefficient diagnosis and ineffective treatment. From the perspective of ‘accountability’, inappropriate services are a threat to the effective allocation of resources to health care.
A significant threat to appropriate imaging is lack of knowledge. The task of keeping up-to-date with developments in clinical practice is increasingly difficult due to expanding research activity and publications, clinical specialization and subspecialization, and new applications of technology. It is difficult to acquire and maintain the necessary breadth and depth of knowledge for confident and correct decision-making. A ‘medical knowledge crisis’ is well recognized, as is the need for improved knowledge management [3] and electronic decision support systems [4] in the health sector. The development and deployment of decision support systems has been suggested as a specific strategy to increase the appropriateness of diagnostic testing [1].
Against this background, there is recent history of public funds being committed by governments in Australia to achieve the appropriate application of medical imaging technology to diagnosis. One strategy is to develop, disseminate, implement, and evaluate diagnostic imaging pathways, to inform and guide requestors and providers of diagnostic imaging. The objective of this article is to report the method used to develop imaging pathways. Methodologies for dissemination and implementation are also described. The outcome of a small scale trial is reported in terms of the acceptance of imaging pathways, and the impact on knowledge of diagnostic imaging and requesting practice. Barriers to implementation are described and future directions outlined.
Development of imaging pathways
Clinical and academic content
Imaging specialists at Royal Perth Hospital began developing pathways for diagnostic testing in 1992, and a document was made available to assist hospital-based requesting clinicians (especially junior doctors) in their choice of appropriate examinations for common presenting conditions. During subsequent revisions of the pathways, it became apparent that there was a need to close the gap between evidence and clinical practice, and to develop an appropriate electronic environment to facilitate dissemination and continuous updating of the pathways. There was also a need to encourage implementation and to evaluate the pathways. The Government of Western Australia approved funding in 2001 for further development, electronic dissemination and evaluation. Target users were expanded from hospital-based clinicians to general practitioners and medical students.
The suite of imaging pathways was conceived both as a decision support tool and an educational tool. Starting with previous editions, the range of topics was reviewed. Topics had been developed as much as possible from a ‘symptom complex’ point of view, that is, how patients present in a ward, clinic, or general practice (e.g., ‘dyspepsia’ rather than ‘investigation of possible peptic ulcer’).
Although only one suite of pathways was to be further developed, the needs of differing groups of users were taken into account. For general practitioners, pathways were considered for common clinical conditions (e.g., chronic headache, hypertension), complex conditions in regard to imaging options available (e.g., dyspepsia), conditions which are subject to inappropriate imaging (e.g., plain X-rays for low back pain) and conditions which have new imaging options available (e.g., Magnetic Resonance Imaging). For hospital-based junior doctors and medical students, additional criteria for pathways included conditions involving acute presentations where immediate access to expert radiological opinion may be lacking (e.g., aspects of the ‘acute abdomen’) and conditions for which there is need for education (e.g., renal colic, imaging of pancreatitis).
The process of drafting a pathway engaged a Research Fellow and a panel of imaging specialists in a number of steps. A topic was nominated or recalled from an earlier edition, and a draft was created. Literature was searched for evidence to support particular steps using PubMed [5] and Cochrane Database of Systematic Reviews [6]. Available evidence was assessed and graded using the Oxford Centre of Evidence-Based Medicine levels of evidence [7]. Nationally or internationally published and accepted guidelines were referred to, where available; for example Ottawa rules for ankle injury [8]. The draft pathway was modified on the basis of evidence and information. The draft was provided to appropriate subspecialty imaging consultants and specialist clinicians, opinions were received and revisions were made based on consensus opinion. The draft was provided to general practitioners nominated by the Royal Australian College of General Practitioners, some of whom had experience in rural and remote settings, opinions were received, and revisions were made. For selected topics, usually involving controversy, draft guidelines were presented to joint clinico-radiological meetings, and revisions were made based on consensus opinion.
A suite of 78 pathways was developed covering all organ systems except breast imaging, grouped according to human systems (gastrointestinal, central nervous system, musculoskeletal, cardiovascular, respiratory, endocrine, genito-urinary, ear/nose/throat, obstetrics and gynaecology), and prepared for common presenting symptoms (e.g. chronic headache, shoulder pain, lower urinary tract symptoms) and preliminary diagnoses (e.g., suspected ectopic pregnancy, intra-abdominal abscess). The pathways were limited to adults. Most of the topics were presented as a flow-chart, with notes on imaging procedures and references quoting the level of evidence. Other content and features included a statement on the purpose of imaging pathways, general principles regarding the requesting of examinations, an explanation of levels of evidence, an introduction to common imaging modalities, and a statement about biological effects and risks of ionizing radiation.
Areas of unresolved controversy were acknowledged in the text with reference to current expert opinion and possible alternative options. The non-availability of diagnostic modalities in particular circumstances was acknowledged, for example general practitioners in Australia cannot refer patients for Magnetic Resonance Imaging, but this diagnostic modality was included in the pathways as an indicator for referral to appropriate specialists.
Electronic environment
The suite of imaging pathways will never be stable, but subject to expansion, as the need for additional topics is identified. Also, existing pathways will be subject to perpetual review and revision as new applications of technology and new evidence become available. Given this dynamic state, a decision was made to contain and deliver the clinical and academic content in an electronic form. An environment was created which allows for easy input of revised clinical and academic content, operates on most personal computers, and enables distribution via compact disk, a local area network, or the Internet.
Early drafts and ideas were submitted to the Research Fellow and Editors for suggestion and comment. The environment was then presented to potential users for their review, leading to functional enhancements. Clinical and academic content was loaded into the environment progressively after it was approved for release. Testing was carried out to ensure the product was free of content and technical error. Functional features of the electronic environment include intuitive navigation but with a user guide provided, ‘pop-up’ summaries of information when the cursor was placed over shaded elements, internal links to references, a search facility, and ‘printer friendly’ versions. Figure 1 provides an example of an imaging pathway.
Diagnostic imaging pathways menu and an example of one imaging pathway (suspected bowel obstruction).
Dissemination and implementation of imaging pathways
Imaging pathways were distributed in two ways; via the hospital local area network and thus available on any computer that was logged onto the network, as well as on compact disks. Network delivery had the advantage of being fast and also allowing for delivery from a master source, so that changes in content could be made available instantaneously across the network. Network delivery, however, has the disadvantage that computers need to be connected to the hospital network to gain access to imaging pathways. Distribution on a compact disk was designed to address this shortcoming, making the pathways available to general practitioners and any other clinician unable to log onto the network. Imaging pathways could be copied from the compact disk to a personal computer.
Imaging pathways were available to all hospital-based doctors whenever they logged onto the network from a personal computer or laptop. Selection of a prominent icon on the ‘desktop’ resulted in the display of the homepage which included a drop-down menu of locations and topics, and the search facility. Sixty compact disks were also distributed to hospital-based doctors.
Sixteen compact disks were distributed to general practitioners. There were problems with the installation of imaging pathways on some general practitioners’ computers which affected some aspects of its functionality. All problems were not resolved during the trial, and this affected the way some general practitioners judged the product.
It is recognized that simply making guidelines available to clinicians is insufficient to get them accepted and used [9]. A vigorous and multifaceted approach is required to change clinicians’ behaviour [10]. Ideally, the approach should include the use of opinion leaders and ‘champions’, endorsement by clinical groups, local adaptation, practice visits from influential experts, small group workshops, felicitous use of information technology, reminder systems, provision of incentives (or at least the removal of disincentives), etc. The dissemination and implementation methodology employed most of these facets and had two phases; raising general awareness, followed by intensive marketing activities. For the latter, two groups of clinicians were targeted; hospital-based clinicians in the Departments of General Medicine, General Surgery and Emergency Medicine, and general practitioners in two Divisions of General Practice. Table 1 summarizes the methodology.
Dissemination and implementation methodology for hospital-based clinicians and general practitioners
| Phase | Hospital-based clinicians | General practitioners |
|---|---|---|
| Raising general awareness | Presentations to hospital executive committees, hospital grand round, professional development meetings of medical imaging technologistsPosters in strategic areas of the hospitalArticles in newsletters | ‘Mail-out’ advertising imaging pathways, the trial dissemination and evaluation, and the award of continuing professional development points for participation |
| Intensive marketing | Senior registrars recruited to encourage junior medical officers to refer to imaging pathwaysHeads of Department support enlistedPresentations to Departmental meetings including demonstration of imaging pathways and method of access via the hospital network, distribution and collection of evaluation material, notification of reward for ongoing participation in trial | Education seminar and visits to individual general practitioners including demonstration of imaging pathways and provision of compact disk, distribution of evaluation material, notification of reward for ongoing participation in trial |
| Phase | Hospital-based clinicians | General practitioners |
|---|---|---|
| Raising general awareness | Presentations to hospital executive committees, hospital grand round, professional development meetings of medical imaging technologistsPosters in strategic areas of the hospitalArticles in newsletters | ‘Mail-out’ advertising imaging pathways, the trial dissemination and evaluation, and the award of continuing professional development points for participation |
| Intensive marketing | Senior registrars recruited to encourage junior medical officers to refer to imaging pathwaysHeads of Department support enlistedPresentations to Departmental meetings including demonstration of imaging pathways and method of access via the hospital network, distribution and collection of evaluation material, notification of reward for ongoing participation in trial | Education seminar and visits to individual general practitioners including demonstration of imaging pathways and provision of compact disk, distribution of evaluation material, notification of reward for ongoing participation in trial |
Dissemination and implementation methodology for hospital-based clinicians and general practitioners
| Phase | Hospital-based clinicians | General practitioners |
|---|---|---|
| Raising general awareness | Presentations to hospital executive committees, hospital grand round, professional development meetings of medical imaging technologistsPosters in strategic areas of the hospitalArticles in newsletters | ‘Mail-out’ advertising imaging pathways, the trial dissemination and evaluation, and the award of continuing professional development points for participation |
| Intensive marketing | Senior registrars recruited to encourage junior medical officers to refer to imaging pathwaysHeads of Department support enlistedPresentations to Departmental meetings including demonstration of imaging pathways and method of access via the hospital network, distribution and collection of evaluation material, notification of reward for ongoing participation in trial | Education seminar and visits to individual general practitioners including demonstration of imaging pathways and provision of compact disk, distribution of evaluation material, notification of reward for ongoing participation in trial |
| Phase | Hospital-based clinicians | General practitioners |
|---|---|---|
| Raising general awareness | Presentations to hospital executive committees, hospital grand round, professional development meetings of medical imaging technologistsPosters in strategic areas of the hospitalArticles in newsletters | ‘Mail-out’ advertising imaging pathways, the trial dissemination and evaluation, and the award of continuing professional development points for participation |
| Intensive marketing | Senior registrars recruited to encourage junior medical officers to refer to imaging pathwaysHeads of Department support enlistedPresentations to Departmental meetings including demonstration of imaging pathways and method of access via the hospital network, distribution and collection of evaluation material, notification of reward for ongoing participation in trial | Education seminar and visits to individual general practitioners including demonstration of imaging pathways and provision of compact disk, distribution of evaluation material, notification of reward for ongoing participation in trial |
Evaluation of imaging pathways
It is recognized that any dissemination of clinical practice guidelines should include an evaluation of the quality and acceptability of the guideline, the dissemination and implementation methodology, and whether the guidelines make any difference to clinical practice [9]. The evaluation had two phases; Assessment 1 at the onset of intensive marketing and Assessment 2 following a period of intensive marketing. Table 2 summarizes the evaluation methodology.
Evaluation methodology for hospital-based clinicians and general practitioners
| Parameter | Hospital-based clinicians | General practitioners |
|---|---|---|
| Alignment between knowledge of diagnostic imaging and imaging pathways | Multiple choice questionnaire | Multiple choice questionnaire |
| • seven common clinical presentations and 36 diagnostic imaging statements | • seven common clinical presentations and 35 diagnostic imaging statements | |
| • requesting clinicians marked each statement as ‘true’, ‘false’ or ‘unsure’ | • requesting clinicians marked each statement as ‘true’, ‘false’ or ‘unsure’ | |
| • correct responses were summed and expressed as per cent of 36 items | • correct responses were summed and expressed as per cent of 35 items | |
| Alignment between actual requesting practice and imaging pathways | Record obtained of all admissions on a single day to the Emergency Department. For each admission, the presenting symptom, provisional diagnosis, and diagnostic imaging examination obtained. Imaging specialist rated each admission in the following categories: | General practitioners completed a form for each of 20 patients seen in their clinical practice who required diagnostic imaging, providing details of presenting symptoms/preliminary diagnosis, duration of symptoms, relevant physical signs, previous imaging, imaging currently ordered. Imaging specialist assessed the information and rated each consultation in one of the following categories: |
| • ‘imaging compliant with imaging pathways’ | • ‘compliant with imaging pathways’ | |
| • ‘appropriate imaging but topic not included’ | • ‘not compliant’ | |
| • ‘imaging not compliant’ | ||
| • ‘insufficient information to rate’ | ||
| Requesting clinicians’ judgement of usefulness of imaging pathways, barriers to use and suggestions for improvement | Qualitative questionnaires which included items relating to: | Qualitative questionnaire which included items relating to: |
| • ‘awareness and knowledge’—awareness and use of imaging pathways | • ‘awareness and knowledge’—knowledge of diagnostic imaging, confidence in requesting, knowledge of imaging pathways | |
| • ‘attitude to pathways’ — perceived need for, confidence in, and barriers to use | • ‘attitude to pathways’—perceived need for, confidence in, and barriers to use’ | |
| • ‘attitude to electronic format’—functionality of electronic environment | • ‘attitude to electronic format’ | |
| • ‘comments’—a section for further comments and suggestions | • ‘comments’ |
| Parameter | Hospital-based clinicians | General practitioners |
|---|---|---|
| Alignment between knowledge of diagnostic imaging and imaging pathways | Multiple choice questionnaire | Multiple choice questionnaire |
| • seven common clinical presentations and 36 diagnostic imaging statements | • seven common clinical presentations and 35 diagnostic imaging statements | |
| • requesting clinicians marked each statement as ‘true’, ‘false’ or ‘unsure’ | • requesting clinicians marked each statement as ‘true’, ‘false’ or ‘unsure’ | |
| • correct responses were summed and expressed as per cent of 36 items | • correct responses were summed and expressed as per cent of 35 items | |
| Alignment between actual requesting practice and imaging pathways | Record obtained of all admissions on a single day to the Emergency Department. For each admission, the presenting symptom, provisional diagnosis, and diagnostic imaging examination obtained. Imaging specialist rated each admission in the following categories: | General practitioners completed a form for each of 20 patients seen in their clinical practice who required diagnostic imaging, providing details of presenting symptoms/preliminary diagnosis, duration of symptoms, relevant physical signs, previous imaging, imaging currently ordered. Imaging specialist assessed the information and rated each consultation in one of the following categories: |
| • ‘imaging compliant with imaging pathways’ | • ‘compliant with imaging pathways’ | |
| • ‘appropriate imaging but topic not included’ | • ‘not compliant’ | |
| • ‘imaging not compliant’ | ||
| • ‘insufficient information to rate’ | ||
| Requesting clinicians’ judgement of usefulness of imaging pathways, barriers to use and suggestions for improvement | Qualitative questionnaires which included items relating to: | Qualitative questionnaire which included items relating to: |
| • ‘awareness and knowledge’—awareness and use of imaging pathways | • ‘awareness and knowledge’—knowledge of diagnostic imaging, confidence in requesting, knowledge of imaging pathways | |
| • ‘attitude to pathways’ — perceived need for, confidence in, and barriers to use | • ‘attitude to pathways’—perceived need for, confidence in, and barriers to use’ | |
| • ‘attitude to electronic format’—functionality of electronic environment | • ‘attitude to electronic format’ | |
| • ‘comments’—a section for further comments and suggestions | • ‘comments’ |
Evaluation methodology for hospital-based clinicians and general practitioners
| Parameter | Hospital-based clinicians | General practitioners |
|---|---|---|
| Alignment between knowledge of diagnostic imaging and imaging pathways | Multiple choice questionnaire | Multiple choice questionnaire |
| • seven common clinical presentations and 36 diagnostic imaging statements | • seven common clinical presentations and 35 diagnostic imaging statements | |
| • requesting clinicians marked each statement as ‘true’, ‘false’ or ‘unsure’ | • requesting clinicians marked each statement as ‘true’, ‘false’ or ‘unsure’ | |
| • correct responses were summed and expressed as per cent of 36 items | • correct responses were summed and expressed as per cent of 35 items | |
| Alignment between actual requesting practice and imaging pathways | Record obtained of all admissions on a single day to the Emergency Department. For each admission, the presenting symptom, provisional diagnosis, and diagnostic imaging examination obtained. Imaging specialist rated each admission in the following categories: | General practitioners completed a form for each of 20 patients seen in their clinical practice who required diagnostic imaging, providing details of presenting symptoms/preliminary diagnosis, duration of symptoms, relevant physical signs, previous imaging, imaging currently ordered. Imaging specialist assessed the information and rated each consultation in one of the following categories: |
| • ‘imaging compliant with imaging pathways’ | • ‘compliant with imaging pathways’ | |
| • ‘appropriate imaging but topic not included’ | • ‘not compliant’ | |
| • ‘imaging not compliant’ | ||
| • ‘insufficient information to rate’ | ||
| Requesting clinicians’ judgement of usefulness of imaging pathways, barriers to use and suggestions for improvement | Qualitative questionnaires which included items relating to: | Qualitative questionnaire which included items relating to: |
| • ‘awareness and knowledge’—awareness and use of imaging pathways | • ‘awareness and knowledge’—knowledge of diagnostic imaging, confidence in requesting, knowledge of imaging pathways | |
| • ‘attitude to pathways’ — perceived need for, confidence in, and barriers to use | • ‘attitude to pathways’—perceived need for, confidence in, and barriers to use’ | |
| • ‘attitude to electronic format’—functionality of electronic environment | • ‘attitude to electronic format’ | |
| • ‘comments’—a section for further comments and suggestions | • ‘comments’ |
| Parameter | Hospital-based clinicians | General practitioners |
|---|---|---|
| Alignment between knowledge of diagnostic imaging and imaging pathways | Multiple choice questionnaire | Multiple choice questionnaire |
| • seven common clinical presentations and 36 diagnostic imaging statements | • seven common clinical presentations and 35 diagnostic imaging statements | |
| • requesting clinicians marked each statement as ‘true’, ‘false’ or ‘unsure’ | • requesting clinicians marked each statement as ‘true’, ‘false’ or ‘unsure’ | |
| • correct responses were summed and expressed as per cent of 36 items | • correct responses were summed and expressed as per cent of 35 items | |
| Alignment between actual requesting practice and imaging pathways | Record obtained of all admissions on a single day to the Emergency Department. For each admission, the presenting symptom, provisional diagnosis, and diagnostic imaging examination obtained. Imaging specialist rated each admission in the following categories: | General practitioners completed a form for each of 20 patients seen in their clinical practice who required diagnostic imaging, providing details of presenting symptoms/preliminary diagnosis, duration of symptoms, relevant physical signs, previous imaging, imaging currently ordered. Imaging specialist assessed the information and rated each consultation in one of the following categories: |
| • ‘imaging compliant with imaging pathways’ | • ‘compliant with imaging pathways’ | |
| • ‘appropriate imaging but topic not included’ | • ‘not compliant’ | |
| • ‘imaging not compliant’ | ||
| • ‘insufficient information to rate’ | ||
| Requesting clinicians’ judgement of usefulness of imaging pathways, barriers to use and suggestions for improvement | Qualitative questionnaires which included items relating to: | Qualitative questionnaire which included items relating to: |
| • ‘awareness and knowledge’—awareness and use of imaging pathways | • ‘awareness and knowledge’—knowledge of diagnostic imaging, confidence in requesting, knowledge of imaging pathways | |
| • ‘attitude to pathways’ — perceived need for, confidence in, and barriers to use | • ‘attitude to pathways’—perceived need for, confidence in, and barriers to use’ | |
| • ‘attitude to electronic format’—functionality of electronic environment | • ‘attitude to electronic format’ | |
| • ‘comments’—a section for further comments and suggestions | • ‘comments’ |
Hospital-based clinicians
Fifty-four hospital-based doctors were enlisted in the intensive marketing of imaging pathways and completed all components of Assessment 1, but there were fewer participants in Assessment 2. Table 3 summarizes key results from the evaluation.
Summary of results comparing Assessments 1 and 2 with respect to general practitioners
| Parameter | Measure | Assessment | |||
|---|---|---|---|---|---|
| 1 | 2 | ||||
| Alignment between knowledge of imaging and imaging pathways | Correct responses on the 36 item multiple choice questionnaire | Mean % correct responses on the questionnaire | 71 | 79 | |
| Awareness and knowledge of imaging pathways | Respondents identifying where to access imaging pathways | % identifying where to access | 19 | 85 | |
| Respondents using imaging pathways | % using | 15 | 63 | ||
| Respondents giving a high rating to ease of use of imaging pathways | % giving high rating | 11 | 67 | ||
| Attitude to imaging pathways | Respondents having confidence that imaging pathways are up-to-date | % having confidence | 69 | 85 | |
| Respondents having confidence that imaging pathways are comprehensive | % having confidence | 61 | 85 | ||
| Respondents describing barriers to use of imaging pathways | % describing barriers | 78 | 74 | ||
| Attitude to electronic format | Respondents considering elements of the electronic format inadequate | % considering inadequate | 72 | 59 | |
| Respondents suggesting improvement to the electronic format | % suggesting improvement | 15 | 19 | ||
| Parameter | Measure | Assessment | |||
|---|---|---|---|---|---|
| 1 | 2 | ||||
| Alignment between knowledge of imaging and imaging pathways | Correct responses on the 36 item multiple choice questionnaire | Mean % correct responses on the questionnaire | 71 | 79 | |
| Awareness and knowledge of imaging pathways | Respondents identifying where to access imaging pathways | % identifying where to access | 19 | 85 | |
| Respondents using imaging pathways | % using | 15 | 63 | ||
| Respondents giving a high rating to ease of use of imaging pathways | % giving high rating | 11 | 67 | ||
| Attitude to imaging pathways | Respondents having confidence that imaging pathways are up-to-date | % having confidence | 69 | 85 | |
| Respondents having confidence that imaging pathways are comprehensive | % having confidence | 61 | 85 | ||
| Respondents describing barriers to use of imaging pathways | % describing barriers | 78 | 74 | ||
| Attitude to electronic format | Respondents considering elements of the electronic format inadequate | % considering inadequate | 72 | 59 | |
| Respondents suggesting improvement to the electronic format | % suggesting improvement | 15 | 19 | ||
Summary of results comparing Assessments 1 and 2 with respect to general practitioners
| Parameter | Measure | Assessment | |||
|---|---|---|---|---|---|
| 1 | 2 | ||||
| Alignment between knowledge of imaging and imaging pathways | Correct responses on the 36 item multiple choice questionnaire | Mean % correct responses on the questionnaire | 71 | 79 | |
| Awareness and knowledge of imaging pathways | Respondents identifying where to access imaging pathways | % identifying where to access | 19 | 85 | |
| Respondents using imaging pathways | % using | 15 | 63 | ||
| Respondents giving a high rating to ease of use of imaging pathways | % giving high rating | 11 | 67 | ||
| Attitude to imaging pathways | Respondents having confidence that imaging pathways are up-to-date | % having confidence | 69 | 85 | |
| Respondents having confidence that imaging pathways are comprehensive | % having confidence | 61 | 85 | ||
| Respondents describing barriers to use of imaging pathways | % describing barriers | 78 | 74 | ||
| Attitude to electronic format | Respondents considering elements of the electronic format inadequate | % considering inadequate | 72 | 59 | |
| Respondents suggesting improvement to the electronic format | % suggesting improvement | 15 | 19 | ||
| Parameter | Measure | Assessment | |||
|---|---|---|---|---|---|
| 1 | 2 | ||||
| Alignment between knowledge of imaging and imaging pathways | Correct responses on the 36 item multiple choice questionnaire | Mean % correct responses on the questionnaire | 71 | 79 | |
| Awareness and knowledge of imaging pathways | Respondents identifying where to access imaging pathways | % identifying where to access | 19 | 85 | |
| Respondents using imaging pathways | % using | 15 | 63 | ||
| Respondents giving a high rating to ease of use of imaging pathways | % giving high rating | 11 | 67 | ||
| Attitude to imaging pathways | Respondents having confidence that imaging pathways are up-to-date | % having confidence | 69 | 85 | |
| Respondents having confidence that imaging pathways are comprehensive | % having confidence | 61 | 85 | ||
| Respondents describing barriers to use of imaging pathways | % describing barriers | 78 | 74 | ||
| Attitude to electronic format | Respondents considering elements of the electronic format inadequate | % considering inadequate | 72 | 59 | |
| Respondents suggesting improvement to the electronic format | % suggesting improvement | 15 | 19 | ||
In regard to the alignment between requesting practice and imaging pathways, 7% of 135 emergency admissions were rated ‘not compliant with imaging pathways’ in Assessment 1, whereas 1% of 132 admissions were rated not compliant in Assessment 2.
The dissemination and implementation methodology succeeded in raising awareness and knowledge of imaging pathways, and raising confidence in the content and usefulness of the pathways. Imaging pathways were judged to be useful in the following circumstances; in choosing between potentially useful examinations when more experienced staff were unavailable for consultation, in unfamiliar or problematic clinical presentations, for resolving uncertainty arising from conflicting opinion, when justifying the choice of particular examinations, and when justifying not ordering examinations.
Respondents identified some ‘professional’ barriers to the use of imaging pathways in clinical decision-making, for example, in situations where senior and experienced staff were the primary determiners of which examinations were requested, and when specialists ‘knew’ what tests they wanted. Respondents also identified ‘technological’ barriers, for example, when computers were not available where and when decisions were made, or were too slow.
Various suggestions were made for improvements to the clinical and academic content, for example, the inclusion of abstracts of referenced articles, the addition of diagnostic images to illustrate abnormalities, and the addition of more information on imaging processes and imaging modalities. Suggestions were also provided regarding the electronic environment, for example, to achieve availability on wireless devices and to reduce the size of flow-charts to reduce the need for scrolling.
General practitioners
Nine general practitioners were enlisted in the intensive marketing of imaging pathways and completed all components of Assessment 1, but there were only five participants in Assessment 2. The low number of participants makes it difficult to be confident about the generality of results. Nonetheless Table 4 summarizes key results from the evaluation.
Summary of results comparing Assessments 1 and 2 with respect to general practitioners
| Parameter | Measure | Assessment | |||
|---|---|---|---|---|---|
| 1 | 2 | ||||
| Alignment between knowledge of diagnostic imaging and imaging pathways | Correct responses on the 35-item multiple choice questionnaire | No. of respondents | 9 | 5 | |
| Mean % correct responses on the questionnaire | 72 | 78 | |||
| Alignment between actual requesting practice and imaging pathways | Patients with ‘imaging not compliant with imaging pathways’ | No. of patients | 39 | 60 | |
| % ‘not compliant’ | 13 | 8 | |||
| Parameter | Measure | Assessment | |||
|---|---|---|---|---|---|
| 1 | 2 | ||||
| Alignment between knowledge of diagnostic imaging and imaging pathways | Correct responses on the 35-item multiple choice questionnaire | No. of respondents | 9 | 5 | |
| Mean % correct responses on the questionnaire | 72 | 78 | |||
| Alignment between actual requesting practice and imaging pathways | Patients with ‘imaging not compliant with imaging pathways’ | No. of patients | 39 | 60 | |
| % ‘not compliant’ | 13 | 8 | |||
Summary of results comparing Assessments 1 and 2 with respect to general practitioners
| Parameter | Measure | Assessment | |||
|---|---|---|---|---|---|
| 1 | 2 | ||||
| Alignment between knowledge of diagnostic imaging and imaging pathways | Correct responses on the 35-item multiple choice questionnaire | No. of respondents | 9 | 5 | |
| Mean % correct responses on the questionnaire | 72 | 78 | |||
| Alignment between actual requesting practice and imaging pathways | Patients with ‘imaging not compliant with imaging pathways’ | No. of patients | 39 | 60 | |
| % ‘not compliant’ | 13 | 8 | |||
| Parameter | Measure | Assessment | |||
|---|---|---|---|---|---|
| 1 | 2 | ||||
| Alignment between knowledge of diagnostic imaging and imaging pathways | Correct responses on the 35-item multiple choice questionnaire | No. of respondents | 9 | 5 | |
| Mean % correct responses on the questionnaire | 72 | 78 | |||
| Alignment between actual requesting practice and imaging pathways | Patients with ‘imaging not compliant with imaging pathways’ | No. of patients | 39 | 60 | |
| % ‘not compliant’ | 13 | 8 | |||
There were slight differences between Assessment 1 and Assessment 2 questionnaires, making it difficult to achieve direct comparisons as reported for hospital-based clinicians (Table 3). However, in Assessment 2, all respondents demonstrated a working knowledge of imaging pathways, all were confident that imaging pathways were up-to-date, all provided suggestions for improvement to the clinical and academic content, all found imaging pathways quick and easy to access and use, and 33% of respondents provided suggestions for improvement to the electronic environment.
Imaging pathways were judged to be useful in the following circumstances; in assessing uncommon and unfamiliar conditions, in providing reassurance to patients, in acting as a reference tool for the most effective test and recently available examination, and in supporting decisions.
Respondents identified ‘professional’ barriers to using imaging pathways, for example, in situations where patient expectations influenced decisions, or when there was not enough evidence to support a recommendation. Respondents also identified ‘technological’ barriers, for example, when it was difficult to find the required pathway, when there was insufficient time to access and follow the pathways, when computers were not available.
Suggestions for improvement to the clinical and academic content included; more pathways relevant to general practice, a classification system according to anatomy or symptom and not the current hybrid, the inclusion of information about costs. Suggestions regarding the electronic environment included; integration with other information systems used by general practitioners, incorporation into on-line learning programs, access over the Internet, the provision of reminders or prompts to access and follow the pathways.
Outcome and lessons learned
It is our view that the development of diagnostic imaging pathways based on consensus opinion of providers and requestors of imaging services, and supported by the best available evidence, is proving to be a worthwhile endeavour. The pathways encouraged ‘best practice’ in the requesting of diagnostic imaging examinations. When they were disseminated to hospital-based clinicians and to general practitioners, and following a period of education and intensive ‘marketing’, knowledge of diagnostic imaging and requesting behaviour tended to become more aligned with the pathways (Tables 3 and 4). Confirmation of this outcome with larger samples of requesting clinicians is required. Also, it should be determined whether requesting behaviour remains aligned with imaging pathways following the cessation of intensive marketing. However, the central objective to reduce the incidence of inappropriate diagnostic imaging seems to be achievable.
Although most of the participating clinicians expressed confidence in the currency and comprehensiveness of the clinical and academic content (Table 3), and were able to give examples of situations in which the pathways assisted in decision-making, comments were received regarding the need for additional pathways and additional associated information. Addressing such needs in future developments will enhance the education and decision-support value of imaging pathways.
Although most of the clinicians found the electronic method of containing and delivering the imaging pathways generally satisfactory (Table 3), and navigation through the content easy, numerous comments were received regarding the need for modifications, enhancements, and customization. It is essential that the needs of end-users be fully identified and addressed in future developments, if accessing the pathways and following the recommendations is to be encouraged.
It is our view that achieving excellence in the clinical and academic content of imaging pathways and a sophisticated electronic environment to contain and deliver the pathways are not the only the requirements for achieving maximum alignment between the pathways and requesting behaviour. Certainly, the pathways need to be available quickly and easily, at the time and place where clinical decisions are being made, for example on a ward-round at the bedside, or during a consultation with a patient. Ideally, they need to be embedded in other information systems usually deployed by clinicians in the clinical setting.
However, eliminating or minimizing barriers will not guarantee maximum utilization and compliance. We note that it was difficult to recruit and retain large numbers of participants in the trial dissemination and evaluation (Tables 3 and 4), even though rewards in the form of prizes and points for Continuing Professional Development were offered. We also note that after much effort in raising general awareness in Royal Perth Hospital and intensive marketing activities, significant sections of clinical staff remained unaware of the pathways.
We believe that changing the behaviour of requesting clinicians poses a significant challenge. Imaging pathways were supplied as a ‘stand-alone’ product, and accessing and following the recommendations was optional. There was no disincentive for experienced clinicians to continue with past requesting practices, whether or not they were aligned with the pathways, and it was ‘too easy’ for junior doctors to passively accept direction from their supervising clinicians. General practitioners succumbed to patient expectation of action, erred on the side of caution, and requested examinations with negligible benefit.
Because the goal is for clinicians to request diagnostic examinations according to ‘best practice’ as embodied in imaging pathways (unless there are cogent reasons for doing otherwise), perhaps reference to imaging pathways should be made less ‘optional’ and more ‘mandatory’. The following type of requesting process can be envisaged. Imaging pathways could be linked to an electronic process of ordering diagnostic examinations, thus achieving a true decision-support system. Such a system could provide alerts if a request is ‘non-compliant’. Overwriting an alert might be authorized by a senior requesting clinician in consultation with an imaging specialist. But the assignment of a higher cost, or a lower remuneration, to non-authorized and non-compliant imaging would provide tangible incentives to comply with imaging pathways unless there are compelling clinical contraindications.
Acknowledgements
The Health Department of Western Australia, in particular Alphonse Roex and Sally Skevington, provided broad direction and approved core funding for the project.
The Division of Imaging Services at Royal Perth Hospital, in particular the Director (Mark Khangure), provided accommodation and infrastructure for the project and released staff to carry out particular tasks.
The Canning Division of General Practice in particular Carolyn Laurence and John Newman and the Perth & Hills Division of General Practice in particular Shauna Gaebler, provided expertise and resources to all phases of the project, and especially to the trial dissemination and evaluation.
The Royal Australian College of General Practitioners provided input to determine the content of imaging pathways.
The School of Population Health, University of Western Australia, in particular D’Arcy Holman and Judith Finn, provided advice and assistance particularly in the design of dissemination and evaluation methodologies, and in the analysis of data.
Heads of the Departments of Emergency Medicine (Greg Sweetman), General Surgery (Sudhakar Rao) and General Medicine (Simon Dimmitt) at Royal Perth Hospital supported the trial dissemination and evaluation in their departments, including intensive marketing activities.
Senior Registrars (Lai-Kun Kho and Michael Warner) provided advice regarding the dissemination and evaluation methodology deployed at Royal Perth Hospital.
The Research Officers, Lisa Towler for the hospital trial, and Taryn Hunt and Sarah Jones for the general practice trial, were responsible for conducting the trial dissemination and evaluation.
References
OpenClinical Knowledge Management for Medical Care. The medical knowledge crisis and its solution through knowledge management: http://www.openclinical.org/whitepaper.html Accessed 8 April 2005.
PubMed. National centre for biomedical information: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi Accessed 8 April 2005.
The Cochrane Database of Systematic Reviews. The Cochrane Library: http://www.nicsl.com.au/cochrane/index.asp Accessed 8 April 2005.
National Health and Medical Research Council.
