Aims Identifying and reducing low-value care is a vital issue in Australia, with pathology test ordering a common focus in this field. This study builds on previous research and aimed to quantify the impact of the implementation of an electronic ordering (e-ordering) system on the volume of pathology testing, compared with manual (paper based) ordering.
Methods An audit and analysis of pathology test data were conducted, using an interrupted time series design to investigate the impact of the e-ordering system on pathology ordering patterns. All medical and surgical adult inpatients at a tertiary referral hospital in Newcastle, Australia, were included over a 3-year period.
Results Overall, there were no statistically significant differences in the volume of orders due to the implementation of the e-ordering system. There was a slight increase in the aggregated volume (tests per admission and tests per bed day) of tests ordered across the entire study period, reflecting a secular trend.
Conclusions Despite providing greater visibility and tracking of orders, we conclude that the implementation of an e-ordering system does not, in and of itself, reduce ordering volume. Efforts to identify and reduce low-value care will require intentional effort and specifically designed educational programmes or hard-wired algorithms.
- Pathology Department, Hospital
Data availability statement
Data may be obtained from a third party and are not publicly available. De-identifed data were applied for and supplied by New South Wales Health Pathology, using the Health Information Exchange.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
What is already known on this topic
Pathology testing is used to guide clinical decision-making in hospitals but can also contribute to low value care. Changes from paper-based to electronic pathology test ordering may make ordering easier and, therefore, increase the overall volume of tests.
What this study adds
This study quantifies the volume and pattern of pathology test ordering in one Australian hospital, comparing paper-based and electronic test ordering. This study found a temporal increase in test numbers but no change attributable to the introduction of an electronic ordering system.
How this study might affect research, practice or policy
Efforts to identify and reduce low-value care may need to consider hard-wired algorithms built into electronic ordering systems and other educational tools integrated into medical training.
With increasing healthcare costs and an ageing population, identifying and resolving causes of low-value care is an important issue.1 Low-value care is broadly defined as care that provides little or no additional benefit to patients with respect to the associated risks of harm and healthcare costs.2
It is estimated that 70% of clinical decisions are informed by pathology testing.3 4 This has likely driven the increase in ordering observed over the last two decades, with a rise from 40 million tests in the year 2000 to 90 million in 2015, accounting for 3% of the Australian healthcare budget in 2015.5 Furthermore, pathology testing can trigger a cascade of additional tests, investigations and procedures.6 Underordering of pathology tests is a broad and complex problem which is outside the scope of this study.3 It is vital to optimise efficiency in this area of healthcare to maintain and improve the quality and cost-effectiveness of care.2 5
It is estimated that between 21% and 69% of all laboratory tests are unnecessary.3 7 There is extreme variability in outcomes from these studies because the analyses have ranged from subjective to objective, with both restrictive and permissive criteria.3 Test appropriateness is determined through risk/benefit analysis and only some clinical scenarios have good evidence for appropriate test or retest intervals.8–10 Objective measures, such as minimum retest intervals, are useful tools because they can categorise tests as clearly appropriate or inappropriate regardless of the clinical context.10
Studies have shown patterns of extended, habitual, repeat testing for common laboratory tests that are low yield and clinically unjustified.11 12 Our previous study found that repeat full blood count (FBC) and Urea-electrolyte-creatinine (UEC) tests were ordered every 24 hours for patients with a Length of Stay (LOS) >2 days, even when the previous result was within normal limits.11 Only 0.3% of patients received a high-risk result following a normal UEC test. This suggests that routine pathology testing to detect emerging problems is very low yield.11 Most physicians (90%) cited a training culture that emphasised repeat daily tests when they were asked why they ordered unnecessary tests.13 Clinical guidelines that explicitly address this low-value clinical culture, and lessen the subjectivity and permissiveness of laboratory test ordering, may help reduce low-value care.14
It has been hypothesised that electronic ordering (e-ordering) (given various names in the literature, including electronic requesting, e-requesting or computerised physician order entry) systems could address problems of low-value care by decreasing laboratory test turnaround time and reducing the volume of tests ordered, including reducing duplicate tests.15–17 E-ordering systems are electronic ordering systems for medications and clinical investigations such as pathology and imaging.15 To date, the effect of e-ordering systems on pathology test ordering patterns and patient outcomes has been mixed.18–21 Studies have shown improved outcomes following e-ordering implementation in various settings (emergency department (ED), inpatient, outpatient). These include decreased total volume of tests ordered (ED, outpatient),18 19 22 reduced repeat labs (inpatient)22 and decreased test turnaround time and patient LOS.21 23 24 Others have shown no effect on volume of inpatient test orders18 20 or patient outcomes including intensive care readmission, hospital mortality or ventilator days.25–28 Yet other studies have shown a negative impact of the e-ordering system such as increased test orders (ED)21 and increased medical errors and adverse drug events.15 29–31; this may potentially be facilitated by habitual reordering and creating standing orders.7 13 32 Previous studies have investigated various pathology tests to measure overordering. In this study, only the most commonly ordered, or commonly over-ordered, pathology tests were examined.33 For example, FBC was found to be the most commonly ordered test in our previous study11 and vitamin D has been identified as a commonly over-ordered test by the ‘Choosing Wisely’ campaign.34 35
E-ordering is still relatively new in Australia and few studies have investigated the effect of digital systems on laboratory test ordering patterns in local hospitals.20–22 These studies produced inconsistent results, likely caused by study and technological limitations, for example, short collection periods, inadequate ward involvement and narrow patient populations.20–22 To address these issues, this study aimed to analyse the volume of test ordering before, during and after e-ordering implementation to determine if the new e-ordering system had any impact.
An audit and analysis of pathology test data were conducted, using an interrupted time series design,36 to investigate the impact of the e-ordering system on pathology ordering patterns for inpatients at John Hunter Hospital (JHH), NSW. This paper is a continuation of the Pathology Laboratory Unnecessary Test Ordering study.11 The current study focuses on the impact of the introduction of an e-ordering system on the volume and pattern of test ordering when compared with manual, paper-based, ordering. The e-ordering system implemented does have an alert—a small blue triangle—that indicates when a test has been ordered within the previous 24 hours.
Setting and sample
JHH is the largest tertiary hospital within the Hunter New England Health District. It has onsite laboratory services and approximately 650 beds across medical and surgical wards.11 JHH began the rollout of an e-ordering system in 2018. Pathology test orders were divided into three time periods: 1 January to 31 December 2017 was classified as the ‘pre’ implementation period, 1 January to 31 December 2018 as the ‘during-’ implementation period and 1 January to 31 December 2019 comprised the ‘post’ implementation period. This allowed observation of any pattern changes in laboratory test ordering from preimplementation to postimplementation of the e-ordering system, including the during-implementation period where e-ordering was rolled-out in a phased introduction across all wards over 12 months. Of note, the COVID-19 pandemic had no impact on this study as the data were collected prior to 2020.
Data were collected on all adult (≥18 years of age) medical and surgical patients admitted at JHH from 1 January 2017 to 31 December 2019. Emergency medicine, paediatric, obstetric, gynaecological, intensive care, psychiatric, elective day procedures (eg, endoscopy), rehabilitation, physiotherapy, general practitioner and renal dialysis admissions were excluded as use of pathology (types and frequency of tests) in these wards differs from medical/surgical wards. Contiguous admissions (readmission within 24 hours post-discharge) were collapsed into one admission. Admissions that started or finished outside the audit period were truncated (date of admission assigned as 1 January 2017, date of discharge assigned as 31 December 2019). For each admission, the LOS was rounded up to whole days. The number of occupied hospital ‘bed days’ for each month was determined per patient and summed across patients.
Deidentified data were extracted from the Patient Administration System and were supplied by New South Wales Health Pathology (NSWHP) using the Health Information Exchange. This included relevant patient demographics; gender and age for each patient, admission and discharge dates/times (from which LOS was calculated), diagnoses (as per the International Classification of Disease, edition 10 codes) and Charlson Comorbidity Index (CCI). (For specific demographic information related to the assignment of CCI, see online supplemental appendix table A1). Laboratory data, including pathology tests ordered and results, were extracted from Auslab, the laboratory information system used in the Local Health District. Pathology tests in this hospital are ordered using CAP-Orion software.
Seven types of pathology tests were included in this study: UEC, FBC, troponin, thyroid-stimulating hormone (TSH), 25-hydroxyvitamin D (vitamin D), glycated haemoglobin (HbA1c) and C-reactive protein (CRP). Test results were classified as normal, moderate risk and high risk, based on the corresponding reference ranges used by AUSLAB.11
Statistical analysis was undertaken by an independent statistician. Descriptive statistics were presented as count (%) for dichotomous variables or mean and SD (or median (min, max)) if continuous. χ2 test was used for comparison of categorical characteristics among periods. Analysis of variance and Kruskal-Wallis were used for comparison of parametric and non-parametric distributions of continuous data, respectively.
The monthly change over time in aggregated test orders (overall, and by test type) was compared among groups using segmented negative binomial regression modelling (count outcome). Crude modelling included group (preimplementation, during implementation and postimplementation) and three segmented time intervals (pretime, during time, post-time) in months. Robust SEs were used to account for autocorrelation over time. Adjusted modelling included aggregated bed days (mean-centred), to account for any changes in bed days over time. Assumptions for negative binomial regression modelling were checked and found to be reasonable.
A slope estimate, in the form of an incidence rate ratio for a 1-month change in number of tests ordered, and corresponding 95% CIs were reported for each group for the crude and adjusted models. An overall type 3 p value (interaction between group and time) was calculated to assess overall statistical differences in slopes among the preimplementation, during-implementation and post-implementation groups. The comparison in slope estimates between preimplementation and postimplementation groups and its corresponding p value was also reported. Changes over time were visualised by plotting estimated counts (exponentiated least squares means (LS-means)) from adjusted regression models. Estimated intervention effect was calculated in two forms, level change and slope change, between preimplementation and postimplementation groups. This estimates the change in outcome attributable to the intervention while accounting for the preintervention slope for test ordering,37 see figure 1.
All statistical analyses were programmed using SAS V.9.4 (SAS Institute, Cary, North Carolina). A priori, p<0.05 (two-tailed), was used to indicate statistical significance.
A total of 78 760 distinct admissions (comprising of 54 532 patients) were included in this study. Of these, 62 503 (79.4%) had pathology tests and, in total, 2 386 682 pathology tests were ordered over the 3-year study period.
There were no clinically important differences observed in age, gender, diabetic status, CCI or LOS across the preimplementation, during-implementation and postimplementation periods (see table 1).
Pathology test-ordering patterns by period
The pathology test-ordering patterns by period (preimplementation, during-implementation and post implementation) are shown in table 2. There was a slight increase in the number of tests ordered overall across time (total number of tests per admission and per bed day). Test orders also slightly increased for UEC, FBC, troponin and CRP, but not for TSH, vitamin D or HbA1c. Figure 2 visualises this increasing trend of total number of tests ordered during the three periods.
Pathology test results by period
The pathology test results were individually examined for changes across the three time periods. Each test result was categorised as ‘normal’, ‘moderate risk’ and ‘high risk’. The proportion of moderate and high-risk tests was similar across time periods, except for TSH and HbA1c, where there was a slight increase in the proportion of moderate and high-risk results from preimplementation to postimplementation, see online supplemental appendix table A2.
Overall pathology test ordering
A comparison of test ordering among time groups showed increases in overall test ordering seen in the preimplementation (0.4% increase per month) and postimplementation (0.7% increase per month) periods, representing overall average increases of 242 and 489 tests per month, respectively (table 3). However, this was not found to be statistically significant with no significant slope difference (p=0.108) or estimated intervention effect (step difference) (p=0.645) between these periods. The mean-centred bed days were included as a covariate in all adjusted models as it was statistically associated with the outcome in all models (p value <0.001).
Individual pathology tests
There were no significant differences seen in the monthly change (slope) between preimplementation and postimplementation periods for FBC (p=0.092) and TSH (p=0.433), despite significant type 3 p values for both FBC (p=0.017) and TSH (p=0.005). Although there was no significant estimated intervention (step change) for FBC (p=0.429), there was a significant intervention effect (step change) for TSH from preimplementation to postimplementation periods (23.1% estimated reduction in tests as of January 2019; p=0.038), see table 3 and figure 3. Figure 3 depicts a large increase in TSH tests ordered between preimplementation and during-implementation periods and progressive decline throughout the during-implementation period. This was unexpected as it deviates from the increasing trend seen in the preimplementation and postimplementation periods, and likely accounts for the significant type 3 p value (p=0.005).
A significant difference in the slopes between the preimplementation and postimplementation periods for UEC (p=0.030), vitamin D (p=0.010) and CRP (p=0.044) was observed, see figure 4. These may be false positives, as the respective type-3 p values were non-significant: UEC (p=0.090), vitamin D (p=0.099) and CRP (p=0.154). Unlike UEC and CRP, there was a significant estimated intervention effect for vitamin D (p=0.011) with a significant decrease in number of tests estimated in January 2019 (postimplementation period), compared with extrapolated estimates using preimplementation period slope, see table 3 and figure 4.
There were no significant differences in pretest and post-test orders for troponin and HbA1c, see table 3. The remainder of the regression lines, all non-significant, for other pathology tests is found in online supplemental appendix figure A1.
Volume of pathology tests overall
This study detected a monthly increase in test orders over the 3-year period. This likely represents the background increase in test ordering that has been reported in previous literature,3–5 rather than a change attributable to the e-ordering system implemented in this study. The small percentage increase in ordering rates (<1%) found in this study, though statistically insignificant, practically represents a large volume of laboratory tests. This increasing trend represents a resource burden on the healthcare system and is a valuable opportunity to reduce low-value care. E-ordering systems offer interventions like feedback strategies and lockout rules, which may be used to reduce inappropriate test ordering.11
Volume of individual pathology tests
There were clinically meaningful increases in the volume of tests ordered for FBC and UEC tests when comparing preimplementation and postimplementation periods. These two tests comprise the bulk of all tests ordered for inpatients and pose the greatest risk for unnecessary repeat daily orders.11 UEC and FBC should be the subject of order-intervention studies in the future considering their prominence in low-value care.11
Vitamin D and CRP showed decreases in the volume of tests ordered preimplementation to postimplementation. Vitamin D showed a large step change (drop) from the preimplementation period as well as a slope change (decreasing slope) in test order rates during implementation. No other test exhibited this change to ordering patterns during the implementation of the e-ordering system. There was no education campaign targeting vitamin D during this time in our hospital and we speculate that unnecessary vitamin D testing may already be on the decline as it has featured in the ‘Choosing Wisely’ campaign for the last few years.34 35 CRP showed no step change but did show a levelling out in its slope, again indicating that ordering for this test is plateauing. No changes were observed for troponin and HbA1c. These four tests are generally ordered for specific clinical contexts, which means they are unlikely to be habitually reordered within admissions like FBC and UEC.
The findings for TSH were unusual. The during-implementation period showed a sharp reduction in TSH ordering, creating a large step change (drop) from the preimplementation period to the postimplementation period, but the slope continued to increase at virtually an identical rate in both the pre and postperiods. No other test exhibited this change to ordering patterns during the implementation of the e-ordering system. No anomalies or errors in the data or ordering system have been identified to account for this. No interventions aimed at reducing test over-ordering were carried out at JHH during this audit. We speculate that this finding is due to individual clinician behaviour but investigating this further is unfortunately not within the scope of the study.
Strengths and limitations
This study is the first of its kind and was built on a strong theoretical and methodological basis. Potential confounders were avoided where possible, including the effect of COVID-19 on pathology ordering, by using data prior to the pandemic. Seasonal variation is accounted for by taking 12 months of data (one calendar year) per period: preimplementation-, during-implementation and postimplementation. A model adjusted for CCI found a non-significant association with the outcome, indicating no confounding. Conversely, mean-centred bed days was significantly associated with test ordering and, therefore, was accounted for in the adjusted model. There were no pathology ordering interventions implemented at JHH during the study period, which could have confounded results. Additionally, the data collection site was a regional hospital, as opposed to the predominantly metropolitan hospitals studied previously in Australia20–22 and this study, therefore, offers novel data. The methodology and power of the study were strengthened by the nature of the audit as an interrupted time series with long periods of data collection (3 years of data in total). Power was also increased by the equal distribution of data points (in number and seasonality) before, during and after the intervention.36
This study was limited by the number of sites and range of tests that were included. A rural site was initially slated to be involved but was following a year behind the JHH e-ordering rollout. This meant that the COVID-19 pandemic would have confounded the data, leading to its exclusion. Additionally, this study did not compare differences by admitting ward and, therefore, could not account for any potential variation between medical and surgical pathology-ordering patterns. More granular-level data, rather than hospital level trends, could be beneficial in future studies. Importantly, our previous study found no difference in ordering pattern by admitting ward at the same study site.11 There was no explanation found for the unexpected ordering pattern for TSH which limits the interpretation of this result.
Conclusion and future research implications
This study indicates that the introduction of an e-ordering system did not, in and of itself, affect the volume of pathology ordering. The ease with which repeat orders could be placed with the e-ordering system did not lead to increased volumes, but neither did the increased visibility afforded by the system mitigate the pre-existing upward-ordering trend. This means that efforts to tackle low-value ordering will need to intentionally target and intervene on this behaviour. E-ordering systems do not achieve this intrinsically. Further work is needed to test the effectiveness of different educational prompts and lockouts on test-ordering behaviour and, importantly, patient outcomes. Pathology test ordering should also be covered in undergraduate medical training curriculum, specialist training programmes and other educational courses to bridge the gap in medical knowledge and awareness of test limitations.
The results of this study confirm the general secular trend in increasing ordering of pathology tests, with a trend line continuing unchanged throughout the study. By extrapolating preimplementation period data and showing no jump-up in ordering during the implementation period, the data demonstrate that the increasing number of tests ordered were due to background rates previously described.3–5 Several international studies have focused on interventions designed to prevent overordering (eg, displaying the cost of the test to the ordering physician, utilising provider education or applying restrictive changes to the e-ordering).14 25 38 Further research into the long-term effects of these interventions on patterns of pathology ordering are needed as these study periods ended 6 to 12 months after the intervention was implemented. Another interesting future research direction is to investigate whether the increased pathology tests provide value by shortening the length of hospital stay and, therefore, lead to overall financial savings.
Patient outcomes should be a primary focus for future studies, given the ongoing trend of increased pathology test ordering. Blood tests inevitably involve pain and discomfort for patients. Reducing unnecessary tests will both address costly low-value care as well as minimise iatrogenic harm. Additionally, the greater the number of tests ordered, particularly tests with poor pretest probability, the greater the chance of finding spurious results.6 This can lead to further testing and interventions that may be unnecessary and potentially harmful to patients.6 14 Future studies need to investigate what effect this trend of increased pathology testing has on patient outcomes more deeply, especially with respect to new e-ordering systems. Is this culture of increased test ordering justified by improved outcomes for patients or does it merely represent the decreased tolerance for uncertainty that pervades modern medical practice?13
Data availability statement
Data may be obtained from a third party and are not publicly available. De-identifed data were applied for and supplied by New South Wales Health Pathology, using the Health Information Exchange.
Patient consent for publication
This research was approved as a clinical audit and quality improvement project by the Hunter New England Local Health District’s Director of Clinical Governance and received a waiver of approval from the Hunter New England Human Research Ethics Committee.
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
MdC, BM, JU and JH-D are joint first authors.
Handling editor Patrick J Twomey.
Contributors JA, TdM, KP and AH contributed to the conceptualisation and design of this study. CS contributed to data access and integrity. ET and KP conducted the statistical analysis. MdC, BM, JU and JH-D were involved in the study planning and contributed equally as co-first authors to drafting the manuscript. All authors edited and revised the content and have approved the final version for publication. JA is guarantor of the overall study.
Funding This study was funded by the HCF Foundation and the University of Newcastle.
Disclaimer These sources of funding did not contribute to or influence the research project.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.