Evidence that pathology order entry (POE) systems improve care delivery or patient outcomes is limited. Yet Australia, like the United Kingdom, is implementing POE systems across their public hospital systems. The NHS Modernisation Agency announced, in September 2004, 10 high impact changes predicted to lead to significant improvements in health care delivery.¹ Number 2 is to improve patient flows by improving access to diagnostic tests, often a major bottleneck in patient care “…leading to long waits, communication problems and a lack of certainty and choice for patients.” Information technology (IT) will underpin such health system redesigns.² The NHS is making unparalleled IT investments, including £6.2 billion for health IT.³ The National Programme for IT forecasts that in 2006, test order and results reporting systems will be in place across the NHS.⁴ What contribution can these systems make to achieving efficiencies and improvements in health care delivery?

Many argue that a major benefit of computerised pathology order entry systems is faster delivery of test results to clinicians because of more efficient processes, with reduced time spent filling in forms, chasing up illegible orders, and following up results.⁴ In addition, significant improvements in test ordering is anticipated by reducing duplicated and unnecessary tests.^4–6 These changes are expected to lead to benefits such as earlier diagnosis and earlier start of appropriate treatment.⁴

However, few studies have measured the impact of POE systems on the speed of availability of results.⁷ Only three studies since 1990 have assessed the impact of such systems on turnaround time, and none has included general hospital wards. These studies found improved turnaround in American⁸ and Canadian⁹ intensive care units and two surgical wards in a Norwegian hospital¹⁰ following system implementation.

There is evidence that decision support systems in specific clinical areas influence clinicians’ management decisions, though few have demonstrated improvements in patient outcomes.¹¹ Computer alerts, for example, have been shown to increase the percentage of corollary orders made (that is, orders for laboratory tests to check for adverse reactions as a result of a drug correctly ordered, such as a check of electrolytes when ordering potassium).¹² An American hospital study found computer alerts resulted in faster adjustment of drug treatment by highlighting patients receiving nephrotoxic or renally excreted drugs who were experiencing rising creatinine levels.¹³ While the absolute risk of renal damage in these patients was low, the alerts significantly reduced this by 55%.

Reviews of factors influencing test ordering behaviours^6,14 have consistently concluded that the most successful interventions to improve appropriate testing are multifaceted. Computerised systems provide this capability. Order screens can be structured to improve information collection. Decision support on the appropriateness and cost of tests can be provided when ordering. Test order sets can be changed easily and the impact on ordering behaviour reviewed with the availability of real time data.

In order to build upon the limited evidence regarding the impact of POE systems we undertook a controlled before and after study at a teaching hospital. We examined time taken to process tests and assessed the potential of the system to influence test ordering through the use of structured entry screens and the manipulation of test order sets.

METHODS

Setting

The study was undertaken at a 650 bed Sydney teaching hospital with 60 000 inpatient admissions annually. All laboratories operate on a single PathNet Millennium system which encompasses a database of all tests ordered. In November 2003 the hospital began implementation of the Cerner Millennium PowerChart (version 7.8) POE system which allows clinicians to electronically order, verify, and review pathology orders. The system was linked to an existing results reporting function and draws patient information from hospital databases.

Procedures

Data relating to discrete pathology test assays ordered by clinicians on 11 wards during September to October 2003 formed the “before” system database. Data generated between July to August 2004 formed the “after” system database. All data were abstracted from the PathNet system. One ward did not receive the order entry system within this timeframe and served as a control. We included tests from all major categories (for example, full blood count, arterial blood gas, liver function tests). These account for 85% of all tests ordered. Before system implementation all orders were written on forms and delivered to the laboratory where they were entered into the PathNet system. The new POE system eliminated this procedure.

The study was approved by the hospital’s human ethics committee.

Outcome measures

Laboratory turnaround time was defined as the time from receipt of a specimen in a laboratory to availability of a test result. The impact of system use on test ordering was assessed in two ways. First, by examining, before and after implementation, the proportion of orders for plasma levels of gentamicin and vancomycin for which physicians specified blood specimens as peak, trough, or random. The order entry system required selection of the specimen type (fig 1). The paper-based order form resulted in many of these specimens being labelled as random by the laboratories because insufficient information was provided on the form. This reduced the value of test results for patient care. Second, we measured the impact of removing three discrete test assays (albumin, aminotransferase (AST), and total protein) from the liver function test order set on test volumes and on the average number of these tests per patient. This change was initiated when the POE system was implemented, because of concerns that these tests were unnecessary in many instances.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

 Structured order entry screen which required the specification of gentamicin samples.

Analysis

Comparisons of laboratory turnaround times and numbers of tests and blood specimens before and after system implementation were made using Student’s t test. Turnaround data were also stratified by prioritised (for example, generated from intensive care) and non-prioritised tests and time when processed; within (8 am–5 pm) or outside business hours (5.01 pm–7.59 am). We compared the proportions of patients having particular tests before and after implementation by χ² analyses.

RESULTS

Changes in turnaround times

Average laboratory turnaround time decreased significantly by 15.5 minutes per test assay for tests generated by the intervention wards following system implementation (table 1). No significant change in turnaround time occurred for the control ward (table 1). Reductions in turnaround were significant for prioritised and non-prioritised tests and for tests carried out within and outside business hours. The smallest reduction occurred for prioritised tests (4.5 minutes).

Changes in pathology test orders

In the preimplementation period 97 851 test assays were done for 1058 patients (mean = 92.5 per patient). Following system implementation the average number of tests (mean = 103.2 per patient, t = −1.21, df = 2015, p = 0.228), and blood specimens (10.8 v 11.7 per patient, t = −0.987, df = 2158, p = 0.324), did not change significantly.

There was a significant improvement in the proportions of gentamicin and vancomycin specimens identified as peak or trough by clinicians following system implementation (table 2). The removal of three discrete test assays from the liver function test order set resulted in a decrease in both the proportion of patients having one or more albumin assays and the average number of albumin assays per patient (table 2). For both AST and protein test assays the proportion of patients having one of more of these tests also decreased significantly, while the average number of tests in patients undergoing such tests increased significantly.

DISCUSSION

The POE system resulted in a significant 21% reduction in laboratory turnaround time. These results are likely to be generalisable to other large hospitals generating a wide range of test assays, unlike the few previous studies that have looked only at the impact in smaller specialised units introducing POE.^8,9,10 Turnaround comprises a fixed component, which is assay dependent (that is, the time required to analyse a specimen), and a variable component (the time taken to receipt the specimen and order, and post the result). The POE system significantly influenced the latter by eliminating transcription of paper orders into laboratory computers, a process sometimes slowed by illegible orders. Thus non-prioritised tests benefited most from system implementation. Laboratory turnaround time is a useful indicator of whether POE systems are fulfilling promises of improved efficiency and is more accurate than staff’s perceptions of the impact of the system.¹⁵

The extent to which improvements in laboratory turnaround enhance patient outcomes is still unclear.¹¹ A critical issue is clinicians’ capacity to respond to, and make clinical use of, faster results. The limited data available to date are not encouraging. A UK study investigating the impact of ward computers allowing access to laboratory results found that 45% of urgent requests for biochemistry tests from accident and emergency, and 29% from inpatient wards, were never accessed. Of results never read, 3% were assessed as necessitating an immediate change in patient management.¹⁶ Clinicians report dissatisfaction with current tracking and follow up of test results,¹⁷ so unless clinicians’ behaviour changes to utilise faster results, we risk overoptimising a single system. Additional system features such as email inboxes which post important results to clinicians directly, or computer alerts to highlight urgent results, may help support better test management.¹³

In relation to testing patterns we found that system implementation was not associated with an increase in test volume, negating concerns that these systems make it easier to order tests and thereby cause an increase in inappropriate testing. We found no reduction in the average number of specimens taken per patient, which might be expected if the system was effective in reducing duplicate orders. It is possible that this may be a longer term outcome or that at baseline there was a low rate of duplicate orders.

We examined two system features to determine their impact on ordering patterns. Structured order entry screens improved the quality of information provided to laboratories. In our example this improved information would be expected to enhance patient care. Removing discrete test assays from the order set resulted in a significant decrease in test numbers, yet for patients for whom an AST or protein test was undertaken the average number of these tests increased. The removal of these assays from the order set reduced the total pool of patients having AST and protein assays and left only patients for whom clinicians had specifically requested this test assay for a specific clinical need. Similar and long term changes to test volumes have been demonstrated in an American study of changes to computerised test order sets.¹⁸

Interventions involving changes to order sets within paper based order systems are effective.^19–21 An advantage of computerised systems is the ease of reviewing the impact of these strategies through the availability of real time data. These data also provide important opportunities to develop feedback loops to clinicians regarding testing patterns and to promote discussion of variations in practice. This approach has been shown to have a positive influence on clinical practice²²—particularly where baseline adherence to recommended practice is low—yet this is generally an unexploited benefit of computerised systems, rarely discussed in the published reports.

The use of POE systems requires clinicians to change the way they work. Orders take longer to enter, yet this may decrease over time,²³ and efficiencies have been observed in other clinical tasks.²⁴ This, along with the availability of computer terminals, may subsequently influence the time at which clinicians prepare an order. For instance they may batch the activity until a terminal is free and they feel they have sufficient time available to prepare orders, rather than write orders as they see the patients. These work practice changes will have an impact upon the total test turnaround time, and thus the efficiencies gained through improved laboratory turnaround may decrease or even be lost. Following system implementation clinicians spend more time alone and less time with other clinicians.²⁵ Opportunities to gain input from colleagues about patient management may be lost and substituted with time at a computer. Thus the implementation of POE may change the very nature of the way in which clinicians work.

While often viewed as large IT implementation projects, these systems have major implications for the ways in which clinicians and pathology staff work and patient care is provided. Our results demonstrate the value in quantifying the benefits of the technical features of order entry systems for efficiency and ordering behaviour. We have shown that POE systems are an important element in achieving faster test results. However, improving patient flows and care will rely upon system and behaviour change. Future POE evaluations should assess the ways in which systems integrate, or fail to integrate, with clinical work practices, including the impact of system use on test order preparation, processing, and the time taken to respond to results. Failure to investigate and understand these interactions may risk a proliferation of “workarounds”, and a health system with faster test results but no supporting work practice changes to reap the benefits.