Aims: To evaluate the characteristics of manual haemoglobin methods in use in Malawi and provide evidence for the Ministry of Health in Malawi to enable them to choose a suitable method for district hospitals.
Methods: Criteria on accuracy, clinical usefulness, user friendliness, speed, training time, and economic costs were determined by local health professionals and used to compare six different manual haemoglobin methods. These were introduced sequentially into use in a district hospital in Malawi alongside the reference method.
Results: HemoCue was the optimal method based on most of the outcome measures but was also the most expensive (US$0.75/test). DHT meter and Jenway colorimeter were the second choice because they were cheaper (US$0.20–0.35/test), but they were not as accurate or user friendly as HemoCue.
Conclusions: The process for choosing appropriate laboratory methods is complex and very little guidance is available for health managers in poorer countries. This paper describes the development and testing of a practical model for gathering evidence about test efficiency that could be adapted for use in other resource poor settings.
- HCN, haemiglobincyanide
- haemoglobin method
- laboratory economics
Statistics from Altmetric.com
As part of the Malawi government’s Poverty Reduction Strategy, the Ministry of Health and Population has developed an Essential Health Package.1 This aims to provide better access to quality services for common conditions that disproportionately affect the poor. In Malawi, all health care is free so patients do not pay a fee for the service. To complement the Essential Health Package, the Ministry of Health and Population is developing a model package of essential laboratory services to improve the cost effectiveness of district hospital laboratories. The model is being developed and piloted in selected typical district hospitals in Malawi before being adapted for national implementation. Haemoglobin measurement is a key component of this service, so it is necessary to identify a method that is practical, accurate, and economically viable for use at district level.
“This is the first time that such a detailed effectiveness study involving all the processes required to determine the suitability of a test has been undertaken in a district hospital in a developing country”
Haemoglobin measurement is the most commonly performed laboratory test worldwide and is an essential component of any health system. In sub-Saharan Africa, haemoglobin estimations are used in district hospitals for individual patient management, to guide transfusion practice, and in the management of antiretroviral therapy. Surveys of haemoglobin concentrations are also used as tools to provide public health data, such as nutritional status, and to monitor malaria interventions. Despite the wide range of manual methods available for measuring haemoglobin in developing countries, no single technique has emerged as the most appropriate for this setting.
To support their Essential Health Package,1 the Ministry of Health and Population in Malawi needed to identify a method for measuring haemoglobin that was simple, accurate, fast, and cheap and required minimal training and supervision that could be used in district hospital laboratories. Therefore, our study assessed the effectiveness, including technical, economic, and managerial aspects, of different manual techniques for measuring haemoglobin in routine practice in a typical district hospital in Malawi against criteria predetermined by local heath professionals. To our knowledge, this is the first time that such a detailed effectiveness study involving all the processes required to determine the suitability of a test has been undertaken in a district hospital in a developing country. The project received ethical approval from the Liverpool School of Tropical Medicine, UK and the Ministry of Health and Population in Malawi.
MATERIALS AND METHODS
Our study was carried out in Ntcheu District Hospital in the central region of Malawi. This district has a population of 500 000 and is served by a single hospital with 240 inpatient beds. The laboratory in the Ntcheu District Hospital performs approximately 6000 haemoglobin tests each year. It normally has a complement of two or three staff (laboratory assistants and qualified technicians), but two extra technicians were seconded to assist with our study.
The six haemoglobin methods to be evaluated were chosen because they were already in use in Malawi or at district level in other countries, and they could be supplied quickly for widespread use in Malawi.
The Lovibond comparator (ECHO International Health Services Limited, Coulsden, Surrey, UK)
This is the predominant method used in Malawian district hospitals and assessments were carried out using the equipment already in routine use. It is based on the oxyhaemoglobin method and involves matching the colour of the diluted blood sample with colours on a disc equivalent to known haemoglobin values in 20 g/litre increments.2
WPA colorimeter (WPA CO700D from WPA, Pocklington, York, UK) and Jenway colorimeter (Jenway 6051; Jenway, Dunmow, UK)
These methods are based on the conversion of haemoglobin to haemiglobincyanide. The Jenway colorimeter has a direct readout.
DHT meter (HMB.010; Developing Health Technology, Ipswich, UK)
This is based on the oxyhaemoglobin method and has a direct readout of the haemoglobin estimation.
HemoCue (HemoCue AB-haemoglobin photometer; Angelholm, Sweden)
Undiluted blood is placed directly into a microcuvette where the haemoglobin is converted to haemiglobinazide. The microcuvette is inserted into the photometer and the machine provides a direct readout of the haemoglobin result.
Haemoglobin colour scale (Copack GmbH; Oststeinbek, Germany)
A drop of undiluted blood is placed on specific chromatography paper and matched against a range of colours representing different haemoglobin values in 20 g/litre increments.
All methods were performed in accordance with the manufacturers’ instructions. The methods were evaluated sequentially on a total of 600 consecutive, unselected EDTA blood samples (that is, 100 samples for each method) sent to the laboratory for routine haemoglobin estimations during a two month period during the malaria season. To assess the accuracy of each method in routine use, the haemoglobin value of all samples was also measured by the haemiglobincyanide (HCN) method3 simultaneously with the test method, but by different technicians who were unaware of each other’s results.
Local laboratory technicians and clinicians devised the criteria by which each method was assessed. The criteria were designed to identify the method that was most appropriate for use in district hospitals in Malawi. The criteria comprised: (1) accuracy and clinical usefulness, (2) user friendliness, (3) time taken to learn and perform tests, and (4) economic costs.
Accuracy and clinical usefulness
The level of agreement between haemoglobin values generated by the individual methods compared with the reference HCN method was determined according to the statistical method of Bland and Altman.4 Statistical analyses were performed using SPSS version 10. The percentage of tests for each method that varied from the reference value by more than ±20 g/litre were regarded as wasted tests. This cutoff was chosen because local clinicians indicated that a result that deviated more than 20 g/litre from the true value would probably result in mismanagement of the patient and was therefore of no clinical use.
Local technicians themselves identified the five key attributes of an ideal method for haemoglobin measurement in a district hospital in Malawi as: technically undemanding, minimal training and supervision requirements, direct reading of results, simple quality control checks, and not reliant on electricity supply. The technicians independently rated each of these criteria on a scale of 1 (very poor) to 5 (excellent) for each test method and this was used to devise a score of user friendliness for each method (maximum score for each method, 25).
Time taken to learn and perform tests
As each new method was introduced, the time taken to learn the method so that laboratory technicians and attendants could perform it independently was recorded. The time taken to perform each test during routine use was documented, and the time spent on internal quality control and calibration processes for each batch of tests was included in the calculation and divided between the number of tests for each batch. The training costs for each method were calculated from the cost of practice tests and the salary of the trainee for the total training time. The Lovibond comparator was excluded from this analysis, because it was the method already in routine use in the laboratory, so it was familiar to the technicians.
To estimate the costs for each method, the actual value of all resources used to perform the 100 haemoglobin estimations (including control and calibration procedures) was calculated from: the cost of apportioned annualised capital equipment, supplies, and technician’s time taken to perform each test. An appropriate portion of the laboratory’s total electricity costs was included for all methods that used power. Building maintenance costs were assumed to be the same, irrespective of the method, and were therefore excluded from the cost analysis. These estimations were used to project the total annual costs of the haemoglobin workload for each method. Prices were obtained from the Malawi central medical store catalogue or directly from the suppliers, and included the standard 12% government discount rate. Costs were expressed in US$.
Accuracy and clinical usefulness
HemoCue was the most accurate method because it had mean and range values closest to those of the HCN reference method (table 1); 95% of HemoCue results differ by only 6% below and 16% above the reference results. None of the tests showed systematic bias and the degree of imprecision is reflected in the limits of agreement. The proportion of tests that were too inaccurate to be clinically useful varied from 1% for HemoCue to 59% for the Lovibond comparator (table 2).
HemoCue was rated as the most user friendly method by the technicians, achieving the maximum possible score of 25. The least user friendly methods were the reference HCN method and the WBA colorimeter because neither of these had a direct readout of the haemoglobin result (table 3).
Time taken to learn and perform tests
Competency was achieved most quickly for the HemoCue method by both grades of laboratory staff (table 4).
In routine use, the average time taken to perform a single test varied between three minutes (haemoglobin colour scale), five minutes (DHT meter, HemoCue, and Lovibond), and 15 minutes (colorimeters and HCN method). The training costs to achieve competency varied from US$18 to US$246, depending on the method. They were lowest for the DHT meter, the haemoglobin colour scale, and HemoCue. They were five to 10 times higher for the WBA and Jenway colorimeter methods and the HCN reference method.
Economic costs (table 5)
There was a fivefold variation in cost/test, from US$0.12 for the haemoglobin colour scale to US$0.75 for the HemoCue method.
The projected annual total costs of resources (staff, supplies, equipment, and overheads) that would be used for the 6161 haemoglobin estimations performed in Ntcheu laboratory in 2001 varied from 2001 US$752.4 to 2001 US$4638.3 for the haemoglobin colour scale and HemoCue, respectively.
To our knowledge, this is the first study to provide a detailed analysis of all the resources, including training inputs and recurrent costs, actually used to establish and maintain a variety of haemoglobin methods in an African district hospital. The criteria for the most appropriate method for their circumstances were determined by local users. On the basis of simplicity, speed, accuracy, and short training times, HemoCue would be the most appropriate method of all those assessed for district hospitals in Malawi. As a result of our study, the Ministry of Health and Population in Malawi has decided to introduce HemoCue as the standard haemoglobin technique at all their 32 district hospitals and urban health centres. Its major drawback is the cost of the disposable cuvettes which comprise 86% of the overall annual cost of this method, resulting in HemoCue having the highest annual costs at US$0.75/test. For all other methods except the haemoglobin colour scale, supplies make up less than 50% of the annual costs. Despite its advantages, the high recurrent costs will probably be a major disincentive for governments of poor countries to implement HemoCue at subdistrict levels where the burden of anaemia, and hence workload, is highest.5 It may be acceptable to compromise slightly on accuracy for haemoglobin methods that are used to screen large numbers at community level, as long as the method is fast and cheap,6 and a more accurate method is available at the referral hospital. Therefore, it has been suggested that the haemoglobin colour scale might be considered for use in antenatal and paediatric screening programmes. None of the other methods assessed in our study was ideal for district hospital use, where an accurate method is essential, because they involve a dilution step. This is a major source of inaccuracy in poorer countries where automatic pipettes are unavailable.
“One of the major roles that international organisations could play would be to provide detailed, independent evaluations of new laboratory methods”
The results of our study confirm findings from elsewhere that show that the Lovibond method currently in use in Malawi has an unacceptably low accuracy.7 It also indicates that methods that do not give a direct readout and which require the construction of calibration curves are technically too demanding for use in rural laboratories in poorer countries where the workload is high, many laboratory staff may have no technical qualifications, and there are no supervisory systems. The inputs needed to support the general laboratory system, such as training programmes, supervisory visits, and participation in external quality assurance systems, have not been included in our analysis. However, the simplest methods with the shortest training times will probably be those that require the least external supervision and monitoring.
Our study demonstrates the complexity of the decision making processes that are involved in an apparently simple task of deciding on an appropriate haemoglobin method for district hospitals in Malawi. In consultation with local users, we have developed and tested a model that can be used to assist in this process, and which could be adapted to suit other types of health technology and other settings. The appropriateness of tests will vary according to local clinical need and resources available, so the results of our study cannot be extrapolated directly to other studies. However, the evidence gathering, analysis, and decision making processes that we have described and that were developed in close collaboration with local users and suppliers of laboratory services, provide a model that could be adapted to other tests and settings.
Health managers in poorer countries do not have technical training in laboratory issues and have great difficulty in obtaining the evidence that will enable them to make rational decisions about purchasing technological services. They need to take account not only of costs, but also simplicity, accuracy, speed, available manpower, and technical skills of their laboratory workforce and the health needs of the population. Although the World Health Organisation does provide guidance on the selection of laboratory tests,8,9 details of all the components needed to make an evidence based judgement, such as we have provided in our study, are lacking. One of the major roles that international organisations could play would be to provide detailed, independent evaluations of new laboratory methods, which include information about start up and recurrent costs, training duration and costs, complexity of the method, and the inputs needed to maintain calibrations and quality. This type of independent and comprehensive information would be invaluable to health purchasers in developing countries, who face conflicting pressures over equipment purchase from agencies such as bilateral donors, charities, and the commercial sector.
Take home messages
We assessed six haemoglobin tests currently in use in Malawi, taking into account accuracy and clinical usefulness, user friendliness, time taken to learn and perform tests, and economic costs
On the basis of simplicity, speed, accuracy, and short training times, HemoCue was the most appropriate method and will be introduced as the standard haemoglobin technique in all Malawian district hospitals and urban health centres
However, the high recurrent costs of this test mean that the haemoglobin colour scale might be considered for use at the community level, with the more accurate HemoCue method being available at the referral hospital
We found that the process for choosing appropriate laboratory methods is complex and very little guidance is available for health managers in poorer countries
We have developed and tested a practical model for gathering evidence about test efficiency that could be adapted for use in other resource poor settings
We acknowledge the support received from the Ministry of Health and Population in Malawi, in particular Mr A Khuwi. The project was funded by the Department for International Development, UK. The Department for International Development accepts no responsibility for the information or views expressed in this paper.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.