Since 2007, we have gradually been building up infrastructure for digital pathology, starting with a whole slide scanner park to build up a digital archive to streamline doing multidisciplinary meetings, student teaching and research, culminating in a full digital diagnostic workflow where we are currently integrating artificial intelligence algorithms. In this paper, we highlight the different steps in this process towards digital diagnostics, which was at times a rocky road with definitely issues in implementation, but eventually an exciting new way to practice pathology in a more modern and efficient way where patient safety has clearly gone up.
- pathology department
- image processing
- computer systems
- medical informatics computing
- quality assurance
- health care
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
- pathology department
- image processing
- computer systems
- medical informatics computing
- quality assurance
- health care
The perfect storm of introduction of both video cameras that could be mounted on microscopes and personal computers in the early 1980s made it possible to project live microscopic images on computer screens. Through telephone lines, images could be viewed on distant locations, allowing, for example, remote consultations or frozen section diagnosis (‘telepathology’).1 Software programmes were developed to perform measurements of cell and tissue features on these live images (‘morphometry’). Soon after that, digitising boards (‘frame grabbers’) were developed that could transfer these video signals into digital images (initially 64×64, later from 128×128 to 256×256 pixels) that could be processed by image analysis software. This allowed taking digital snapshots, which became easier when digital cameras became available. Having these digital images available allowed development of image analysis software for, for example, mitoses2 or vessel recognition3 and DNA content measurement.4 Even complete systems came on the market for analysis of Pap smears based on artificial intelligence (AI).5 However, computer hardware at the time was too slow to allow digitising full sections and analysing them, limiting applications to snapshots.
The first scanner to fully digitise sections that came on the market was BLISS in 1994 (later acquired by Olympus) followed by several other companies. The breakthrough was probably the introduction of an affordable whole slide scanner by Aperio in 2005 that could digitise a full section 20× in several minutes, accompanied by a freely downloadable viewer. This marked the birth of ‘digital pathology’.
Apart from being a lab traditionally interested in digital technology in pathology, the introduction of the Aperio scanner collided with several problems we encountered in daily practice and an opportunity. As to the former, we discovered that about 8% of our archived slides were misplaced, and we wasted too much time searching for slides to reassess for multidisciplinary meetings. For instance, for the weekly oncology multidisciplinary meeting, a resident would spend about a full day collecting slides from the archive or the desks of our 30 pathologists and residents. Further, all multidisciplinary meetings where slides were to be shown had to take place in a conference room with a projection microscope, of which there were only two in the entire hospital. There was a clear need to be more flexible here using digital pathology images, as with radiology images.
The opportunity was an internal UMC Utrecht innovation call in 2006 for grants to do things ‘Smarter & Better’. We applied with a plan to acquire a whole slide scanner to start building up a complete digital slide archive to optimise preparing multidisciplinary meetings using digital slides, at the same time solving the problem of the incomplete slide archive. Additional advantages would be business development through remote digital consultations and remote digital diagnostics for labs requiring support because of understaffing. The grant proposal scored high and we were lucky enough to get the money to get started.6
Digital pathology implementation: first generation
Laboratory setting and slide volume
In 2007 we started with the implementation of the first iteration of our digital pathology system. Our department of pathology at the University Medical Centre Utrecht (The Netherlands) is a medium-sized academic pathology laboratory with about 20 000 surgical pathology cases in 2007, 7000 cytology cases, and 330 autopsy cases with approximately 70 000 blocks produced each year. That amounted to ~1 37 000 histological stains and 30 000 immunohistochemical (IHC) stains. That meant that as a laboratory, we produced about 3211 surgical pathology slides per week.
The main drivers for choosing scanners back in 2007 were scanning performance (speed and reliability) and openness of the system. That meant support for an application programming interface (API) and software development kit (SDK) by the vendor for accessing the system and the image format. Both the API and SDK were used locally to integrate with our pathology reporting system U-DPS (Stichting Palga, Utrecht, The Netherlands) and our laboratory information system (LIS) (Finalist, Groningen, The Netherlands).
Using the money from the Smarter & Better grant, matched by an additional grant from our division management, we decided to buy two Aperio ScanScope XT scanners, supplemented by a third one courtesy of Aperio to create the desired long-term scanning capacity. These scanners were equipped with an Olympus×20/0.75 Plan Apo objective, resulting in a resolution of 0.50 µm/pixel, whereas a magnification doubler could increase the resolution to 0.25 µm/pixel (a technician had to stop the scanner and push a lever to activate the doubler). The scanners used a line scanner as an image sensor and they used points spread through the tissue to construct a focus plane. Images were stored as a proprietary .svs file to form a pyramid multiresolution file format where the compression factor and method could be configured as scan parameter (LZW, JPEG or JPEG 2000). The .svs file format later formed the basis for the pathology DICOM (Digital Imaging and Communications in Medicine) format.7 The .svs file format remained very popular throughout labs for research purposes because of its relative openness to access the pixel data with external programmes and libraries. With the release of the OpenSlide library, however, that point became less relevant since all formats generated by vendors were more of less supported by the OpenSlide library.8 The scanners could be loaded with eight trays of which each could hold 15 slides. That made it possible to scan 120 slides per scanner in one batch.
Slides were scanned after the diagnostic process with the microscope was finished in the traditional way. Basically, that meant that scanning slides was just an additional step before archiving slides. Since slide archiving was taken care of by the morgue and autopsy assistants, they were the ones that were trained to perform slide scanning and they very much appreciated this challenging extension of their work. They developed an efficient scanning schedule where they would start a batch in the early morning and would return to the scanning in between other work to put new batches into the scanners, putting in a last batch before going home. Also in the weekends when they had to come in for morgue or autopsy service they would run batches. No quality control of the wholeslides (WSI) was done at the time, although scanning failures that were encountered by chance were manually corrected, usually by putting in more focus points.
The scanners provided some problems in the beginning especially with regard to glass fragments from the slides that tended to clog the scanners’ fine mechanics, but later ran pretty much like clockwork with little regular maintenance that included vacuum cleaning the scanners of glass fragments.
Some adjustments in the lab processes for producing slides turned out to be necessary. The scanners would sometimes get stuck because of lopsided coverslips, and sometimes slides could not be completely scanned because of too large sections or poorly mounted sections or ribbons that would reach beyond the coverslip. The lab technicians were therefore carefully instructed to not cut too large sections and to mount them centrally, and to make sure coverslips fitted the sections perfectly. Convincing the pathologists and residents to stop scribbling and putting marks on the slides (that would often result in focusing problems) failed miserably, but fortunately, the scanner operators did not mind cleansing the slides of greasy fingerprints and marks with xylene before scanning. To prevent double scanning, all scanned slides were marked with a dot on the label.
There was at the time no alarm system in place to signal scanner failures, meaning that occasionally the operators would encounter failed batches in the morning, necessitating a rescan after fixing the problem.
The first iteration of our storage system took already into consideration that it had to be fast and large enough to store our slides that were to be scanned every day for the coming years. From the start, it was our vision that all images would be kept indefinitely. We therefore had to set up a large scalable storage facility in cooperation with the central information technology (IT) department of our hospital, and we implemented a hierarchical storage management solution (Sun Microsystems, Santa Clara, California, USA), where initially all images were stored on fibre channel hard disk drives and also copied to tape in a buffered way. The total hard disc capacity for rapid access was initially 6 TB. This size was chosen to accommodate the production of 1 month at the time, since cases are usually discussed during multidisciplinary meetings within 1 month after diagnosis. The tape library consisted of two tape streamers accessed by a robot with an initial capacity of about 120 TB, consisting of tapes with a capacity of 750 GB each. The tape system worked as a second-tiered storage system, so when a user requested an image that was no longer online, it would copy it from the tape back to disk to make it available again, which could take a few minutes, allowing pulling out of old cases even during the multidisciplinary meetings if necessary. This system remained in place up to the point when we migrated to an all disk system. However, the first iteration of the all disk access system was again a two-tiered object-based storage system where access to the data went through a gateway that would retrieve from the second tier to the online storage. The reason was that our digital pathology system was incompatible at that moment with object-based storage systems and required traditional protocols to access the data, which were provided in the form of a gateway system which would handle all queries. For the tape storage system, we had built a software layer (which was possible for that system) that would notify the user if the data were still archived and the approximate time needed to retrieve the data. The disk system had no such possibility, which resulted in users very frequently getting timeouts while trying to retrieve archival cases. The user experience during that time plummeted, which resulted in pathologists not using the system as actively as before.
Migrating to the object-based storage system also changed the business model in terms of storage that we operated up to that point. We went from a system that we invested upfront to a pay-as-you-go model. The continued support for the iteration of that system at that point required a level of faith and trust that was incompatible with the business aspect of reality. We were storing massive amounts of data (at the point of migration, we had almost 350 TB of archive data) and a system that was underperforming due to the storage issues.
After a series of meetings and escalations to the central IT storage group, it was decided that hospital-wide, we would migrate to a new bulk storage system that would natively support use cases such as ours, storing large files for archive purposes without the use of middleware layers for access. The second iteration of the disk-based storage system that we migrated to had superb performance in comparison and made pathologists quite excited to again use the system. At that point, however, the rest of the system was getting quite old, so it was slowly falling behind in terms of business necessities.
Integration and applications
To automate the laboratory workflow process, the scanner system was integrated to our patient reporting system via the barcode on the slide. The slide barcode at that time contained a slideID that was used to identify the slide through our LIS. The scanners would scan the slide and query the LIS for information which would then be used to update the image database. That enabled assignment of slides to cases and add information about block, specimen and staining. Using the API of the image management system, we built a software layer on top that would link a report in UDPS to the digital slides. Clicking on a link in the pathology report would present the pathologists with a collection of thumbnails of all scanned images, macroscopy photos, as well as scanned order forms. Clicking on a thumbnail would prompt opening of that image in the Aperio viewer. On the bottom of the pop-up window, there was a link to add slides to a worklist for a specific multidisciplinary meeting.
That system worked quite well since the integration relied on simple API calls and existing functionality that existed on both the slide scanner system as well at the LIS side. There were, however, several issues that became apparent later on. The barcode on the labels at the time was a type code 128 1D barcode, which included a checksum to verify the contents, significantly lowering the chance of misreading barcodes. However, the information density was low, which did not permit addition of more information than needed. At the time, we included only a numeric ID which by itself was quite error prone. If another barcode were to be scanned with an ID that might happen to correspond to a slideD, then that slide would be added to the wrong pathology report number and have the wrong metadata added to it. That posed several issues that came to light much later when we realised that several consultation cases, where the label was situated in way that both our barcode and the requesting labs barcode where visible, where misassigned. However, the largest batches of slides that were misclassified were the IHC slides due to the double barcode they used. We use labels for our IHC slides that contain two types of barcodes, one used by the LIS system for identification and the other for our staining system to link the glass slide to a staining protocol. As soon as we had used the IHC system long enough, the number range used started to overlap with the slideIDs. That meant that almost 16 months of IHC slides were misassigned. This only became apparent when we were reviewing the scanned slide archive to migrate to another system. All of those slides had to be manually corrected before migrating, which cost us 3 person months, as well as a 3 months of delay to start the migration. We have since switched to 2D datamatrix barcodes and added more information to make the slideID truly unique.
Research, teaching and outside consultation
Soon after implementation of the scanners, many researchers started to find their way to our department to have their tissue slides scanned. They found it much easier to carry an external hard drive with their WSI that could be viewed on any computer after downloading the free Aperio viewer than to drag slides around that needed to be viewed in a (usually remote) room with a microscope. Moreover, WSI allowed researchers to easily capture snapshots from digital slides for, for example, publication and presentation purposes without the need of having an expensive photomicroscope. Regularly, this led to scanning capacity problems but also brought in some business, as we would charge slide scanning to people from outside the department.
The process of switching to practical sessions with WSI instead of slides with microscopes was started soon after we had the scanners installed. Within a year, we had all practical sessions digitised within the Digital SlideBox environment (Slidepath, later acquired by Leica). The students appreciated the digital practical sessions much better than the microscopic sessions.
Anticipating a rapid growth in digital consultations from outside labs, we purchased a licence to the mScope application (Aurora, Montreal, Canada), where, based on our wishes also, an expert panel module was built in. Getting serious numbers of digital consultations miserably failed, although the expert panel module was successfully used for a number of years by the Dutch Breast Panel and SKION, the Dutch Paediatric Haematology Panel. We came up with the innovative but untimely idea to buy a small mobile Aperio whole slide scanner to instal locally in labs requiring instant diagnostic support in case of understaffing, scanning locally and importing the images into mScope for remote digital diagnostics. This never worked and the scanner has pretty much been collecting dust for years.
The first generation of our digital pathology system, which began on November 2007, started showing signs of its age by the end of 2014. The scanners were running on Windows XP SP3 and were end of life by 2014. In order to upgrade them we would have to seriously invest in new scanners. Also the current home-made software system was not fast enough anymore, with the database of the image management system growing larger at the same time that more users were logging on to the system and the rate of scanned slides per day increasing every year. By that time we were producing almost 50% more slides compared with when we first invested in the system so capacity started being an issue. At the same time, the scanners needed several minutes per slide 20× and scanning at 40× was prohibitive in terms of scan time.
The needs of our department were changing, and the system at the time no longer satisfied the requirements of daily diagnostics. There was an increase in the number of multidisciplinary meetings per week. Further, slide scanning after the diagnostic process with the microscope had been finished was no longer perceived as satisfactory. Although preparing and doing multidisciplinary meetings with digital images was considered a great asset saving much time, not all slides were digitally available for all meetings since they were perhaps too recent and not yet scanned, and switching between digital and microscope during meetings was less than easy. Further, slides were untraceable if they were in the scanner.
Therefore, we realised that slides needed to be scanned before they were delivered to the pathologists to make sure that all slides would be digitally available to all residents and pathologists at all times, at the same time providing them with the option to do diagnostic work digitally. These factors were the main drivers for deciding it was time to take the next step and move to the second-generation digital pathology system, a primary digital diagnostics workflow.9 To prepare for this, we conducted a series of studies to validate digital diagnostics.10–17
Digital pathology implementation: second generation
In 2015, we decided that we needed to take the next step and not only renew our fleet of scanners but also change the way we do diagnostics. After having discussions with key industry people and scanner manufacturers, we realised that the available hardware on the market was mature enough to handle a fully digital workflow and that there were several manufacturers that could compete for an implementation contract. The ambition was born, a professional highly efficient workflow system for primary digital diagnostics without negative effects on throughput time, ready for implementation of image analysis/AI while maintaining our principle of a complete digital archive.9
The new system would comprise several high throughput scanners, a workflow system to manage it, an image management system as well as the IT hardware to host it. The amount of capital investment needed and the fact that our institution is publicly funded warranted a European tender process to select the vendor.
We opted not to follow the traditional tender process, that is, compiling pages of technical characteristics that the solution had to adhere to and ending up selecting the lowest twice daily, but instead followed a ‘best-value procurement’, where we would request a very limited set of requirements that were knock-off criteria, which were prerequisites to two times per day.
Among the requirements was that the system delivered would include scanners with enough capacity to scan the daily production and cover slides produced during peak hours: at least 800 slides/day and 120 slides/hour. In this way, the throughput of the system would not act as a bottleneck for the diagnostic workflow. Apart from throughput, we did not set any requirements concerning image and scan quality.
Eventually, a consortium composed of Sectra AB (Linkoping, Sweden) and Visiopharm (Hoersholm, Denmark) (as the reseller of Hamamatsu Photonics K.K. in The Netherlands) came out as the winner, proposing a system based on Hamamatsu (Hamamatsu City, Japan) scanners and Sectra PACS as the image management and workflow system.
We began immediately implementing the solution in one of the most ambitious projects of our department since we quickly realised that changing the workflow to a primary digital diagnostic workflow left no department unit unaffected, which meant that we had to set up workgroups that were composed of representatives of every unit within our department.18 This exercise in extreme change management helped involve all of our staff in the new project, allowing everybody to partake in the implementation and have them take ownership of the project. That might have been one of the most important aspects in successfully implementing this system.
The proposed solution offered by the vendors was three Hamamatsu XR scanners and one Hamamatsu RS scanner with a fluorescence unit. The Hamamatsu XR scanner is a high-throughput scanner which works as our main workhorse, whereas the RS was selected to scan 2×3 slides and fluorescence so focused on all specialty cases. It quickly became evident that even though the scanners could achieve the scan times advertised, in practice the scan times were lower due to several factors. First, average tissue size is larger than then popular advertised 15×15 mm tissue size. Tissue placement on the slide as well as the use of control cores for IHC stainings make the average slide size about 18.5×18.5 mm. Second, IHC stained slides when negative are very lightly stained, which in return are hard to be recognised by the scanner tissue detection algorithm. That required rescanning of the slides with extra focus points adding to the total scan time. Third, more H&E slides than anticipated needed to be rescanned due to focus issues.
Even though we had enough capacity to scan our daily production, we seem to have very few margins for error, so we had to make some changes in order to scan more efficiently. By having the scanner to query the LIMS using the unique slide identifier contained in the barcode, it retrieved the slide metadata. We then used the metadata to instruct the scanner to use a different profile per slide in order to scan more efficiently. The goal was to minimise the total number of rescans. We constructed around 10 different profiles which ensured that the slide would always be scanned without fail. That allowed us to create a margin of error and also manage to create some room to anticipate great peaks of slide production.
One of the requirements of the project was to use DICOM in order to store our scans. Unfortunately, when we started the project, there were very few vendors that even had a DICOM implementation let alone a full integration. We started a pilot programme to use DICOM for our scans but quickly scaled it back due to several integration issues. The DICOM integration was developed to convert the scanner’s proprietary format files to DICOM. That created a bottleneck in the scanning process, which was prohibitive for our production scanners, since we would have to transfer the files to an intermediate conversion server and then send them to our Picture and Archiving Communication System (PACS). At the end, we opted to run the DICOM conversion only on one scanner as a beta process in order to learn from it. Other issues that we encountered trying to use DICOM were
DICOM populates the headers with patient information and slide metadata, which it retrieves using the unique slide identifier. However, when the scanner fails to read the barcode, the conversion fails and the process of rectifying the failed data retrieval costs a significant amount of time per slide. We have not yet found a solution around this problem.
The DICOM conversion needed an extra middleware infrastructure to handle the conversion. Since it requires high read and write operations, we needed to invest in high bandwidth storage system to handle the Input/Output (I/O) load. That would have increased our IT support and budget.
Our PACS system supports DICOM but does not support DICOM operations for pathology images such as C-STORE and C-MOVE (as of writing this article, they have implemented some of the operations with some restrictions), which meant that after generating the DICOM files, we sent them to PACS via the import nodes reserved for the proprietary scan formats such as ndpi from Hamamatsu and not via a C-STORE operation. That means that there is a non-zero chance that a DICOM slide would be imported in PACS missing part of the data (usually the 40 x layer).
Our policy moving forward is when choosing a scanner, to opt for a native DICOM generation (scanner produces DICOM from the source and not via a conversion step).
Since the start of our digital pathology journey, we have opted to keep all of our scanned slides in order to create a digital archive to reduce reliance on our glass archive. That has resulted in having a 12 year digital archive of almost 2 PB of data. That has created strain in the digital storage system that we are using as well as to our budget. Thankfully, because of the demands that we have as an archive, it has pushed our central IT to invest in storage systems and expertise, which subsequently has reduced costs for all hospital users.
The way our storage system was set up was to ensure maximum redundancy of all events even, making sure that the complete archive was duplicated in two data centres, so in the event of a catastrophic failure in one of the data centres, the other location would be available. Keeping an archive mirrored in two locations doubles the cost of storage, so we weighted the costs against the benefits and at the end it boiled down to one question: if we were to lose our entire archive to a catastrophic event, how would that affect our daily diagnostic practice? Since our archive is separate than our short-term storage, we would still be able to use the system and we would just lose access to our digital archive. So, we decided to revert to a single location archive, reducing our costs to almost 50%. That has created a situation where we expect a reduction in price per TB every 3 years approximately, reducing the overall cost of our complete archive even though it keeps increasing in size every year.
The specifications for the storage needed to support a digital pathology operation are actually quite low end, which translates into using archive storage (high-density nodes with low throughput) for our daily operations without any noticeable drop in performance.
To be able to scan slides within 15 min after being coverslipped, quick slide drying was necessary to prevent slides from getting stuck in the trays due to mounting medium remnants. We therefore switched back from a xylene replacement to xylene which provided much quicker drying, and reinstalled an airflow slide drying table in the lab that had been in storage for some years. The scanner operators, who were now the histotechnicians, implemented a procedure for slide cleansing, including removing mounting medium remnants before putting slides in the scanner tray.
Initially, we had in mind to instal the scanners in the lab according to lean principles, but they turned out to be noisy and producing too much heat and ozone, necessitating putting them into a separate room with proper climate control. A large format display was installed, which receives the output of all scanners and combines them in one view to monitor all operations from one place (figure 1). A quality control procedure of scanned images is in place as described in detail before.18
We encountered image quality issues shortly after the system had gone live, related to the limited numerical aperture of the scanner lenses, leading to fairly thick optical slices blurring nuclear detail.18 Helicobacter pylori and intraepithelial lymphocytes were difficult to see; dysplasia could be overdiagnosed; and counting of mitoses was more difficult. We therefore created a workaround by making the gamma adjustable in the PACS system and switching from sections 4–3 μm thick.
After 6–12 months of double workflow to get used to the new digital workflow, we have been happy with the system for almost 4 years now. We probably sign out 90%–95% of our cases digitally but do occasionally revert back to slides for cases where image quality limits diagnostic confidence for, for example, paediatric pathology, mitoses and microorganisms recognition, and haematopathology. Cytology is still being done in the traditional way because of lack of scanning capacity and storage for 3D scanning (and perhaps a slight lack of confidence in digital diagnostics with current image quality). The residents also love the system and need to be trained in using the microscope before their rotations to regional non-digital labs. We have seen several important developments in the PACS, such as implementation of the first image analysis tool for Ki67 counting, a bidirectional link between our reporting system and the PACS, empty thumbnails for stains requested and lagging images, and magnification-sensitive tracking of our movements through the slides to preclude missing tissue parts. The pathology cockpits have been optimised. During the COVID-19 crisis, the digital workflow allowed seamless working from home as extensively described elsewhere.19 A national platform for exchange of WSI between the Dutch pathology labs and expert panels has been implemented and is up and running.20 In view of our growing practice, we are on the lookout for a fifth scanner. Since our current system is probably end-of-life in 2021, we are thinking about our third-generation digital workflow.
We expect to upgrade our current setup in the next year to a regional PACS where we can work together with several neighbouring labs. Our current scanner park is ageing, and we will be doing market research to evaluate the newest generation of scanners for image quality, throughput, interoperability and reliability, starting with purchase of a fifth scanner any time soon. By the end of 2020, a dedicated cytology whole slide scanner is expected to come onto the market, which we hope to evaluate and purchase to make the jump to digital cytology without seriously impacting our scanner capacity and storage. A very serious topic in the upcoming years will be implementation of AI algorithms in our digital diagnostic workflow. We have a number of algorithms available that have been developed within the framework of collaborations with the Radboud University in Nijmegen and the Technical University of Eindhoven, The Netherlands, that are suitable for further testing and validation in daily practice.21–24 We also hope to work with companies bringing AI algorithms to the market as implementation site. We expect to make pathology diagnostics faster, more objective and intellectually more satisfying while benefitting our patients with the best tissue diagnostics basis for treatment.
Take home messages
Switching to a complete digital workflow might require several iterations involvement from the entirety of the staff and lab. It is completely normal not to get it with the first try.
We still haven’t reached the point where we can work 100% digitally but we realised that it is not necessary anymore. The benefits are evident also on lower percentages. The focus now is in AI-assisted diagnosis.
A digital workflow will offer the opportunity to apply lean principles and make the lab and diagnostic work more efficient and of higher quality.
A complete digital workflow is the only viable option when a lab wants to transition to an AI-assisted workflow.
Handling editor Runjan Chetty.
Contributors All authors have contributed to this article.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Commissioned; internally peer reviewed.