ARUP has invented an automated camera system that uses sophisticated software and optical character recognition (OCR) technology to identify specimens that are potentially mislabeled for the patient name. A prototype system was installed on our automated track in October 2012. A validation study of one million specimens was completed on May 31, 2013. The prototype system continues to operate on our automation system and is clearly reducing the number of corrected reports due to mislabeled specimens for that portion of ARUP’s workload that is analyzed by the OCR system. This is a major invention that is clearly having a positive impact on patient safety.
This invention and its validation have just been published in the prestigious journal, Clinical Chemistry1. The article was accompanied by an editorial2 written by Dr. Robin Felder of the University of Virginia, one of the world’s leading experts on clinical laboratory automation.
The incidence of patient-identification errors, including mislabeled and misidentified specimens, is much too high in clinical laboratories. The best data on US errors is derived from three separate CAP Q-Probe studies, in which the reported rates of mislabeled specimens were 0.39/1000 in 120 institutions (2006)3, 0.92/1000 in 147 clinical labs (2008)4, and 1.12% of blood bank specimens in 122 clinical labs (2010)5. For professionals engaged in providing life-sustaining health services, these estimates are clearly a serious problem.
At ARUP, our measured incidence of mislabeled specimens from our pre-analytical accessioning process is ~1/8000. While this incidence is lower than the ~1/2500 to 1/100 reported by the CAP, nonetheless our goal, as it should be for all laboratories, has been zero misidentified specimens. For years, we have relied on education, training, vigilance, reminders, and other techniques to prevent errors, but despite these efforts, some errors still escape our pre-analytic QA processes, sometimes resulting in incorrect reports being issued to clients. Thus, the conception and invention of an automated camera system arose as a potential solution that could truly achieve an error rate of zero and have a major impact on patient safety.
The figure below is a photograph of the automated camera system, which uses a six-axis robot to carefully lift specimen tubes out of the transport carriers on our automated track system to a position centered between four equidistantly spaced, five-megapixel cameras. Using strobe illumination from four bar lights, photographs of the exterior of the tube are simultaneously captured by the four cameras. The robot grippers hold each tube by its cap so that none of the tube exterior is blocked by the grippers. The system’s software then stitches the four photographs together to make a single photograph, which is essentially a two-dimensional view of the entire tube exterior.
The software reads the barcode on the ARUP label to determine the unique tracking ID of the tube and then queries our LIS to obtain the content of the patient name record. This is reported to the system as a sequential character string without spaces or punctuation. It can be both alphabetic characters and numeric and the alpha characters can be either upper case or lower case. An OCR engine then searches the entire label image that is outside the perimeter of the ARUP label to see if a matching character string can be found.
If an exact match is found, the image is classified by the system as a “pass.” Specimens with a single label (as on an aliquot tube) are automatically classified as a “pass” since there is not an original label for comparison. If an exact match is not found, the image is classified as a “fail.” There are many reasons, other than a mislabeled specimen, as to why label images can fail the OCR analysis. These include non-standard fonts not yet trained in the system, labels that are dirty or of poor quality with characters crammed too close together either vertically or horizontally, marks on the label that touch the patient name, colored labels or labels with colored stripes, labels on which the patient name is split as two lines of text, labels with the patient name turned 90 degrees to the length of the tube, handwritten labels, and truncated names. However, human inspection can immediately see that most of these OCR fails are not mislabeled and can detect those images that are mislabels, which is why our processes require an employee to inspect all fails.
ARUP plans to build new OCR systems to be installed on our new automation system coming in fall 2014. The process that will be implemented when the new systems are in full operation will ensure that every single fail image is inspected by an employee trained to identify mislabeled specimens.
View a video, narrated by Dr. Charles Hawker, of the automated OCR inspection system in operation:
Since the prototype OCR system was implemented on our automation system, more than 2.4 million images have been obtained. Approximately 70 percent of images have been classified as passes by the system. The vast majority of the 30 percent of images classified as fails are not mislabeled specimens. They simply did not pass the automated OCR analysis for the reasons noted above that mostly relate to label quality. Since every one of the 30 percent of images that fail requires human inspection, a significant goal for this project is to improve the pass rate in order that the necessary labor can be minimized. However, many of the possible engineering refinements that can improve the pass rate have already been implemented. The greatest opportunity for improving the pass rate depends on improving the quality of the labels our clients place on the specimens submitted to our laboratory.
The OCR system has detected more than 300 mislabeled specimens since it was placed in operation on our automation system in 2012. This is an error rate of approximately one mislabeled specimen per each 8,000 specimens analyzed by the system. Approximately 53 percent of these mislabeling errors were detected by our normal QA procedures prior to analysis. For the vast majority of the others, the OCR detection of the errors has prevented the reporting of incorrect results on patients. Further, we have also been able to demonstrate that prompt inspection of the images (or the tubes themselves) for all those that fail to pass OCR analysis, will prevent errors.
During the validation period of 1 million specimens (October 3, 2012, to May 31, 2013), when all images were inspected, and since that time, when only images that fail the OCR analysis are inspected, our approach has to been to review the images one to three days after they have been obtained. Logistically, it has not been possible to inspect images on an immediate basis. When new OCR systems are built and installed on our new automation, this process will change and prompt review of all images that fail OCR analysis will occur. Nevertheless, we have had the opportunity to test the OCR system on two different tracks out of the four on our automation system and have seen a reduction in corrected reports sent to clients even with the delay in review of images.
The first track on which the OCR system was installed serves our highest-volume testing section where testing is performed 24 hours per day, seven days per week. During the time when the system was used in that location, 636,064 images were obtained. There were 79 mislabeled specimens detected by the OCR system, or one mislabel for every 8,051 images. Of these 79 errors, 17 resulted in corrected reports issued to clients, a ratio of one corrected report per 37,416 images.
However, in February 2013, the system was moved to the second track to gain additional experience with it, especially since that track carried a higher percentage of non-standard tubes. This track location serves lower-volume laboratory sections in which much of the testing volume is performed only two to three times per week. Even though the review of the OCR images was delayed, it was often in time to prevent an incorrect test from being performed and reported. Out of a total of 1,719,977 images in this location, there have been 223 mislabeled specimens detected by the OCR system, a ratio of one per 7,713 images, only slightly higher than the previous rate. However, only 18 of these mislabeled specimens required a corrected report, a ratio of one corrected report per 95,554 images. This has clearly demonstrated that once we are logistically able to implement immediate review of all specimens that fail the OCR analysis, the incidence of corrected reports due to mislabeled specimens can be decreased to zero, which will meet the goal that led to the invention of this system. This is a huge contribution to patient safety.
There are many reasons why specimen images analyzed by the OCR system do not pass. Only one in approximately 2,000 images is actually from a mislabeled specimen. The vast majority of the OCR failures are caused by:
Interestingly enough, the issues noted above that prevent the OCR system from passing correctly labeled specimens are the very same issues that contribute to mislabeled specimens in all laboratories, potentially causing harm to patients. In all our laboratories, we expect our employees not to make mistakes, but often it is the very systems that we have designed that cause errors to occur. In short, we cause our employees to fail. This, in fact, was the main message of the now-famous book published by the Institute of Medicine in 1999, To Err Is Human: Building a Safer Health System. As a result of this report and published error rates on mislabeled and misidentified specimens, the Joint Commission has included patient and specimen ID standards in its National Patient Safety Goals (NPSG).
The most important action that all laboratories (including ARUP’s clients) can take to improve the quality of their specimen labels is to follow the new CLSI Standard, AUTO12-A, which is discussed in the next section. Some laboratorians may view this standard as applying only to reference laboratories, which often have to relabel specimens. However, many hospital laboratories serve as core laboratories and receive specimens from other hospitals or facilities, outpatient clinics, and outreach physician offices. Often, these different facilities have different systems from the system of the core laboratory and relabeling must occur. Even if relabeling isn’t required, labels frequently have different formats and fonts because of different printers, requiring the laboratory’s employees to search for the patient name or other identifiers in different locations on the label.
The Clinical and Laboratory Standards Institute (CLSI) developed the AUTO12-A Standard, Specimen Labels: Content and Location, Fonts, and Label Orientation6, specifically because of the unacceptable incidence of mislabeled and misidentified specimens that has been discussed above. The main consideration for the standard was to address the human readable content. The CLSI committee saw many examples of labels from different laboratories in which the patient name was on the first line, second line, third line, etc., on the label, sometimes left-justified, sometimes right-justified. Sometimes the patient name was above the barcode, sometimes below. In other words, across the US healthcare system, the human readable content was so variable it was no wonder errors occurred.
CLSI has indications from the CAP and the Joint Commission that this standard will be referenced in its inspection checklists by 2015 or 2016. ARUP Laboratories recently completed a redesign of its specimen labels in order to comply with the standard, and we strongly urge our clients to do the same.
The AUTO12-A Standard has the following key features:
The following figures, reproduced from the standard with permission from CLSI, show examples of labels with and without the optional vertical priority zone on the left end of the label:
ARUP has invented and validated an automated camera system that uses OCR technology to detect mislabeled specimens. An article detailing the validation of the system has just been published in the journal, Clinical Chemistry. When improved systems are installed on ARUP’s new automation, we expect to lower the error rate due to mislabeled specimens to zero for all specimens analyzed by the systems. The system has not been patented and may become available for other laboratories to install on their automation.
ARUP hopes that its clients will improve the quality of their own labels. This will result in fewer errors and misidentified specimens for our clients, and will improve the performance of the OCR system in passing correctly labeled specimens. One important step clients can take to improve label quality is to follow the CLSI Standard AUTO12-A. Improved patient safety is a goal shared by all laboratories and healthcare professionals.