Use of a Web-Based Calculator and a Structured Report Generator to Improve Efficiency, Accuracy, and Consistency of Radiology Reporting

Introduction

Medical calculators are common and are used to perform complex equations more efficiently and accurately [1–4]. These calculators are typically used by health care providers to determine values for specific formulas and require the user to input laboratory and/or demographic-based variables. Radiologists have not typically used formula-based calculators as part of their clinical practice. The purpose of this manuscript is to describe the use of a leg-length calculator and structured report generator and then determine its effect on the accuracy and time required to interpret and dictate a leg-length discrepancy study, a common radiographic study performed in pediatric radiology departments. We hypothesized that the calculator application would reduce the time required to interpret and dictate a study in addition to decreasing the number of errors in interpretation.

Methods

A novel, web-based application was created to help radiologists calculate leg-length discrepancies (Fig. 1). The application was designed so that once the radiologist types the measurement value for the location of the right and left femoral head, tibial spine, and talar dome, the following lengths are calculated: the right and left femur (defined as the distance between the femoral head and the tibial spine), the right and left tibia (defined as the distance between the tibial spine and talar dome), and the right and left lower extremity (defined as the femoral length plus tibial length). In addition to these lengths, the differences in lengths between the right and left side are also calculated and a standard, structured report is generated to be copied into the dictation window.

A time motion study was performed to evaluate if the use of the calculator decreased the time required to determine and dictate the leg lengths. Two board certified pediatric radiologists individually evaluated two sets of ten randomly selected radiographs performed for leg-length discrepancy using one of two methods. For both methods, the radiologists interpreted the images in the same manner; they placed a line annotation on the image at the right and left femoral head, tibial spine, and talar dome. The line extends to a radiopaque ruler included on the image. The radiologist then either types these numbers into the calculator application (automated methods) or calculates the leg length via pen and paper and a traditional calculator. Each radiologist evaluated ten studies using the traditional, manual method where all leg lengths were calculated using a pen, paper, and calculator (if needed) (Fig. 2). After making the calculations, the radiologist then dictated the study using a prepopulated structured report [5]. After completing the first set of radiographs, the radiologists then reviewed the second set of ten radiographs using the leg-length calculator. A head-to-head comparison using the same radiographs was not performed in order to minimize recall bias.

While one radiologist evaluated the radiographs, the other radiologist used a stopwatch to determine the following times for each interpretation: time required to calculate the leg lengths (defined as the time from when a study was opened to the time when all calculations were complete), time required to dictate and sign the report (defined as the time from when all calculations were complete to the time when the sign report button was selected in the voice dictation system), and the overall time required to complete and dictate a leg-length calculation (defined as the time from when a study was opened to the time when the sign report button was selected in the voice dictation system). In addition to recording times, the second radiologist also determined the accuracy of each calculation. Overall mean times and reporting accuracy were calculated and compared between the two groups using the Student’s t test. Differences were determined to be significant if the p value was less than 0.05. The yearly time savings to the department was estimated using the mean difference between the two methods and the overall number of leg-length studies performed during the 2011–2012 academic year (July 1, 2011 to June 30, 2012).

Results

Each radiologist evaluated a total of 20 leg-length radiographs: ten using the manual method and ten using the leg-length calculator. Using the manual method to calculate leg-length discrepancies, reviewer 1 took a mean time of 150.7 s to calculate the leg lengths and 31.1 s to dictate the report for a total mean time of 181.8 s. Reviewer 2 took a mean time of 113 s to calculate the leg lengths and 32.5 s to dictate the report for a total mean time of 145.5 s. Both radiologists were 90% accurate when calculating leg lengths manually. Reviewer 1 incorporated a dictation error into his final report while reviewer 2 made an error in calculating the leg lengths.

Using the leg-length calculator to calculate leg-length discrepancies, reviewer 1 took a mean time of 62.3 s to calculate the leg lengths and 10.2 s to dictate the report for a total mean time of 72.5 s. Reviewer 2 took a mean time of 57.1 s to calculate the leg lengths and 11.9 s to dictate the report for a total mean time of 69 s. Both radiologists were 100% accurate when calculating leg lengths with the leg-length calculator.

When comparing the two methods of calculating the leg lengths, the manual method was significantly slower than the calculator for all time points measured: the time to calculate the leg-length discrepancy (131.8 vs. 59.7 s; p < 0.001), the time to dictate the report (31.8 vs. 11 s; p < 0.001), and the total time (163.7 vs. 70.7 s; p < 0.001). The improvement in accuracy seen by using the leg-length calculator did not reach significance (p = 0.16).

Using the mean times from the two reviewers, the use of the leg-length calculator to determine the leg-length discrepancy was 92.9 s faster compared to the manual method. Between July 1, 2011, and June 30, 2012, there were 168 leg-length studies performed in our department. The estimated time savings to our department over that time from using the leg-length calculator was 4.3 h.

Discussion

While portable, electronic, formula-based calculators have been used in medicine for over 15 years, there have been few studies evaluating the time savings associated with their use [1, 2, 6]. Several studies have shown that calculators are more accurate than traditional calculations. Many of these studies have focused on medication dosing and have shown that calculators out-perform human-based calculations, even when double and triple checked [7–11].

In diagnostic radiology, calculators are relatively uncommon. Prior reports have described their use for such tasks as determining the radiation dose imparted by a study [1], determining liver iron content [12], or, more recently, to predict the likelihood that a solitary pulmonary nodule represents a malignancy [13]. The limited use of calculators in diagnostic radiology is likely due to several factors unique to the radiology practice. First, unlike many clinicians, radiologists are relatively stationary. Their main work task involves sitting in front of a computer interpreting images. Because radiologists do not need to move from room to room caring for patient, they are able to keep their references nearby and, thus, have little need for mobile/portable calculators.

The second reason why radiologists have not typically used calculators is that most of their interpretations are subjective. There is little need for a calculator when radiologist are describing findings or synthesizing their findings into an impression. As the field of radiology continues to perform quantitative imaging with increasing frequency [14, 15], we anticipate that calculators will become more mainstream.

Finally, one of the most important reasons why radiologists have not used calculators is because their main work output is a dictated report. If a radiologist was to use a more traditional calculator, he or she must first generate the result, then interpret the result, and then, finally add the result to his or her interpretation. By combining the calculator with a structured report generator, we have been able to consolidate these steps. We believe that the structured report generator is one of the most novel aspects of this calculator. Because a structured report is created, the calculator not only improves the efficiency and accuracy of interpretation, it also helps standardize interpretations within our department.

There have been efforts to further automate many aspects of the radiologist practice related to reporting. Standard-based formats such as DICOM structured reporting (DICOM SR) and Annotation and Imaging Markup (AIM) have shown promise in clinical practice (DICOM SR) and in the research environment (AIM) [16, 17]. Even with these efforts, there are many data types that are not accessible via one of these methods. Determining the leg-length discrepancy in the method described above serves as an example of a type of measurement that is not accessible via either method. In this instance, the annotation on the image is used as a straight line to determine an anatomic landmark in relation to a radiopaque ruler included on the image. Other research has described methods for including calculators directly into the reporting template [18]. This work has not yet translated into a commercial product where users can create their own calculators. This lack of flexibility has hampered implementation in the general clinical practice. We anticipate that as this technology matures, vendors will add more functionality including the ability for users to create calculators relevant to their practice.

While the overall mean decrease in time of 92.9 s seems trivial, it adds up. Over the course of the year, we estimated that the leg-length calculator saved 4.3 h of a radiologist’s time. If calculators can be used for other common studies, the time savings could be substantial. We have taken a similar approach in creating other calculators in our department. Currently three calculators are in use in our department: a bone age calculator, a leg-length discrepancy calculator, and a tibial torsion/femoral anteversion calculator. These three calculators help radiologists interpret nearly 2500 studies in our department each year accounting for approximately 1% of all studies. At each clinical workstation in our department, we have tried to make locating the calculators as easy as possible by placing links on the desktop, the quick launch tool bar, and as a favorite button in the web browser.

There are several limitations of this study. Most notably, this time motion study was performed under an idealized situation where the reviewers interpreted films consecutively outside of the true clinical setting. This may have affected the times in several ways. First, the radiologist was overly prepared for each method: each radiologist had a pen, paper, and calculator ready for the manual method and the online calculator open for the automated method. Next, by performing each calculation multiple times, it is likely that the reviewers became more efficient in their leg-length determination process. Finally, the pressure of time may have forced the reviewers to rush through their calculations and dictations making them more likely to err. While each of these limitations may have had an effect on the experiment, we believe that the overall observations remain valid.

A second limitation of this study is that it was underpowered to detect a difference in the frequency of errors. Even though we did not see an error with use of the calculator, we recognize that in its current iteration, errors are possible. Radiologists must still manually enter data in order for this calculator to work. While the potential for error still exists, we believe that the potential for error is less than in the manual method. This is supported by prior studies which have shown that simple mathematical errors are common [19] and that dosing errors decrease when dosing calculators are introduced [7–11].

Conclusion

The use of our leg-length discrepancy calculator with a structured report generator significantly reduced the time required to calculate and dictate leg-length discrepancy studies. While reports trended towards being more accurate, this result did not achieve significance. Calculators with the ability to generate structured reports have the potential to help standardize reporting and simultaneously improve efficiency in a radiology department.