PURPOSE: To examine the combined effects of monitor luminance and ambient ... luminance of the monitors was related to the ambient light for any of the abnor ...
Radiology
Jin Mo Goo, MD Ja-Young Choi, MD Jung-Gi Im, MD Hyun Ju Lee, MD Myung Jin Chung, MD Daehee Han, MD Seong Ho Park, MD Jong Hyo Kim, PhD Sang-Hee Nam, PhD
Index terms: Diagnostic radiology, observer performance Images, display Lung, abnormalities, 60.31, 60.73, 60.917 Lung, radiography, 60.1215 Radiography, digital Published online before print 10.1148/radiol.2323030628 Radiology 2004; 232:762–766 Abbreviations: CRT ⫽ cathode ray tube PACS ⫽ picture archiving and communication system ROC ⫽ receiver operating characteristic 1
From the Department of Radiology, Seoul National University College of Medicine and the Institute of Radiation Medicine, SNUMRC, 28 Yongon-dong, Chongno-gu, Seoul 110 –744, Korea (J.M.G., J.Y.C., J.G.I., H.J.L., D.H., S.H.P., J.H.K.); Department of Radiology, Samsung Medical Center, Seoul, Korea (M.J.C.); and Department of Biomedical Engineering, Inje University, Kim-hae, Korea (S.H.N.). Received April 21, 2003; revision requested July 3; final revision received January 5, 2004; accepted January 21. Supported by Seoul National University Hospital Research Grant (0920010060). Address correspondence to J.G.I. (e-mail: imjg@radcom. snu.ac.kr).
Effect of Monitor Luminance and Ambient Light on Observer Performance in Soft-Copy Reading of Digital Chest Radiographs1 PURPOSE: To examine the combined effects of monitor luminance and ambient light on observer performance for detecting abnormalities in a soft-copy interpretation of digital chest radiographs. MATERIALS AND METHODS: A total of 254 digital chest radiographs were displayed on a high-resolution cathode ray tube monitor at three luminance levels (25, 50, and 100 foot-lamberts) under three ambient light levels (0, 50, and 460 lux). Six chest radiologists reviewed each image in nine modes of combined luminance and ambient light. The observers were allowed to adjust the window width and level of the soft-copy images. The abnormalities included nodule, pneumothorax, and interstitial disease. Observer performance was analyzed in terms of the receiver operating characteristics. The observers reported their subjective level of visual fatigue with each viewing mode. A statistical test was conducted for each of the abnormalities and for fatigue score by using repeated-measures two-way analysis of variance with an interaction. RESULTS: The detection of nodules was the only reading that was affected by the ambient light with a statistically significant difference (P ⬍ .05). Otherwise, observer performance for detecting a nodule, pneumothorax, and interstitial disease was not significantly different in the nine-mode comparison. There was no evidence that the luminance of the monitors was related to the ambient light for any of the abnormalities. The fatigue score showed a statistically significant difference due to both the luminance and ambient light. CONCLUSION: When adequate window width and level are applied to soft-copy images, the primary diagnosis with chest radiographs on the monitor is unlikely to be affected under low ambient light and a monitor luminance of 25 foot-lamberts or more. ©
RSNA, 2004
Authors stated no financial relationship to disclose. Author contributions: Guarantors of integrity of entire study, J.M.G., J.G.I.; study concepts, J.M.G., J.G.I., J.H.K.; study design, J.M.G., J.Y.C., S.H.P.; literature research, J.M.G., J.Y.C., S.H.P.; clinical studies, J.G.I., J.Y.C.; data acquisition, J.Y.C., J.G.I., S.H.P.; data analysis/interpretation, J.M.G., H.J.L., M.J.C., D.H., J.H.K., S.H.N.; statistical analysis, J.M.G., J.Y.C., S.H.P.; manuscript preparation, J.M.G., J.Y.C.; manuscript definition of intellectual content, J.M.G., J.Y.C., J.G.I., S.H.P., J.H.K.; manuscript editing, J.G.I., S.H.P., J.H.K.; manuscript revision/review and final version approval, all authors © RSNA, 2004
762
Digital image acquisition and display technologies are undergoing rapid development, and the work flow in radiology departments has improved through a direct link between the detector and display unit, which is usually achieved by means of a picture archiving and communication system (PACS). With the current technology, image interpretation on cathode ray tube (CRT) monitors can be as efficient and accurate as hard-copy interpretation and is now widely accepted in medical practice. However, a CRT monitor has limited spatial resolution and luminance compared with a radiograph viewed on a standard light box. Although the American College of Radiology has published a standard specifying that monitors used for primary interpretation should have a minimum luminance of 50 foot-lamberts (1), there are unresolved issues relating to the display and viewing standards of the CRT monitors. Accurate image interpretation with the use of CRT viewing stations is one of the major requirements for implementing a PACS. A number of problems, which may influence
Radiology
diagnostic effectiveness, need to be taken into consideration. These include the resolution and luminance of the monitor, the operability of the display station, the operator’s fatigue, and the room illumination (2–7). The objective of our study was to examine the combined effects of monitor luminance and ambient light on observer performance for detecting abnormalities in a soft-copy interpretation of digital chest radiographs.
MATERIALS AND METHODS This study was designed as a multiobserver, multiple-abnormality, observer performance (receiver operating characteristic [ROC]) study, in which the observer performance was measured for three monitor luminance levels and three ambient light levels. The study was conducted at Seoul National University Hospital; our institutional review board did not require its approval or informed consent for this study.
Selection of Cases and Controls One senior resident (J.Y.C.) selected posteroanterior chest radiographs that were obtained in patients with and those without disease between January 1, 2000, and December 31, 2001. The abnormalities of interest were nodule, pneumothorax, and interstitial disease. Cases involving these abnormalities were identified by means of a retrospective search of the hospital’s radiology information system database. Cases of nodule or interstitial lung disease were selected on the basis of confirmation with computed tomographic (CT) findings. Only chest radiographs obtained within 1 week of CT scanning were included (mean interval, 4 days). Selection of criteria for nodules included the following: They were solitary, were no larger than 3 cm in diameter, and were solid nodules or focal air-space opacities that were seen as nodules on chest radiographs. Cases of interstitial lung disease were selected on the basis of CT findings of ground-glass opacity, reticular opacity, and/or honeycombing, and either clinical or pathologic confirmation. Cases of pneumothorax were selected on the basis of either CT scans or follow-up chest radiographs. Subtle pneumothoraces were confirmed when a disappearance or an increase in size was observed on a follow-up image. Chest radiographs obtained with the patient in the supine position or in a patient with chest tube placement were excluded. Negative radiographs were selected on the baVolume 232
䡠
Number 3
sis of the CT scans. Because we reviewed the cases retrospectively, we could not confirm the reasons for CT examination in all cases. Most frequently, reasons for CT examination in patients with normal findings at chest radiography were staging work-up for extrathoracic malignancies or further work-up to explain a patient’s respiratory symptoms. From the cases initially selected on the basis of aforementioned criteria, one chest radiologist (J.G.I., 28 years of experience in chest radiography) and one senior resident (J.Y.C.) viewed and categorized each radiograph according to the presence or absence of abnormalities. Radiographs with image artifacts such as respiratory motion, inadequate field of view, or poor positioning were excluded. Decision was made by consensus. We selected consecutive images that showed the abnormalities and did not group these cases according to the degree of subtlety of abnormality depiction. Finally, 254 posteroanterior chest radiographs were selected for this study. The numbers of occurrences of a nodule, pneumothorax, and interstitial disease were 113, 66, and 50, respectively, on a total of 190 radiographs. Thirty-nine images showed more than one abnormality. Sixty-four radiographs were negative for all three abnormalities.
foot-lamberts by using a luminance meter (Putlamp; UDT instruments, San Diego, Calif), and the room illuminance was set at 0, 50, and 460 lux at the console desk by using an illuminance meter (Lux HiTester; HIOKI, Nagano, Japan). The lowest luminance level of the monitor in the study was selected to simulate the average luminance level of monitors after 3 years of use in the clinical wards of our hospital. The middle level of the monitor was equal to the average luminance level of monitors in our reading rooms, which are set at the minimum luminance level recommended by the American College of Radiology for display workstations (1). The highest monitor luminance level was the maximum brightness level of CRT monitors. The highest level of room illuminance (ambient light) was chosen to simulate the illuminance of the clinical wards or emergency rooms and was achieved by turning on all the lights in the reading room. The middle level of ambient light was equal to the approximate illuminance level of the reading rooms and was achieved by using a dimmer to lower the level of light. The lowest room illuminance was achieved by turning off all lights in the reading room. The room lighting did not cause direct glare on the PACS viewing monitors.
Image Acquisition and Display
Image Evaluation
The posteroanterior chest radiographs were obtained with an FCR-5501 unit (Fuji, Tokyo, Japan) by using 35 ⫻ 43-cm imaging plates (model ST-55; Fuji), thus generating images with a 1760 ⫻ 2140 ⫻ 10-bit matrix and a 0.2-mm pixel size. The digital data were sent to a PACS server (Radmax; MaroTech, Seoul, Korea) and distributed to the workstations. All images were downloaded onto the local hard drive of a display workstation prior to interpretation. A 21-inch monitor (DR110; Dataray, Denver, Colo) with a resolution of 2048 ⫻ 2560 ⫻ 8 bits was used for the study. The monitor was operated at 71 Hz in an interlaced mode and had a maximum brightness level of 100 foot-lamberts. The monitor was calibrated with a Society of Motion Picture and Television Engineers test pattern to make 5% of the squares visible at both ends of the gray scale. The CRT faceplate was covered with an antireflection screen coating. The softcopy images were displayed without image postprocessing such as spatial frequency enhancement. The monitor luminance was set at 25, 50, and 100
The study consisted of a nine-mode comparison in which each of the three monitor luminance levels was paired with each of the three levels of ambient light. Six observers read and rated each of the 254 cases in each of the nine modes. The image reading sessions were conducted in the reading room of the radiology department. Six board-certified chest radiologists (including J.M.G., H.J.L., M.J.C., D.H.), each experienced in using PACS, independently reviewed all the digital chest radiographs. The radiologists who participated in the preparation of the study and case selection were not among the six observers. Among the observers, the level of experience regarding chest radiography ranged from 6 to 12 years (mean, 8.5 years), and the level of experience with PACS ranged from 2 to 5 years (mean, 3.3 years). Prior to the study, all observers were instructed on the definitions of the abnormalities and the presence of normal cases and were trained in the use of the computerized scoring system. A separate set of radiographs was used for the training sessions. Each reading session included 84 or 85 Monitor Luminance and Ambient Light
䡠
763
Radiology
TABLE 1 Area under the ROC Curve for Detection of Abnormalities at Three Levels of Monitor Luminance and Ambient Light Monitor Luminance (foot-lambert) and Ambient Light (lux) 25 0 50 460 50 0 50 460 100 0 50 460
Abnormality Nodule
Pneumothorax
Interstitial Disease
0.87 ⫾ 0.06 0.86 ⫾ 0.08 0.84 ⫾ 0.05
0.95 ⫾ 0.07 0.92 ⫾ 0.08 0.90 ⫾ 0.07
0.98 ⫾ 0.01 0.96 ⫾ 0.03 0.97 ⫾ 0.02
0.86 ⫾ 0.06 0.86 ⫾ 0.05 0.85 ⫾ 0.07
0.95 ⫾ 0.08 0.94 ⫾ 0.08 0.92 ⫾ 0.07
0.96 ⫾ 0.04 0.97 ⫾ 0.02 0.96 ⫾ 0.02
0.85 ⫾ 0.07 0.85 ⫾ 0.09 0.84 ⫾ 0.10
0.95 ⫾ 0.06 0.94 ⫾ 0.09 0.91 ⫾ 0.10
0.97 ⫾ 0.02 0.98 ⫾ 0.01 0.95 ⫾ 0.06
Note.—Data are means ⫾ standard deviations, unless otherwise indicated. The detection of nodule showed a statistically significant difference due to ambient light (P ⬍ .05, repeated-measure two-way analysis of variance).
TABLE 2 Fatigue Score for Each of Three Monitor Luminance and Ambient Light Levels Monitor Luminance (foot-lambert) and Ambient Light (lux) 25 0 50 460 50 0 50 460 100 0 50 460
Fatigue Score* 2.33 ⫾ 0.52 2.67 ⫾ 0.82 4.33 ⫾ 0.52 2.83 ⫾ 0.75 2.00 ⫾ 0.20 3.83 ⫾ 0.41 3.17 ⫾ 1.17 3.67 ⫾ 0.82 5.00 ⫾ 0.00
Note.—Results were significant for both monitor luminance and ambient light (P ⬍ .001, repeated-measure two-way analysis of variance). * Data are means ⫾ standard deviations.
randomized cases displayed in one of the viewing modes. Each observer interpreted the cases in a different order to avoid bias. The images and viewing modes were randomized. The sessions were held at least 1 week apart to reduce the learning bias. No limit was imposed on the reading time. The viewing time was recorded automatically; viewing time began with the selection of each case and ended with the completion of marking the scores. For ROC experiments, a continuous rating scale from 0% to 100% was used to represent each observer’s confidence level regarding the presence or absence of a nodule, pneu764
䡠
Radiology
䡠
September 2004
mothorax, or interstitial disease. A rating of 0% indicated absolute certainty that the abnormality in question was not present on the image, and a rating of 100% indicated absolute certainty that it was present. For scores between 0% and 100%, the observer selected a percentage that corresponded to the probability that the abnormality was present. Three sliding scales, one for each abnormality, were presented. The observers were allowed to adjust the window width and level of the images. In daily practice with our PACS viewer, radiologists routinely adjust the window width and level (by simple manipulation with the computer mouse) to confirm any abnormality in the obscured area, such as retrocardiac or subdiaphragmatic lung. We did not track how often they manipulated window settings electronically. However, after the completion of the experiment, we asked the observers how often they manipulated window settings in this experiment on a 10% scale. The observers did not know the proportion of abnormalities. Subjective visual fatigue was scored at the end of each session by using a scale of zero to five. Prior to the study, all observers were instructed that this score referred to the subjective visual fatigue that was increased by reading cases in this study.
Data and Statistical Analysis To test the observer performance, the ROC analysis with a calculation of the area under the ROC curve was performed for each abnormality (n ⫽ 3), mode (n ⫽
9), and observer (n ⫽ 6) (3 ⫻ 9 ⫻ 6 ⫽ 162 values) by using the ROCKIT program (Metz C, University of Chicago, Ill). A statistical test of the possible effects of monitor luminance and ambient light on the observer performance was conducted for each of the abnormalities by using a 3 ⫻ 3 repeated-measure two-way analysis of variance with an interaction, where the areas under the ROC curves for the different observers were considered as replicates. A statistical test of the possible effects of the monitor luminance and ambient light on the time required by the observers and the subjective fatigue score was conducted by using the same method. Statistical analysis was performed by using SPSS for Windows (version 10.0; SPSS, Chicago, Ill), and P ⬍ .05 was considered to indicate statistical significance.
RESULTS Table 1 summarizes the mean areas under the ROC curves for all six observers for the three levels of monitor luminance and the three levels of ambient light for each of the three abnormalities. The values in Table 1 are the means among all six observers. There were no significant interactions between the monitor luminance and ambient light for any of the three abnormalities, which indicated that the effect of monitor luminance on the observer performance was not affected by the ambient light and vice versa. Monitor luminance had no statistically significant effect on the observer performance for any of the three abnormalities (P ⬎ .05), according to a repeated measure of the two-way analysis of variance results. In the detection of nodules, there was a significant decrease in the mean area under the ROC curves with increasing ambient light (P ⫽ .035) according to a repeated measure of the twoway analysis of variance results. However, for the detection of pneumothorax or interstitial disease, the ambient light did not have a statistically significant effect on observer performance. When viewing time was averaged over all observers for a particular mode, the mean time (⫾ standard deviation) needed by observers to view and score each case ranged from 17 seconds ⫾ 5 to 21 seconds ⫾ 4. There were no significant differences (P ⬎ .05) in the duration between the individual modes. Table 2 summarizes the mean fatigue score for all six observers for the three levels of monitor luminance and three Goo et al
Radiology
levels of ambient light. There was no significant interaction between monitor luminance and ambient light levels for fatigue. The level of fatigue was significantly affected by both the monitor luminance and the ambient light (P ⬍ .001). The mean fatigue score for the middle luminance level of the monitor was 2.89, which was lower than that for the low (3.11) or high (3.94) luminance levels. The mean fatigue score for the high ambient light level was 4.39, which was higher than that for the low (2.78) or middle (2.78) ambient light levels. All observers responded that they manipulated window settings in more than 90% of cases.
DISCUSSION The use of PACS and primary soft-copy interpretation in radiology is growing rapidly. Results of studies on the comparison of the various digital detectors by means of soft-copy reading have been reported (8,9). However, one potential impediment to the regular use of digital systems is that the optimal display design and performance factors for reading the radiographs on monitors are not yet completely understood. A wide range of views concerning the spatial resolution of the monitors has been presented (4,6). For chest radiography, the pixel size should be no larger than 0.2 mm, which provides a resolution of 2.5 line pairs per millimeter. The systems that are currently available can satisfy this requirement. The inherent advantages of digital radiologic equipment can be fully exploited only when the primary image evaluation is also based on a digital format. The level of luminance is important in detecting structures with low contrast, particularly those in the regions of high absorption. In this study, the level of monitor luminance had no significant effect on detection of any of the three abnormalities. In previous studies on the display luminance, a number of findings led to varying conclusions. In the results of a phantom study by Otto et al (6), the areas under the ROC curves were higher with the use of a high-luminance monitor. Herron et al (4) reported that the detection of a pneumothorax, interstitial disease, and a rib fracture showed statistically significant differences due to the luminance. However, because these cases were displayed in a view box and with use of digitized images, the setting was different from that in our study. Ikeda et Volume 232
䡠
Number 3
al (10) showed that the deterioration in the CRT monitor luminance has a detrimental effect on the detection of nodules on chest radiographs. The study by Ikeda et al was performed under the relatively high ambient illumination of 200 lux. In most of these described studies, adjustments of the window width and level settings were not permitted (4,6,10). Window width or level setting adjustment is the most commonly used workstation tool in a soft-copy interpretation. Because most digital images have a much wider dynamic range than do monitors, inhibiting the adjustment of window width and level settings of an image brings about a sacrifice of an important advantage of digital images. By adjusting the window width and level settings, image information over a wide dynamic range can be better delivered to an observer’s visual system than can a conventional hard-copy interpretation. For example, a subtle pneumothorax is difficult to perceive with a low-luminance monitor. However, if the observer adjusts the window width and level of the region of interest, the observer can perceive the subtle lesion by enhancing the detail contrast ratio, which is defined as the ratio of the luminance between two small adjacent areas (2). Magnification of the soft-copy images was not permitted in our study because the spatial resolution was not a concern of the study. The level of ambient light is an important characteristic of the viewing environment both in the radiology reading rooms and in the clinical patient care areas. By the nature of their design, reading rooms tend to provide improved viewing conditions when compared with other clinical areas. However, substantial variations can also occur in these locations. Ambient light is reflected off the CRT faceplate, which is added to the light emitted by the phosphor and can reduce the contrast of an image (11). The observed image quality is always affected by the scattering of ambient light in the direction of the viewer. Veiling glare and ambient light reflection substantially degrade display quality by increasing the luminance in the black regions, with a consequently reduced contrast ratio and a diminished structural display contrast (12). As a consequence, the discrimination of the low-contrast stimuli is reduced as the ambient light is increased. Although antireflection coatings were applied to the CRT faceplates in our study to help reduce the effect of glare and reflection, they could not completely be eliminated. These results confirm those
of previous reports, which showed that bright ambient light has substantial potential to degrade the detection limit of low-contrast signals such as those from pulmonary nodules (11,13). The monitor luminance and ambient light conditions, which impose stricter lighting requirements in the reading room, may result in observers inability to engage in long reading sessions because of fatigue and an inability to concentrate. The results of the fatigue scoring showed that the observers felt more fatigue with a high level of ambient light and monitor luminance than with other conditions. There were limitations in this study. Because in this study we evaluated only the performance of experienced chest radiologists, they might have detected subtle abnormalities at a low-luminance monitor by adjusting the window settings. Less-experienced observers may show different results. These results do not suggest that the level of display luminance does not affect the detection of abnormalities in soft-copy reading, because the lowest luminance level of the monitor in our study was 25 foot-lamberts. Although a monitor luminance level of 25 foot-lamberts does not meet the recommendations of the American College of Radiology, this luminance represents the average brightness of most older-generation monitors (4). Many authors strongly recommend the implementation of a quality control program because the CRT display function degrades over time (3,10,14). Although we did not electronically measure how often observers manipulate window settings in this study, all observers responded that they used this function in most cases. We could not use a Digital Imaging and Communications in Medicine grayscale standard display function, or DICOM GSDF, for calibration of monitors because the monitors used in this study did not support the function. However, the authors calibrated the monitor with a Society of Motion Picture and Television Engineers test pattern to make both lowcontrast squares (those in backgrounds with 0% digital driving value and another with 100% digital driving value) visible at each level of room illumination. This ensures the monitor characteristic curve to shift up to an appropriate level so that the observers might perceive a certain amount of just noticeable differences, at the darkest backgrounds of displayed images, which is one of the most important points of DICOM GSDF in soft-copy image presentation. Therefore, we believe that the fact that the Monitor Luminance and Ambient Light
䡠
765
Radiology
monitors were not exactly calibrated with the DICOM GSDF did not have a significant effect on the results of this study. In conclusion, when adequate window width and level of the soft-copy images are applied, a primary diagnosis of chest radiographs on a monitor is unlikely to be affected under a low ambient light and monitor luminance of 25 foot-lamberts or more. References 1. American College of Radiology (ACR). ACR standard for digital image data management. Available at: www.acr.org. Accessed April 10, 2002. 2. Arenson RL, Chakraborty DP, Seshadri SB, et al. The digital imaging workstation. Radiology 1990; 176:303–315. 3. Groth DS, Bernatz SN, Fetterly KA, et al. Cathode ray tube quality control and acceptance testing program: initial results for clinical PACS displays. RadioGraphics 2001; 21:719 –732.
766
䡠
Radiology
䡠
September 2004
4.
5.
6.
7.
8.
9.
Herron JM, Bender TM, Campbell WL, et al. Effects of luminance and resolution on observer performance with chest radiographs. Radiology 2000; 215:169 –174. Ishihara S, Shimamoto K, Ikeda M, et al. CRT diagnosis of pulmonary disease: influence of monitor brightness and room illuminance on observer performance. Comput Med Imaging Graph 2002; 26: 181–185. Otto D, Bernhardt TM, Rapp-Bernhardt U, et al. Subtle pulmonary abnormalities: detection on monitors with varying spatial resolutions and maximum luminance levels compared with detection on storage phosphor radiographic hard copies. Radiology 1998; 207:237–242. Krupinski E, Roehrig H, Furukawa T. Influence of film and monitor display luminance on observer performance and visual search. Acad Radiol 1999; 6:411– 418. Goo JM, Im JG, Kim JH, et al. Digital chest radiography with a selenium-based flatpanel detector versus a storage phosphor system: comparison of soft-copy images. AJR Am J Roentgenol 2000; 175:1013– 1018. Goo JM, Im JG, Lee HJ, et al. Detection of
10.
11.
12.
13.
14.
simulated chest lesions by using soft-copy reading: comparison of an amorphous silicon flat-panel-detector system and a storage-phosphor system. Radiology 2002; 224:242–246. Ikeda M, Ishigaki T, Shimamoto K, et al. Influence of monitor luminance change on observer performance for detection of abnormalities depicted on chest radiographs. Invest Radiol 2003; 38:57– 63. Roehrig H, Krupinski EA, Furukawa T. Evaluation of a flat CRT monitor for use in radiology. J Digit Imaging 2001; 14: 142–148. Flynn MJ, Kanicki J, Badano A, et al. High-fidelity electronic display of digital radiographs. RadioGraphics 1999; 19:1653– 1669. Alter AJ, Kargas GA, Kargas SA, et al. The influence of ambient and viewbox light upon visual detection of low-contrast targets in a radiograph. Invest Radiol 1982; 17:402– 406. Ly CK. Softcopy display quality assurance program at Texas Children’s Hospital. J Digit Imaging 2002; 15(suppl 1):33– 40.
Goo et al
