mediate surrounding area determined. As expected, the perception of sharp edged objects was much greater than for the smoothed edged objects. To illustrate ...
Perception of Computer Simulated Pulmonary Lesions in Chest Radiographs J.M. Degroot, E.L. Hall, R.N. Sutton, G.S. Lodwick, S.J. Dwyer, Ill University of Missouriat Columbia
ABSTRACT
INTRODUCTION
Simulated abnormalities in radiographs may be used to investigate a number of problems of interest to radiologists. First, in the study of radiographic image quality, the manner in which physical image quality affects perception and diagnostic a b i l i t y may be quantitatively explored. Second, since the increasing demand for diagnostic radiological examinations has stimulated studies to determine whether the effectiveness and efficiency of radiologists can be increased by the use of trained technical assistants, the simulated abnormalities provide a useful method for trained technical assistants, the simulated abnormalities provide a useful method for t r a i n ing and evaluation of the technical assistants. Previous radiographic simulations have been made using photographic and television systems; however, d i g i t a l computer simulation provides greater f l e x i b i l i t y and quantitive control. The purpose of this paper is to describe a system for digital simulation of nodular lesions in chest x-rays. The effects of lesion size, contrast, and shape are easily studied with the simulation system. For example, lesions of sizes from 2 to lO mm were generated at various contrast levels to test the Weber-Fechner law for a complex image with two different shape profiles - a sharp edge and Gaussian. The threshold of perception for each lesion was determined and the ratio of the contrast of the lesion to the contrast of the immediate surrounding area determined. As expected, the perception of sharp edged objects was much greater than for the smoothed edged objects. To i l l u s t r a t e this use a series of test i mages with lesions of various contrast levels and sizes were generated. This series of images was then shown to several radiologists and their lesion detection a b i l i t y as a function of contrast, size, and viewing distance, was measured. This work was supported in part by NIGMS Grant 17729-01.
After reading thousands of radiographs, the radiologist's error rate may be alarmingly high. Garlandi (1959) found that experienced radiologists often incorrectly diagnosed 30% of positive chest roentgenograms in tests. Efforts have been made to decrease diagnostic error rate from the standpoint of both the reader and X-ray. Tuddenham2 has suggested that the perception of the reader might be improved by using systematic scanning aids and by improved learning with programmed instruction. While these methods may be helpful, considerableattention has been focused on the improvement of the image quality of the X-ray. The information content of the object being radiographed depends upon the absorption charact e r i s t i c s of the object, which are determined by the thickness, density, and material properties of the object exposed at a selected X-ray energy level. The resolution of a radiographic recording ~ystem depends upon the X-ray focal spot size°, the spe~d of the image i n t e n s i f i e r screen(s) used~, object scatter and motion and the object-film distance. X-ray image contrast is affected primarily by kilovoltage rating and in a secondary way by object scatter. In addition, the random effect of quantum mottle, resulting from spatial changes in X-ray illumination, adds independent noiseto the image. Provided that the combination of milliamperage and the source-object distance is adjusted so that sufficient film exposure is available and object-film distance is minimized, the image contrast and resolution depends primarily on the kilovoltage and focal spot size, respectively. Thus by proper selection of the variables, the most detailed f i l m would be obtained. However, i t is possible that further processing of the original X-ray might f a c i l i t a t e diagnosis. Several image enhancement methods have been applied toward this end. These include the use of photography, xerography, coherent optics, black and white and color video, and diQital Dic-
146
ture processing. Digital methods have the advantage of being able to process images with great f l e x i b i l i t y and accuracy. Nonlinear and linear transformations are possible by digital means, and may be used to improve apparent quality of the X-ray. The gray level histogram ( f i r s t order prob a b i l i t y density function) of most images exhibits a brightness peak which is heaxity biased to the dark side of the histogram5,6, I . The effect of having a large percentage of the picture elements (pixels) concentrated into such a narrow portion of the histogram is an image with l i t t l e visible detail. Manycontrast ratios that define edges are simply not visible to the human eye. One technique frequently used to correct this is the position invariant nonlinear application of a software or hardware logarithmic conversion of the brightness pixels. This yields pixels which are proportional to image film density rather than brightness 8. The log operation has the effect of expanding the gray range of the lower brightness pixels while compressing that of the higher brightness pixels. However, because of the shape of the brightness histogram, there is much Jnore contrast expansion than compression.
Linear techniques may also be used to enhance X-ray images. This may be accomplished by taking the two dimensional Fourier transform of the digitized picture points, then f i l t e r i n g in the spat i a l frequency domain, and f i n a l l y taking the inverse Fourier transform. The higher energy, lower frequency components contain much of the image contrast information. The low energy, higher frequency coefficients are primarily responsible for image edge information. When an image is low pass f i l t e r e d , the image contrast is generally unaffeRted. However, edge detail is effectively removed~. The most common application of low pass f i l t e r i n g of images is to reduce the effect of random picture noise when the image is used for measurement selection and pattern recognition purposes. The effect on an image of high pass f i l t e r ing is to remove the contrast information of the image while enhancing edges. The edge outlining effect can be seen most obviously at sharp, high contrast ratio edges such as along the patient diaphragm. A major application of high pass f i l ters is in the visualization of small, low contrast features superimposed onto uniform back-
147
groundsl O ' l l . I t is also possible to combine l i near and nonlinear f i l t e r i n g . Obviously, there are many linear and nonlinear methods by which the X-ray may be processed to improve the apparent quality, but most radiologists have been reluctant to work from altered X-rays, and no method has become standard for evaluating the diagnostic quality of processed images.
between two objects such as lines. The separation is measured at the distance at which the half power responses of the two lines cross. Resolution may also be considered as the minimum discernible object size. Clearly, improving the resolution in an image should improve the q u a l i t y ; however, a point of no return could be reached. For example, one may consider the retina to be covered by a checkerboard type mesh with checker~shaped in each square. Minimumresolution occurs when each discrete photoreceptor is covered by a single checker. The distribution of photoreceptors and thus resolution is high in the fovea and decreases toward the retina edges. The discrete d i s t r i b u tion of photoreceptors is the physiological basis for considering an image as a discrete set of points. However, for subjective image evaluation, the resolution of fine details does not appear as important as sharpness in the image. Threshold relationships in visual perception are indicated by two phenomenae of vision. F i r s t , l i g h t is composed of discrete photons and thus l i g h t intensity is a random variable. Secondly, the response of neurons is described by a threshold process. Thus, to expect the entire human visual system to behave l i k e a linear optical system would involve a very gross assumption. The concept of threshold visual perception and i t s relationship to photon fluctuation and frequency response are discussed by Morganl4o He proposes that threshold visual perception is governed by both the prevailing noise contrast and the spatial frequency response of the visual system to the pattern under observation. He also uses the model to predict the performance of some complex visual systems, including radiographic systems. Morgan also proposes that the response of the visual system to noise is independeDt of the object under observation. That i s , a given level of noise is equally annoying whether one is viewing a l a r ~ or small c i r c u l a r object in a frame. Schreiber i~ states some other observations on noise effects. He indicates that noise is more v i s i b l e in blank f i e l d s than superimposed onto objects° Also, the more complicated the image, the less v i s i b l e the noise. Whennoise is added to an image, i t is generally more v i s i b l e i f i t is correlated with the image than i f i t is random. Thus systematic noise and quantizing noise are more annoying than random noise. The presence of noise reduces contrast and acuity very s i g n i f i cantly. Three types of measures are commonly proposed for evaluating image quality. These are weighted mean square error, correlation measures, and information measures. Each proposed measure has at least an i n t u i t i v e j u s t i f i c a t i o n . For example, one might consider a frequency domain weighted mean square error for which the frequency weighting function "matches" the frequency response of the HVS. For this measure, errors which occur at frequencies emphasized by the HVS, e.g. lO lines/mm, would be weighted more heavily than errors at frequencies which are attentuated by the HVS, e . g . O . l lines/mm. Also, one could consider "low pass" or "high pass" frequency weighted mean square errors.
EVALUATING IMAGEQUALITY No method has been developed for evaluating the quality of these processed X-rays. The problem of evaluating image quality for processed radiographic images is important since i t is directly related to diagnostic a b i l i t y . The most common method for evaluating image quality is subjective "expert" human evaluation; however, the disagreements and inconsistencies encountered motivate the search for further understanding of this problem and an objective measure of quality. The frequency response of the human visual system has been re@Rrted by several researchers, including Campbell~L and Morgani ° . Spatial frequency is defined as the reciprocal of the distance between successive maxima in a sinusoidal i n t e n s i t y variation measured at the retina. The response curve has a parabolic shape with I0% values at 0 and 75 cycles/mm, and a peak at lO cycles/mm. Thus we perceive, at an attenuated level, blank images and very high frequency images and perceive best at lO cycles/mmo From the results of Campbell and Morgan, i t is reasonable to model the human visual system up to and including the retina as a spatial f i l t e r , which is characterized by a modulation transfer function° These results may be used to describe the subjective effect known as sharpness or acuity of an object. Acuity is related to the rendition of objects much larger than those barely v i s i b l e . That i s , humans look for sharp edges on rather large objects. The transform of a c i r c l e was known to be a Bessel function with side lobes. The main lobe of this transform is inversely related to the diameter of the c i r c l e . One may now consider the composite response of a human observer to be the product of the transform of the c i r c l e and the modulation transfer function of the visual system. For a small c i r c l e , only the main lobe w i l l be passed by the retina. The effect would be similar to low pass f i l t e r i n g , thus the edges of the c i r c l e would be blurred. For a larger c i r c l e , the main lobe and several side lobes might f a l l within the passband of the visual system, thus the edge rendition would be much sharper. Therefore, humans may expect sharp edges on large objects because the frequencies involved are within the capability of the HVS, but would not expect sharp edges on barely perceptible objects. Another subjective judgment used by humans is related to discerning fine d e t a i l . The capability of a system for resolving fine detail is measured by i t s resolution. Resolution is an often misused measure of an imaging system, since no system can be characterized by a single number. I t does give a lower bound on available performance. Resolution may be defined as the minimum discernible separation
148
S i m i l a r l y , the appeal of c o r r e l a t i o n measure is due to the "matching" c a p a b i l i t y of a correlation f u n c t i o n . Functions which are a l i k e should have a large degree of c o r r e l a t i o n , while functions which are d i f f e r e n t should have a small degree of c o r r e l a t i o n . Recently proposed frequency weighted c o r r e l a t i o n measures have also been Considered. The frequency weighting function may be selected to emphasize a c e r t a i n portion of the frequency spectrum; f o r example, high frequencies which contain edge information. One p o s s i b i l i t y with a frequency weighted measure is t h a t , f o r a f i x e d r e s o l u t i o n image, one might match the acutance aspect of subjective human image evaluation. Also, with a low pass weighting f u n c t i o n , one might develop a contrast measure which agrees with human evaluation. One method of evaluation is based upon obj e c t i v e psychophysical t e s t i n g procedures ~pmmonl y used in radiology, e . g . , Kundel, et a l . Lu (1968). Using a videl image processing system, normal X-rays were scanned with simulated lesions superimposed. Radiologists were asked to detect the presence of the lesions with and without nonl i n e a r contrast enhancement image processing. False p o s i t i v e and false negative errors in det e c t i n g lesions were recorded. Results showed that video contrast enhancement reduced the detection error rate of the video display, but the best r e s u l t s were obtained from the conventional X-ray view box. Apparently, the 525 l i n e video display degraded the q u a l i t y of the X-ray image s u f f i c i e n t l y so that contrast enhancement did not o f f e r any improvement over the o r i g i n a l X-ray. While the video system is simple to use f o r t e s t i n g purposes, i t is lacking in res o l u t i o n and c o n t r o l . For example, smooth edge shapes f o r tumors cannot be generated° The video system also has l i m i t e d capacity f o r image processing. This project proposes to evaluate the nature of the problem of f a i l u r e to detect the presence of small pulmonary lesions, to determine the r e l a t i v e success of the computer and the human in det e c t i n g such lesions, and to determine what measures may be taken, i f any, to improve human and computer performance.
Fig. I . At the bottom is a l i g h t source, at the top is an image dissector camera. To the l e f t is an analog d i g i t a l converter. The X-ray is placed over the D.C. l i g h t source, and the camera, under computer c o n t r o l , scans the p i c t u r e . The camera moves p o i n t - b y - p o i n t over a 256 x 256 checkerboard and samples the transmitted l i g h t at each point. The video signal from the camera is analog to d i g i t a l converted (A/D), which converts the gray level i n t o a number and then passes i t to the computer, which in t h i s case is an SEL 840A. The computer transfers the d i g i t i z e d informat i o n to i t s tape drive and onto magnetic tape. Then the tape is ready f o r processing on an IBM 360-50. At t h i s point, a normal chest x-ray has been scanned and the d i g i t i z e d x-ray is used as an input to the simulation program. This program produces a simulated d i g i t a l tumor superimposed onto the normal d i g i t i z e d chest x - r a y , and w r i t t e n on an output d i g i t a l tape. Then the d i g i t i z e d image is converted back into a gray level image on the SEL. This conversion is accomplished with the aid of the Dicomed display u n i t and may be viewed e i t h e r d i r e c t l y on i t or from photographs and slides of the images displayed. Now consider the software portion of the proj e c t . A portion of the program showing the simul a t i o n calculations is shown in Fig. 2. I t is w r i t t e n in Fortran IV and is designed to calculate and generate the tumor. As one may see, i t is an extremely simple program. To understand the program operation, consider a small area called the normal area, and suppose that the gray level values are as shown in Fig. 3(a). Suppose one wants to superimpose the tumor shown in Fig. 3(b) in t h i s area. The tumor absorbs X-rays as they pass through the human body, since a tumor has greater mass than any other area surrounding i t . I t w i l l absorb more rays and w i l l consequently come out l i g h t e r . So, the tumor gray level values must be subtracted from the normal area numbers. The r e s u l t i n g area is a s t r a i g h t subtraction from the normal area and is shown in Fig. 3(c). One may see that the lesion is the amount that would be taken from the normal. A reasonable question at t h i s point i s : how does one represent a tumor, what shape, what size, what contrast? An actual tumor image was d i g i tized and is shown in Fig. 4. I f a tumor may be modeled as a spherical mass, then a good representation of the tumor shape is a Gaussian funct i o n . A simulated Gaussian shaped lesion is shown in Fig. 5. One may compare the two areas d i r e c t l y ; however, since the tumor edge shape is an important parameter in detection, i t is also useful to compare the x - p r o f i l e s of the actual and simulated lesions. These p r o f i l e s are shown in Fig. 6.
COMPUTER SIMULATION Simulated abnormalities in radiographs may be used to investigate a number of problems of i n t e r e s t to r a d i o l o g i s t s . F i r s t , in the study of radiographic image q u a l i t y , the manner in which physical image q u a l i t y affects perception and d i agnostic a b i l i t y may be q u a n t i t a t i v e l y explored. Second, since the increasing demand f o r diagnos t i c r a d i o l o g i c a l examinations has stimulated studies to determine whether the effectiveness and e f f i c i e n c y of r a d i o l o g i s t s can be increased by the use of trained technical assistants, the s i mulated abnormalities provide a useful method for t r a i n i n g and evaluating technical assistants. Previous radiographic simulations have been made using photographic and t e l e v i s i o n systems: however, a d i g i t a l computer simulation provides greater f l e x i b i l i t y and q u a n t i t a t i v e c o n t r o l . A b r i e f description of the hardware, s o f t ware, and an example experiment w i l l now be given. The process begins at the image scanner shown in
EXPERIMENTAL RESULTS A series of radiographic images with superimposed simulated lesions such as that shown in Figure 7 were generated in order to study radiol o g i s t s ' a b i l i t y to detect lesions as a function of the size and contrast level of the l e s i o n . The images with simulated lesions were mixed with an equal number of images without simulated l e sions and were presented to three r a d i o l o g i s t s . The r a d i o l o g i s t s were asked i f a lesion was present in each image and i f so, what s i z e , c o n t r a s t ,
149
and location. From the detection experiment, a curve of detection percentage VSo contrast r a t i o ( i . e . , the contrast of the lesion normalized to the background contrast) f o r various size lesions was developed and is shown in Fig. 8. Note that f o r a given size lesion, say the .5 cm radius or 1 cm diameter curve, there is a c h a r a c t e r i s t i c shape of the detection curve. For small values of Cr , the lesion is not v i s i b l e even though i t is there. There is a threshold value f o r Cr where the lesion becomes v i s i b l e but not obvious. Next on the curved portion of the graph, the percentage detection increases with Cr, and f i n a l l y there is a saturation value of Cr where the l e s i o n d e t e c t i o n is certain f o r trained radiologists.
~m~n
O,GIT~. T~'~
,m*~ SCaM£R
~
t
Df~D sm~
T~E D,S~AV
REFERENCES I . L. H. Garland, "Studies of the Accuracy of Diagnostic Procedures," Radiology; 82, 1959, ppo 25-38° 2° Wo J. Tuddenham, "Visual Search, Image Organization and Reader Error in Roentgen Diagnosis," Radiology, 78, 1962, pp. 694-704° 3. Kunio Doi, "Optical Transfer Function of the Focal Spot of X-ray Tubes," Radiation Therapy and Nuclear Medicine, October, 1965. 4: K. Rossman, "Measurement of the Modulation Transfer Function of Radiographic Systems Containing Fluorescent Screens," Phys. Medo Biolo, g, 1964. 5. J. D. Campbell, "Edge Structure and the Representation of Pictures," Ph.D. Dissertation, University of Missouri-Columbia, 1969. 6. MIT Summer Workshop Notes, Image Processing, 1968. 7. A. V. Oppenheim, R. W. Schafer and T. G. Stockham, "Nonlinear F i l t e r i n g of M u l t i p l i e d and Convolved Signals," IEEE Proc., August, 1968, pp. 1264-1291. 8. C. R. Neblette, Photography, I t s Materials and Processes, Van Nostrand, New York, 1951. 9. A. Rosenfeld, Picture Processing by Computer, Academic, New York, 1969. I0. R. Selzer, "Improving Biomedical Image Quali t y with Computers," Jet Propulsion Lab. Tech. Report, 32-1336, 1968. I i o R. P. Kruger, E. L. H a l l , S. J. Dwyer and G. S. Lodwick, "Digital Techniques for Image Analysis of Radiographs," I n t , J. Bicomed. Comput, A p r i l , 1971. 12. R. W. Campbell, "The Human Eye as an Optical F i l t e r , " Proceedings IEEE, 56, June, 1968, pp. 1009-1014. 13. R. H. Morgan, "Visual Perception in Fluroscopy and Radiography," Proceedings 51st Annual Meeting of Radiological Society of North America, Chicago, November, 1965, ppo 403-416. 14. R. H. Morgan, "Threshold Visual Perception and i t s Relationship to Photon Fluctuation and Sine Wave Response," American Journal of Roentgenology, 93, A p r i l , 1965, ppo 982-997. 15. W. F. Schriever, "Picture Coding," Proceedings IEEE, 55, March, 1967, pp. 320-330. 16. G. Reresz, H. Kundel and C. H a l l , "Electronic Techniques f o r Radiographic Image Processing", Med. and B i o l . Engr., 7, 1968, pp. 393-399.
FIGURE I .
BLOCKDIAGRAM OF EXPERIMENTAL CONDITIONS.
C ADD TUMOR RR = R * R LXL = LX-2*R LXU = LX+2*R LXL = LY-2*R LYU = LY+2*R DO 20 1 = LXL, LXU DO 20 J = LYL, LYU F1 = ((I-LX)**2+(J-LY)**2)/(2.*RR) F : (I-LX)**2+(J-LY)**2-R*R IF (F.LE.O.O)A(I,J) : A ( I , J ) = CONT*EXP(-FI) 20 CONTINUE FIGURE 2.
SIMULATION PROGRAM.
8
9
9
2
3
2
8
8
7
2
4
2
7
9
9
2
3
2
a.
NORMAL AREA.
b.
6
6
7
6
4
5
5
6
7
c. FIGURE 3a-c.
150
SIMULATED LESION.
RESULT AREA.
TUMOR SIMULATION PROCESS.
2123
3423121210
1223
4434421211
1223
2336444431
1233
4344554532
1224
4445453432
1113
3535333423
1023
5435453311
2313
3534453232
FIGURE 4. ACTUAL TUMORDIGITIZED. O00'O0000400000 OOOOO5555555OOOOO 00045555655554000 00455666666655400 00556667776665500 05566677777666550 05566777777766550 45667777877776654 05566777777766550 05566677777666550 00556667776665500 00455666666655400 00045555655554000 00000555555500000 00000000400000000 FIGURE 5.
O00
CHEST X-RAY WITH COMPUTER SIMULATED LESION IN LOWER LUNG FIELD.
FIGURE 7.
I00
.
SIMULATEDGAUSSlAN TUMOR.
0
~
I.Oem
6O
I
-7
:LLZ
-6
On
0.26
0.~
FIGURE 8.
-5 .4 -3
\
/
\
/ I I
-2 l
\ \
I
\
I
O/ 1
\
, 2
!
i
3
I
I
4 5 6
FIGURE 6.
I
I
I
1
7 8 9 10 I'I /2 13 14 I~ 16 I~
PROFILES OF ACTUAL AND SIMULATED LESIONS.
151
0.76
i.O0
1.25
o
DETECTIONPERCENTAGE VS. CONTRASTRATIO FOR 0.5 CM AND 1.0 CM TUMORS.
