urban aerial images is proposed. First, a new imaging model of shadows is presented, which indicates that hue values of shadowed image areas are larger than ...
Proceedings of the 5* World Congress on Intelligent Control and Automation, June 15-19, 2004, Hangzhou, P.R. China
Detection of and Compensation for Shadows in Colored Urban Aerial Images* Jianjun Huang and Weixin Xie College oflnformation Engineering Shenzhen Universiq Shenzhen, Guangdong Province, China {huangjin & wxxie}@szu,edu.cn
Liang Tang School of Electronic Engineering Xidian Universiry X i b n , Shaanxi Province, China Tang-liangcn@yahoo .corn.cn
,
Abstract-A new method for shadow detection in colored urban aerial images is proposed. First, a new imaging model of shadows is presented, which indicates that hue values of shadowed image areas are larger than those of these areas nonshadowed. Based on this model, a thresholding technique is employed to detect shadowed areas. After detection, the Retinex technique is applied to shadowed and nonshadowed areas individually to compensate for shadows. Experiment results show the effectiveness of the proposed method. Index Terms-colored urban aerial image, new shadow model, thresholding technique, shadow defection, shadow compensation
I. INTRODUCTION
As the growing demand for geographical information, the auto processing of urban aerial images is more and more concerned. In these works, shadow detection is significant since shadow regions provide not only obvious evidence to objects higher than their surrounding background, but also obstructions fur objects recognition. Two kinds of different detection approaches have been followed [2]: model based and property based. The first kind uses a prior knowledge of the illumination and the 3D geometry of the scene being imaged to calculate positions of shadows, but that the knowledge is not always available leads to the rare use of this kind of methods. The second kind uses the hue, the intensity and the geometry structure of shadows in images for the detection of shadows, and is generally used [3]. In the later kind, shadow areas are believed to be of low intensity than surrounding areas. For gray images, shadows could he detected by appropriate intensity threshold. While for color images, there exist nonshadowed areas with low intensity (e.g. areas with darker colors), threshold techniques may misclassify these areas to he shadows. To overcome this drawback, other methods, such as RGB ratios and photometric color invariants based ones are proposed [1,3]. RGB ratios based methods are based on the fact that for a uniform casting surface, intensity values in R,G, and B channels decrease with same proportion after shadowed, so threshold techniques could he applied to these ratio images to detect shadows [I]. Photometric color invariant based methods detect shadows by subtracting thresholding results of invariants, which is unaffected by shadows, from those of original images [3].
These invariants are invariant on the condition of NIR (Neutral Interface Reflection) [ 5 ] . However, for colored urban aerial images, neither uniform casting surface nor NIR is always satisfied, thus make it difficult for existing methods to detect shadows in these images. In this paper, we develop a new imaging model for shadows in these images and present a new method for shadow detection, which does not require a prior knowledge or satisfy any of the conditions mentioned above, and thus is more practical than existing ones. To compensate for shadows, the Retinex [6] technique is applied to both shadowed and nonshadowed areas separately. Theoretic analyses and experiment results are presented to demonstrate the effectiveness of the proposed method. 11. IMAGING MODELOF SHADOWS According to Phong’s illumination model[4], an observed image of a small patch of a given surface consists of three parts: ambient, diffusion, and specular part. The ambient part is induced by the illumination coming from the environment, while the diffusion and specular part are the diffusion and specular reflection of the illumination from a light source by the surface patch, respectively. Shadows occur when objects occlude direct light from a source of illumination, i.e. shadowed area in an image consists of only ambient part. For urban images, since most urban areas are matte and dull, the specular part could be ignored. Thus the change of the aerial image of an urban area when shadowed and not shadowed is just the diffusion part, that is (1)
AIB = !Kd(l)I,(l)SB(l)cos Bdl
where 2. is the wavelength of the light. hl, , AIG , and AI, denote changes in R, G, and B channels, respectively. S , SG, and S, indicate the three separate color filters of the camera. K , ( l ) is the elbedo of the area surface, I, and 0 are the light radiance and the incident angle of the sun, respectively. For a light source like the sun, the integrated white condition
This work is partially supported by CNSF Grant #60172066 to 1. Huang.
3098
0-7803-8273-0/04/$20.0002004 IEEE
1
A I , = J K ~ ( / E ) I , ( / E ) s ~ ( / E ) c o se d a A I G = ~ K d ( ~ ) I , ( 2 . ) S , ( / Z ) c o sB d l ,
II,(A)S,(A)dA = lIi(A)S,(A)dA = s I I ( A ) S B ( A ) d A , ( 2 ) usually holds [ 5 ] . In existing literatures (e.g. those about photometric color invariants), K d ( A ) is considered to be a constant independent of the wavelength, thus AIR =AI, = A I B , (3) which is the NIR model. But according to the theory of electromagnetic wave, K d ( A ) is positive proportional to the wavelength, and wavelengths of R, G and B channels are in a decreasing order, i.e.
JK.
(w, (AP, (A)dA > JK. m i ,( A I S , ( A w > j K d (mi (w, (wa
'
(4)
So that A I R > ill, > A I B .
(5)
Namely, the intensity decrease in shadow regions is larger in the red channel than that in the green channel, and that in the green channel is larger than that in the blue channel. Since differences between these three quantities are rather small, they are ignored in existing literatures. But urban aerial images consist mainly of regions with low saturation or intensity imaged from surfaces of concrete, asphalt, and greenish grassland, so their differences could not be neglected again. According to the relationship between RGB and HIS, the hue value is given by .
H=arctan(JS(G-B)/[(R-G)+(R-B)]}. (6) Suppose a pixel in the shadow region before shadowed has a color of (R, G, B), its hue value is H(R, G, E) given in (6). After shadowed, its color value becomes ( R - A R , G - A G , E - A B ) , and its huevalue becomes H'= arctan
&(G- B)-(AG -a) ( R -G)+(R- B)-(2AR -AG -AB)
j.
(7)
Since in urban aerial images, shadowed surfaces consist mainly of concrete, asphalt and greenish grassland, which have intensity values in G channel greater or equal than those in B channels, either shadowed or nonshadowed, i.e., numerators of independent variables of arctan function in (6) and (7) are greater than 0. Consider the relationship between illR,Al,,illBin ( 5 ) and the monotonic property of arctan function, we have that H ' is greater than H .
Fig. 1 shows a colored urban aerial image with shadowed regions and its hue image, from the hue image we find that hue values of shadowed regions is much higher than those of adjacent regions from same surfaces, i.e. hue values increased in shadowed regions. This increase could be used to detect shadows. 111. SHADOW DETECTION A m COMPENSATION A. Shadow detection Followed from the privious analysis, we h o w that shadows have large hue values. But in colored urban aerial images, there are objects not shadowed with large hue values, such as blue buildings, rivers, grasslands, etc., which should be distinguished out from shadows. For bluish objects, they always have high intensity values in the B channel, while shadows have low ones. For greenish objects, they have large differences between G and B channels, but shadows do not. So shadows could be described as regions with large hue values, low values in B channel and small difference between G and B channels. To detect them, the histogram threshold technique could be used. Suppose A(i,j) denotes a colored urban aerial image with shadows in it, we can detect them as follow: Stepl, The HSI transform is first used to transform it from RGB fom1 to HSI form. Then a threshold TI is computed to separate the hue component H ( i j ) of A to get the candidate shadow region Sc'={(i,j)1~(i,j)>~1. (8) Here, S, may include shadows, bluish objects and greenish objects. Step2 To remove bluish objects from S,, another threshold T, is chosen and applied to the B channel of A , and we get S',= S, n{(i, j ) 1 B(i,j ) < T,} . (9) Now, S', still have greenish objects to be eliminated. Step3 For the removal of greenish objects, we use a third threshold T, to get S", = F C n { ( i , j )1 G(i,j ) - B(i, j ) < T,} , (IO) which ilIe the shadows. Step4 Finally, in order to eliminate noise points and small patches, and to fill up small holes in S", , morphological operations are applied to it.
B. Shadow compensation The existence of shadows gives a worse perception to human eyes, but in aerial images there are always shadows. To compensate for them, we apply the Retinex technique, which is originally used for the removal of discrepancy existing between images and direct observation of real scenes. The Single-Scale Retinex is given by R, (x. Y) = bdl,(x.Y)/ f j(1,Y)I = ~ o g { [ s j ( x , ~ ) r , ( x , ~ ) ~ / [ S , ( x , ~(11) )~.(x,~)~}, = [log rj(1,Y)14 (x.Y)I
(a) a colored urban aerial image (b) the hue image of the image in (a)
Fig.1. Example of the increase of hue values in shadowed regions
3099
where 12(x,y)is the ith channel of an image, S,(x,y) is the spatial distribution of the source illumination, ri(x,y) is the distribution of scene reflectance, and Ri(x,y) is the associated Retinex output, the bars denote the spatially weighted average value by the normalized surround function
J p ( x , y ) d x d y= 1 (12) The approximation holds if Si(x,y) is constant or changes slowly on the definition area of F(x,y). But for aerial images with shadows, this constraint is violated since shadowed areas and nonshadowed areas receive very different illumination. For this reason,. we apply the Retinex separately to shadowed areas and nonshadowed areas. After R,(x,y) for each area is computed, they are combined together to form the compensated image.
(a) an image with self shadows (b) detected shadow regions Fig.3 Shadow detection result for an image with self-shadow
IV. EXPERIMENTS Fig.2 shows some results of the proposed method applied to Fig.la. Fig.2a is the image of S, after the first step, bright area are candidate shadow regions. By comparing with the original image, we find that there still have some greenish and bluish objects, tow pools and a grassland, and a building with blue roof. The bluish objects have hue values greater than shadows and greenish objects, so a threshold in the B channel can remove them. The greenish objects have greater differences in .G and B channels than shadow areas, a threshold in the G-B image can eliminate them. Fig.2b shows the result of candidate shadow areas S", by elimination of these two types of objects. Greenish, and bluish objects are nearly completely removed. This result still has small patches and holes, thus the morphological operation of open is applied to small patches and close to small holes, and the result is shown in Fig.2~.The result is most promising. Shadows in Fig.la are mainly cast by other objects (called cast shadow). But there are also shadows caused by objects themselves (called self-shadow). Fig3 shows this case. Fig.3a is the original image with self-shadows, Fig.3b is the detection result, and shadows are well detected. Fig.4 Shows the shadow compensation results for Fig.la and Fig.3a, Fig.4a is the result of Fig.la, while Fig.4b for Fig.3a, the shadows are compensated well, they are nearly all removed, especially at the center part of the image in Fig.4a. The main drawback of the proposed method is the distortion
(a) result forFig.la (b) result for Fig.3a Fi14. The shadow comwnsatim results for W Oaerial imaees
of colors, which is still an open problem for the Retinex technique. V. CONCLUSION In this paper, we analyze the Phong's illumination model and present a new model for shadows in colored urban aerial images. On the basis of this model, the histogram threshold technique is employed to detect them. After the detection, the Retinex technique is applied to both shadowed and nonshadowed areas separately to achieve the compensation for shadows. Experiment results show that the proposed method is effective. REFERENCES [I] K. Bamard, and G Finlayson. Shadow Identification Using Color Ratios Proc. IS&T/SID Eighth Color Imaging Conference: Coloi Science, Systems and Applications. 2000: 97-101 [2] E. Salvador, A. Cavallaro, and T. Ebrahimi. Shadow Identification and Classification Using Invariant Calor Models, Proc. IEEE Signal Processing Society lntemational Conference on Acoustics, Speech, and Signal Processing. May.2001(3): 1545-1548 [3] C. Jiang, and M. 0. Ward. Shadow Identification. Proc. IEEE Conference on Computer Vision and Panem Recognition. 1992: 606-612 [4] B. T. Phong. Illumination for Computer Generated Picmres. Communications oftheACM, 1975, lR(6): 311-317 [5] T. Geversw, and A. W. M. Smeulders. Calor-Based Object Recognition. Pattern Recognition, 1999, 32: 4 5 3 4 6 4 [6] E. Land, An alternative technique for the computation ofdesignator in the retinex theory of color vision. In Proc. Nat. Acad. Sci., ~01.83: 3078-3080,1986
(a) image by (b) image after bluish (c) detected shadow thresholding the hue and greenish objects regions image removed Fig. 2 Final m d middle results of the proposed method applied to Fig. l a
3100
