Semi-automatic Road Extraction from SAR Images

The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-7/W4, 2015 2015 International Workshop on Image and Data Fusion, 21 – 23 July 2015, Kona, Hawaii, USA

Semi-automatic Road Extraction from SAR Images Using EKF and PF J.Q.Zhaoa , b, J.Yanga, P.X.Lia, J.M.Luc a

The State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing of Wuhan University, 430079, No.129 Luoyu Road, Wuhan, P. R. China - [email protected], (yangj, pxli)@whu.edu.com b

c

Beijing Key Laboratory of Urban Spatial Information Engineering - [email protected]

School of Remote Sensing and Information Engineering of Wuhan University, 430079, No.129 Luoyu Road, Wuhan, P. R. China – [email protected]

KEY WORDS: SAR image; Semi-automatic road extraction; Extend Kalman Filtering(EKF); Particles Filtering(PF)

ABSTRACT: Recently, the use of linear features for processing remote sensing images has shown its importance in applications. As one of typical linear targets, road is a hot spot of remote sensing image interpretation. Since extracting road by manual processing is too expensive and time consuming, researches based on automatic and semi-automatic have become more and more popular. Such interest is motivated by the requirements for civilian and military applications, such as road maps, traffic monitoring, navigation applications, and topographic mapping. How to extract road accurately and efficiently from SAR images is a key problem. In this paper, through analyzing characteristics of road, semi-automatic road extraction based on Extend Kalman Filtering (EKF) and Particles Filtering (PF), is presented. These two methods have the same algorithm flow which is an iterative approach based on prediction and update. The specific procedure as follows: at prediction stage, we obtain prior probability density function by the prior stage and prediction model, and through prior probability density function and the new measurement, at update stage we obtain the posterior probability density function which is the optimal estimation of road system state. Both EKF and PF repeat the steps above until the extracting tasks are finished. We use these two methods to extract road respectively. The effectiveness of the proposed method is demonstrated through the experiments from Howland by UAVSAR in L-band. And through contrast experiments, we discover that extracting difference complexity of road based on different methods can improve accuracy and efficiency. The results show that EKF has better performance on road with middle noise and PF has better performance on road with high noise.

1. INTRODUCTION With the development of remote-sensing techniques, road extraction has recently become an important research topic. Since extracting road by manual processing is too expensive and time consuming, research based on automatic and semiautomatic has become more and more popular in recent years. Such interest is motivated by the requirements for civilian and military applications, such as road maps, traffic monitoring, navigation applications, and topographic mapping(Liu, J.Y, 2013). The main problem of fully automatic approach is that it needs strict hypothesis of road characteristics. However, road characteristics vary considerably with ground resolutions, road types, and density of surrounding objects, light conditions etc. Therefore, the quality of automatic extraction is not sufficient due to some missed or erroneous segments and much necessary post-edition(Lin, X.G, 2012). What is more, because the complication and variety of roads’ characters may lead to wrong results, human intervention is needed to revise the mistakes. As fully automatic methods for mapping are still far out of reach, semi-automatic methods for feature extraction that interact with a human operator are considered to be a good compromise between the calculation speed of a computer algorithm and the interpretation skills of the operator. In (Liu, J.Y, 2013), an improved particle filtering and snake model is utilized to extract roads in SAR images. In (Lin, X.G, 2012), using parallel angular texture signature get the moving direction of current road centreline point for the medium and low resolution imagery. In (Baumgartner, A, 2002), profile matching is used to extract road axes. In (Vosselman, G, 1995), method of profile matching and Kalman filtering is proposed to trace road. In

(Zhou, J, 2006), performance of road tracking based on HumanComputer Interaction and Bayesian Filtering is optimized. In (Movaghati, S, 2010), the method combined EKF with a special particle filter (PF) in order to regain the trace of the road beyond obstacles, as well as to find and follow different road branches after reaching to a road junction. This algorithm remains robust regardless of abrupt changes in the road direction and variations in the road width and intensity. In (Zhou, G. Y, 2011), combining the multiscale image analysis with polarimetric information solves the problems that relying on the image’s local information heavily makes them vulnerable to the presence of noise in the image. The study for line detection based on two-scale level has more effective results. From the surveys above, we can see that most of the previous semi-automatic methods follow this flow: manual initialization, prediction and update. Furthermore, for the diversity and complexity of roads, there is still no single method that can be successful in road extracting. In this paper, we construct the road model using the prediction model of road and model of profile matching and two semiautomatic methods of extracting road in SAR image. Through the two methods above, we can find that two methods have different performances for different scenes of road. The performance would be best if we use the advantage of EKF and PF to extract road respectively. In section 2 we will outline the characteristics of road and the motivation for using extend kalman filtering and particles filtering. Section 3 describes the result of road extraction while section 4 summarizes the whole paper.

This contribution has been peer-reviewed. doi:10.5194/isprsarchives-XL-7-W4-227-2015

227


2. METHOD

Where

2.1 Road characteristic and model In order to apply EKF and PF to trace a road in SAR images, we need to know the road structures and use these characteristics to build road model. Road has these five characteristics: geometrical, radiometric, topological, functional and contextual. Examples of these characteristics are: 1) geometrical: Roads are elongated. In the limited length, road have a maximum curvature and road width stays the same. 2) radiometric: In the limited length, the road surface is usually homogeneous and is in a good contrast with the adjacent areas. 3) topological: Roads do not stop without a reason. 4) functional: Roads connect cities. 5) contextual: Trees(or others) may occlude the road surface, but, on the other hand, an array of trees(or others) may also indicate a road.

where

zk  h( xk )  vk Where

median of the road denotes the system noise.

Where

(rk , ck )

ck

k

 k  

T

(2)

denote the row and the column numbers of

the centre of the road, respectively

k

h( xk ) denotes the measurement matrix zk and vk represent measurement and

measurement

noise respectively. Through the above two stochastic difference equation, we can build the road model. This model makes exacting road based on EKF and PF more conveniently. r

Prediction Φk

rk

xk

dt

Measurement

Φk-1

rk-1

xk-1

ck-1

c

ck

Figure 1. model of road extracting × denotes the prediction, hollow circle denote the measurement, solid circle denotes the optimal estimate 2.2 EKF and PF

We can use 4-D vector to denote kth state of road as follow:

 xk =  rk 

(4)

(1)

f ( xk 1 ) represents the simple geometrical relationship xk denotes the status of the kth point on the trace of the wk

denotes steps set by operator.

Secondly, measurements are obtained independently by the profile matching and measurement equation expressed by the following stochastic difference equation:

These characteristics above describe many features that are used by a human operator to recognise and map roads. In this paper, we use the previous three characteristics to build the road model for extracting road more efficiently, and we consider coordinates of the road median on an image as a random process and develop a proper discrete state-space model to represent its behaviour. For simplicity, we can use two stochastic difference equation to build the road model. Firstly, we can model the prediction of the system with the artificial time step as follows:

xk  f ( xk 1 )  wk

dt

is the road direction

 k is the change in the road direction or the road curvature in the kth step. Figure 1 denotes the relationship between two adjacent times of system state. Using this geometrical relationship, the system equation(1) becomes

   rk   rk  dt  sin(k   k  dt )   c    k  ck 1  dt  cos(k   k  dt )  xk =      k     k   k  dt      k   k  

EKF and PF based on Bayesian recursive problem contain two steps(prediction and update). The specific procedure is as follows: obtain prior probability density function by prediction stage, and through prior probability density function and the new measurement(update stage), we can obtain the posterior probability density function. 2.2.1 EKF When the nonlinearity of the system and/or measurement equations is not severe, (1) and (2) can be linearized around the state prediction estimate(Movaghati, S, 2010). We assume that both system and measurement noise processes wk and

vk have

Gaussian distributions. EKF can use this two steps to extract road from SAR image: prediction stage:

x k |k 1  f ( x k 1|k 1 ) (5)

T

Pk |k 1  Qk  F k Pk 1|k 1 F k

(3) where step

x k 1|k 1

denotes the optimal estimation in the (k-1)th


228


importance

Pk 1|k 1

easier sampling.

is the covariance matrix of

x k 1|k 1

Pk |k 1 is the covariance matrix of x k |k 1

Fk

is the local linearized version of the nonlinear

function follows

f ( x) ,

u

which acquired through Jacobians as

0 dt cos(u ) 1

dt sin(u )

0 0

1 0

dt 2 cos(u )   2  2 dt sin(u )   2  dt   1 

k  

Where

xki

i k

1 2 D /2 det  Rk 

1/2

(7)

Pk |k  ( I  K k H k ) Pk |k 1

k

is the update weight

di ,k

is the smallest distance of particles

2

(11)

]

is the measurement error covariance matrix

So we can use the equation(10) to estimate the optimal state at kth step and equation(10) can be simplified as follow : N

Kk  Pk|k 1H kT ( H k Pk|k 1H kT  Rk )1

i

E ( xk |k )    k xki |k 1

is Kalman gain representing as follow:

(12)

i 1

(8) The flow of the EKF and PF is showed as the following figure :

denote measurement matrix which can be SAR image

expressed by equation(9)

1 0 0 0 Hk    0 1 0 0  In the equations above, both

x k |k 1 and x k |k

Initiation

(9) Obtain prediction

Pk |k

Obtain optimal estimation by EKF or PF

Obtain measurement

are unbiased

estimation of system state by road, but covariance matrix is smaller than

exp[

di ,k

is the update sample

Rk

Hk

but

i

x k |k  x k |k 1  K k ( zk  H k x k |k 1 )

Where

has the

of

To solve estimation problem by an iterative, during the kth step of update state, the particles of samples and weights updated by the particles in the previous step and the new measurement data. Update stage:

(6)

denotes the angle of prediction in (k-1)th step

Kk

p( xk | z1:k ) which

q( x0:k | z1:k ) instead same pdf of p( xk | z1:k )

function

i

Update stage:

Where

density

xki ~ q  xk | xki 1 , zk 

 1   F k  0  0 0 

Where

To solve the problem of equation (10), we choose the

x k |k 1 denotes the prediction in the kth step Qk denotes the covariance matrix of system noise wk

Restart road extracting

Success to obtain measurement

Pk |k 1 . That means that x k |k from EKF method

Y

N Human interaction

is optimal estimation of road system state. N

2.2.2 PF Unlike the EKF, PF using the Monte Carlo numerical approximation methods to estimate points of road is different in prediction and update stage. In the PF, assuming that through a set of random samples and their corresponding weights

{x ,  }, i  1..n i k

i k

named particles, we estimate the posterior

Finish extracting road

Y

Stop road extracting

Figure 2. The flow of road extraction 3. EXPERIMENT 3.1 Experiment data

density function of the kth of road state as follows: N

p( xk | z1:k )   ki  ( xk  xki ) i 1

Where

（）is the function of Dirac ki is normalized weight.

(10)

In this paper, we use the data from Howland of America by UAVSAR in L-band for verifying the effectiveness and accuracy of our method. The original image size is 2101*1771 pixels and we choose the red region of figure 3 for extracting road.


229


(a)profile matching (b) EKF Figure 5. Contrast experiment

Figure 3. Experiment data 3.2 Experiment on EKF and PF In this image we use the three methods to extract the road which are profile matching, extend kalman filtering and particles filtering. In the figure 4, we can see that curvature of road has changed and after this, road is affected by some regions of same scattering property. In figure 4(a), the method of profile matching is more easily affected when road changes and has 8 mistaken points. In figure 4(b), the method of EKF has a good performance for this situation and using this method we have no mistaken points. In figure 4(c), the method of PF, as particle degeneracy, needs 7 human operations to finish extracting road.

(c)PF

In these two contrast experiments, we find that the profile matching is liable to the change of road and is difficult to check the false measurements, so this method fits the easy scene of road; the EKF has strict mathematical mechanism that ensures the measurements accuracy and can extract road efficiently in the road scene of middle complexity; as the particle degeneracy, the PF cannot perform well in long road extracting but perform better in the scenes of higher complexity. 4. CONCLUSION The proposed trackers, EKF and PF, are robust. The experiments show that using method of EKF has a better performance in road scene of intermediate complexity, and using method of PF has better performance in road scene of higher complexity. In the future work, we can design the threshold that can switch EKF and PF in automatic due to the reality to extract road more accurately and efficiently. ACKNOWLEDGEMENTS This research was Funded by Beijing Key Laboratory of Urban Spatial Information Engineering, NO.2014204; the National Natural Science Foundation of China under Grant 91438203; National Natural Science Foundation of China under grant 61371199; Surveying & Mapping and Geoinformation Research in the Public Interest(201412002). REFERENCES

(a)profile matching (b) EKF Figure 4. Contrast experiment

(c)PF

In the figure 5, the road scene is more complex than previous scene. In this scene, road has changed in road width, curvature of road has big changes and some regions of same scattering property as road. In figure 5(a), the method of profile matching can extract the road points in the corner, but in the region of same scattering property, this method has extracted three mistaken points and needs two human interactions. Though this method can extract accurately in this complex region, the falsely predicted points are obtained and they are affect the subsequent processing. Figure 5(b) shows the result of extracting road by EKF. This method needs three human interventions in the corner of road, but after that it can extract road points accurately. That is because EKF has strict mathematical mechanism to stop the process when false prediction points happen. So process is stopped in this corner, and accurate extract is achieved by human intervention. The result of PF is exhibited in figure 5(c). As the local particles are all used to estimate the road points, this method has good performance to extract points in the road corner and region of the same scattering property as road.

Liu, J.Y, 2013. Road extraction from SAR imagery based on an improved particle filtering and snake model. In: International Journal of Remote Sensing, Vol.34, pp: 8199-8214. Lin, X.G, 2012. Semi-Automatic Road Tracking using Parallel Angular Texture Signature. In: Intelligent Automation and Soft Computing, Vol.18, pp: 1009-1021. Baumgartner, A, 2002. Efficient methods and interfaces for road tracking. In: International Archives of Photogrammetry Remote Sensing and Spatial Information Sciences, Vol.34, pp: 28-31. Vosselman, G, 1995. Road tracing by profile matching and Kalman filtering. In: Automatic Extraction of Man-Made Objects from Aerial and Space Images, pp: 265-274. Zhou, J, 2006. Road Tracking in Aerial Images Based on Human-Computer Interaction and Bayesian Filtering. In: Isprs Journal of Photogrammetry and Remote Sensing, Vol.61, pp: 108-124. Movaghati, S, 2010. Road extraction from satellite images using particle filtering and extended Kalman filtering. In: Geoscience and Remote Sensing, IEEE Transactions, Vol.48, pp: 28072817. Zhou, G. Y, 2011. Linear Feature Detection in Polarimetric SAR Images. In: IEEE Transactions on Geoscience and Remote Sensing, Vol.49, pp: 1453-1463.


230