Estimation of multiple motions: regularization and performance ...

5 downloads 0 Views 237KB Size Report
one frame of the original Mona Lisa sequence. We used the estimated motion vectors for nulling one of the two motions out of the sequence by computing α(u)f ...
I I  V C  P 2003, B V, T. R H, A G. T, T E, ., P  SPIE, . 5022, . 75–86, M 2003.

Estimation of multiple motions: regularization and performance evaluation Ingo Stukea , Til Aacha , Cicero Motab , and Erhardt Barthc a

b

Institute for Signal Processing, University of L¨ ubeck Institute for Applied Physics, J. W. Goethe-University Frankfurt c Institute for Neuro- and Bioinformatics, University of L¨ ubeck ABSTRACT

This paper deals with the problem of estimating multiple transparent motions that can occur in computer vision applications, e.g. in the case of semi-transparencies and occlusions, and also in medical imaging when different layers of tissue move independently. Methods based on the known optical-flow equation for two motions are extended in three ways. Firstly, we include a regularization term to cope with sparse flow fields. We obtain an Euler-Lagrange system of differential equations that becomes linear due to the use of the mixed motion parameters. The system of equations is solved for the mixed-motion parameters in analogy to the case of only one motion. To extract the motion parameters, the velocity vectors are treated as complex numbers and are obtained as the roots of a complex polynomial of a degree that is equal to the number of overlaid motions. Secondly, we extend a Fourier-Transform based method proposed by Vernon such as to obtain analytic solutions for more than two motions. Thirdly, we not only solve for the overlaid motions but also separate the moving layers. Performance is demonstrated by using synthetic and real sequences.

1. INTRODUCTION This paper addresses the problem of estimating multiple transparent motions that can occur in computervision applications, e.g. in case of semi-transparencies and occlusions, and also in medical x-ray projections imaging, when different layers of tissue move independently. An overview of the problem of multiple motions has been given in [1]. To our knowledge, the problem of two motions has been first solved in [2] by the use of spatio-temporal Gabor filters and fourth-order moments derived from these filters. An alternative solution that is also based on the frequency domain is given in [3], where a nonlinear system of four equations is solved to estimate the phase change and from there two transparent motions. In general, frequency-based methods suffer from requiring large local windows. Others have introduced the useful and intuitive notion of ’layers’. 4 As an important extension of the methods mentioned above, we have provided analytic solutions for up to four transparent motions.5 Our approach also delivers numerical solutions for more than four motions. Here we first extend the approach to include regularization. Vernon3 proposed a different framework for multiple transparent motions based on the phase of the Fourier components. This solution, however, is limited to only two motions. We solve the equations introduced by Vernon for the more general case of an arbitrary number of motions. We also identify some basic problems of Vernon’s approach to the separation of transparent motion layers. Further author information: (Send correspondence to E.B.) I.S.: [email protected], T.A.: [email protected], C.M.: [email protected] E.B.: [email protected], University of L¨ ubeck, Ratzeburger Allee 160, 23538 L¨ ubeck, Germany

c

2003 Society of Photo-Optical Instrumentation Engineers. This paper was published in the Proceedings of SPIE (see above) and is made available as an electronic reprint with permission of SPIE. One print or electronic copy may be made for personal use only. Systematic or multiple reproduction, distribution to multiple locations via electronic or other means, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper are prohibited.

E   :    

2

2. THE THEORY OF MULTIPLE TRANSPARENT MOTIONS 2.1. Spatial model for two transparent motions Suppose that an image sequence f (x, y, t) is the overlaid superposition of two other sequences g 1 , g2 moving with constant velocities u = (ux , uy )T and v = (vx , vy )T . The resulting f is then described as f (x, y, t) = g1 (x − ux t, y − uy t) + g2 (x − vx t, y − vy t).

(1)

The task is to determine v and u given f . To do so, we use the optical-flow equation introduced by Shizawa and Mase2 : α(u)α(v)f = 0.

(2)

Note that the equation involves the concatenated directional derivatives along u and v. After expanding the above equation we obtain the following expression: α(u)α(v)f = fxx ux vx + fyy uy vy + fxy (ux vy + uy vx ) + fxt (ux + vx ) + fyt (uy + vy ) + ftt = 0.

(3)

As in [5] we use the following notation: cxx = ux vx cxy = ux vy + uy vx

cyy = uy vy cxt = ux + vx

cyt = uy + vy

ctt = 1.

(4)

With the c’s denoting the mixed-motion parameters. Eq. (2) then becomes: α(u)α(v)f = fxx cxx + fyy cyy + fxy cxy + fxt cxt + fyt cyt + ftt ctt = 0.

(5)

As we shall see, this notation leads to a linear formulation of the problem. The mixed-motion parameters can be separated by using the novel method described by Mota et.al. 5 The velocity vectors are thereby treated as complex numbers: u = ux + juy , v = vx + jvy .

(6)

These complex numbers are related to the mixed motion parameters c by the following equations: uv = A0 = cxx − cyy + jcxy u + v = A1 = cxt + jcyt .

(7)

Note that A0 and A1 are homogeneous symmetric functions in u and v and, by Vieta’s theorem, the coefficients of the complex polynomial Q(z) = (z − u)(z − v) = z 2 − A1 z + A0

(8)

that has the complex roots u and v. These roots can be obtained analytically (even for the case of up to four motions5 ). Thus the main steps in solving for multiple transparent motions are to (i) solve the linear system for the c’s (ii) find the roots of the complex polynomial Q(z), and (iii) take the real parts of the z’s as x- and the imaginary parts as y-components of the motion vectors.

I. S, T. A, C. M,  E. B/P. SPIE, 5022:75–86, M 2004.

E   :    

3

2.2. Spatial model for multiple transparent motions In this section we analyze the case of more than two transparent motions. A number N of transparent motions are described by the following equation: f (x, t) = g1 (x − tv1 ) + g2 (x − tv2 ) + · · · + gN (x − tvN ).

(9)

In this case, the optical-flow equation is given by α(v1 ) · · · α(vN )f =

X

cI fI = 0,

(10)

I

with the notation I = I1 , . . . , IM , M = (N + 1)(N + 2)/2. Ii are ordered sequences with elements (x, y, t). For example in Eq. (4) I1 = xx, I4 = xt, and I6 = tt. As for the case of two motions, multiple motions can be separated by solving for the roots of an N -degree polynomial.5

3. REGULARIZATION FOR MULTIPLE MOTIONS 3.1. Regularization for two transparent motions The regularization procedure is defined in analogy to the case of only one motion. 6 As with one motion we have only one equation for the motion parameters, but now the number of unknowns is five. We therefore need four more equations. We employ the calculus of variation and define, in analogy to the method used in by Horn and Schunck,6 the following regularization term: N = (∂x cxx )2 + (∂y cxx )2 + (∂x cyy )2 + (∂y cyy )2 + (∂x cxy )2 + (∂y cxy )2 + (∂x cxt )2 + (∂y cxt )2 + (∂x cyt )2 + (∂y cyt )2 . (11) We thus obtain the parameters c as the values that minimize the above term together with the squared optical-flow term (2), i.e.: ZZ 2 α(u)α(v)f + λ2 N dΩ, (12)

λ is the regularization parameter and Ω the whole image plane over which we integrate. Note that, at this stage, we work on finding the c’s and not the motion-vector components. This has the great advantage that we obtain an Euler-Lagrange system of differential equations that is linear. As we shall see, this would not be the case, when working directly on the motion vectors themselves. Note that if the velocities u and v are smooth, the parameters c will also be smooth. The five Euler-Lagrange equations that we obtain are the following: 2 λ2 4cxx = fxx cxx + fxx fyy cyy + fxx fxy cxy + fxx fxt cxt + fxx fyt cyt + fxx ftt 2 λ2 4cyy = fyy fxx cxx + fyy cyy + fyy fxy cxy + fyy fxt cxt + fyy fyt cyt + fyy ftt 2 λ2 4cxy = fxy fxx cxx + fxy fyy cyy + fxy cxy + fxy fxt cxt + fxy fyt cyt + fxy ftt 2

λ 4cxt = fxt fxx cxx + fxt fyy cyy + fxt fxy cxy +

2 fxt cxt

(13)

+ fxt fyt cyt + fxt ftt

2

2 λ 4cyt = fyt fxx cxx + fyt fyy cyy + fyt fxy cxy + fyt fxt cxt + fyt cyt + fyt ftt .

3.1.1. Linear and nonlinear formulations of the problem As noted above, the system (13) is linear in the c’s. Let us consider the following example to illustrate that this would not be the case with the motion parameters themselves. To obtain the Euler-Lagrange differential equations, one must differentiate the functional to be minimized with respect to the unknown

I. S, T. A, C. M,  E. B/P. SPIE, 5022:75–86, M 2004.

4

E   :    

variables. If we differentiate only the expression of the squared optical-flow equation with respect to v x , we obtain 2  ∂  α(u)α(v)f = 2 fxx ux vx + fyy uy vy + fxy (ux vy + uy vx ) ∂vx + fxt (ux + vx ) + fyt (uy + vy ) + ftt



 fxx ux + fxy uy + fxt , (14)

i.e., we obtain an equation that is nonlinear in vx , vy , ux and uy . Therefore we note that it is indeed the introduction of the mixed motion parameters c that leads to a linear formulation of the problem. 3.1.2. Update rule In analogy to the case of one motion we obtain the following update rules for the system (13): P D P = cˆlyy − fyy D P = cˆlxy − fxy D P = cˆlxt − fxt D P = cˆlyt − fyt , D

cl+1 ˆlxx − fxx xx = c cl+1 yy cl+1 xy cl+1 xt cl+1 yt

(15)

where P = fxx cˆlxx + fyy cˆlyy + fxy cˆlxy + fxt cˆlxt + fyt cˆlyt + ftt 2 2 2 2 2 D = λ2 + fxx + fyy + fxy + fxt + fyt .

(16)

The iteration will deliver the values of the mixed motion parameters c. From these parameters we still need to extract the velocity vectors u and v as described in Section 2.1. 3.1.3. Boundary treatment We have chosen to extend the image by copying the boundary pixels into an extended margin of size one. Such, the first-order derivatives will be zero outside the boundary of the original image thus minimizing boundary effects.

3.2. Regularization of multiple motions In case of more than two motions, the functional to be minimized becomes: ZZ  X  2 X  (∂x cI )2 + (∂y cI )2 dΩ cI f + λ 2 I

(17)

I\IM

and the Euler-Lagrange differential equations are X  cI fI fIi = λ2 4cIi ,

i = 1, · · · , M − 1.

(18)

I

Discretization with 4cIi = cˆIi − cIi leads to  X cI fI fIi + λ2 cIi = λ2 cˆIi − fIi fIM , I\IM

I. S, T. A, C. M,  E. B/P. SPIE, 5022:75–86, M 2004.

i = 1, · · · , M − 1.

(19)

5

E   :    

It is now straightforward to show that Equation (19) leads to the following update rule: cl+1 ˆlIi − fIi Ii = c where P =

X

cˆlI fI

P , D and

i = 1, · · · , M − 1, D = λ2 +

I

X

(20)

fI2

(21)

I\IM

Since the system is positive definite, we know that even for more than two motions equation (20) is the only possible solution of the system (19). For up to four motions, the motion parameters can be obtained from the mixed-motion parameters analytically as described above since the increased number of motions will just increase the order of the polynomial. For more than four motions, the roots of the polynomial Q(z) can be found by numerical methods.

4. SEPARATION OF MOTION LAYERS In Section 3 we have shown how to estimate the motion fields in case of transparent motions. We will now show how to use them to separate the spectra of the overlaid images. Firstly, we discretize time to tk = k∆t, k = 0, . . . and transform the discretized motion model in (9), i.e., ftk (x) = f (x, tk ) = g1 (x − tk v1 ) + g2 (x − tk v2 ) + · · · + gN (x − tk vN ) (22) to the frequency domain by use of the Fourier transform. Such, we obtain the following system of equations Ftk (ω) = φk1 G1 (ω) + φk2 G2 (ω) + · · · + φkN GN (ω),

k = 0, . . . ,

(23)

where φn = ejω·vn ∆t , n = 1, . . . , N , ω = (ωx , ωy ) is the frequency variable and uppercase letters are the Fourier transform of the respective lower case letters, e.g., Ftk is the Fourier transform of ftk . Secondly, we note that the first N equations in the system (23) can be     1 1 ··· 1 F t0  F t1   φ 1  φ2 ··· φN      =   .. .. ..     . . . N −1 N −1 N −1 FtN −1 φ1 φ2 · · · φN

written in matrix form as  G1 G2   (24) ..  .  GN

or in short notation Ft0 = BN G, where Ft0 = (Ft0 , . . . , FtN −1 ), G = (G1 , . . . , GN ) and BN is the above matrix containing the phase terms. BN is a Vandermonde matrix that is invertible if its entries φ1 , . . . , φN are all distinct. The separation is, therefore, possible by using N successive frames of the image sequence by inversion of BN , i.e., G = B−1 (25) N F t0 . Note, however, that the separation is not possible at all frequencies. The problematic frequencies are those where two or more phase values are identical because the rank of the matrix B N is then reduced. This is an important observation because it defines the support where multiple phases can occur by the following equation: φm = φn ⇐⇒ ej(vm −vn )·ω∆t = 1 ⇐⇒ (vm − vn ) · ω = 2kπ, k = 0, . . . (26) On the above defined lines, the Fourier transforms of at least two transparent layers cannot  be separated. N If the difference vectors vm − vn , m, n = 1, . . . , N are not aligned, we have a number of lines with 2 inseparable frequency components going through the origin of frequency space. A possible solution would be to interpolate the values on these lines from the neighboring frequency values of the separated layers.

I. S, T. A, C. M,  E. B/P. SPIE, 5022:75–86, M 2004.

6

E   :    

5. PHASE-BASED MODEL FOR MULTIPLE TRANSPARENT MOTIONS Frequency-domain based approaches to transparent motions are based on the observation that motion induces a phase shift. This relationship has been used for the estimation of only one motion by Jepson and Fleet.7 For transparent motions, this translates to Equation (23) that we repeat here as an aid: Ftk (ω) = φk1 G1 (ω) + φk2 G2 (ω) + · · · + φkN GN (ω),

k = 0, . . .

(27)

Equation (27) has been analytically solved by Vernon3 to obtain the phase components and to separate the layers for the simplest case of only two motions. In the following we will solve (27) for the general case of N motions. Note that we already have the solution for the Fourier transform of the layers in terms of the phase shifts, see Equation (25). To obtain the phase shifts, we first simplify notation by setting Φ k = (φk1 , . . . , φkN ) and G = (G1 , . . . , GN ). We then obtain the following expressions for the above system : Ftk = Φk · G,

k = 0, . . .

(28)

Our goal now is to obtain the phase-components vector Φ1 = (φ1 , . . . , φN ) by cancellation of the unknown Fourier-transforms vector G of the image layers in the system above. Given N + 1 images of the sequence, we define the polynomial p(z) = (z − φ1 ) · · · (z − φN ) = z N + a1 z N −1 + · · · + aN

(29)

with unknown coefficients a1 , . . . , aN . Now the phase terms φ1 , . . . , φN are the roots of p(z), i.e., p(φn ) = 0, for n = 1, . . . , N and we observe that Φm+N + a1 Φm+N −1 + · · · + aN Φm = (. . . , φnm+N + a1 φnm+N −1 + · · · + aN φm n ,...) = N N −1 m m (. . . , φm + · · · + aN ), . . . ) = (φm n (φn + a1 φn 1 p(φ1 ), . . . , φn p(φn ), . . . , φN p(φN )) = 0. (30)

Therefore by inserting (30) in (28) we obtain Ftm+N + a1 Ftm+N −1 + · · · + aN Ftm = (Φm+N + a1 Φm+N −1 + · · · + aN Φm ) · G = 0 · G = 0

(31)

and Ftm+N = −aN Ftm − · · · − a1 Ftm+N −1

m = 0, . . .

(32)

We can use the first N of the above equations to solve for the unknown vector y = (a 1 , . . . , aN ) of coefficients of p(z) by solving the linear system 

  F tN F t0  FtN +1   F t1     ..  = −  ..  .   . Ft2N −1 FtN −1

F t1 F t2 .. .

··· ···

F tN

···

  FtN −1 aN   F tN   aN −1  ..   ..  . .  . 

Ft2N −2

(33)

a1

With FN for the left hand side vector and AN for the matrix in the above system, we obtain the short notation FN = −AN y. After solving the above system for a1 , . . . , aN , we obtain the unknown phase changes φ1 , . . . , φN as the roots of p(z).

I. S, T. A, C. M,  E. B/P. SPIE, 5022:75–86, M 2004.

7

E   :    

5.1. The case of a singular matrix AN The Equations (28) for Φ1 = (φ1 , . . . , φN ) can be solved as long as the matrix AN in Equation (33) is not singular. However, as we will show, it is still possible to estimate the phase terms even with the singular matrix. To understand why, we will discuss all the cases in which AN is singular. First note that the matrix AN nicely factors as 

  AN =  

F t0 F t1 .. .

F t1 F t2 .. .

··· ···

FtN −1

F tN

···

  G1 FtN −1 0 F tN    ..  = BN  ..  . .  0

Ft2N −2

0 G2 .. .

··· ··· .. .

0

···

 0 0   T  BN  GN

(34)

and since BN is a Vandermonde matrix with entries φ1 , . . . , φN , its determinant is Y det AN = G1 · · · GN (φm − φn )2 .

(35)

m>n

It follows that there are only two non-exclusive situations where the matrix A N can become singular: 1. The Fourier transform of at least one layer vanishes at the frequency ω for which G m (ω) = 0; 2. Phase shifts are equal, that is, φm = φn for at least one pair m, n. If two phase shifts are equal. e.g. φN = φN −1 , the number of terms in the sum of Equation (27) reduces to ˜ N −1 , Ftk = φk1 G1 + φk2 G2 + · · · + φkN −1 (GN −1 + GN ) = φk1 G1 + φk2 G2 + · · · + φkN −1 G

k = 0, . . . ,

(36)

˜ N −1 = GN −1 + GN . Therefore we can suppose without loss of generality that where G ˜R, Ftk = φk1 G˜1 + φk2 G˜2 + · · · + φkR G

k = 0, . . .

(37)

˜ R being all different from zero. Note with the above phase shifts φ1 , . . . , φR being all distinct and G˜1 , . . . , G that a phase term could disappear from the equation due to the fact that the coefficients can sum to zero. Since the left hand side of Equation (37) does not change due to factorization and reordering of the terms, we can rewrite Equation (34) as

AN



F t0  F t1  = .  .. FtN −1

F t1 F t2 .. .

··· ···

F tN

···

˜ G1  0 FtN −1   .. F tN   .   = B  .. N  .  0  . Ft2N −2  .. 0

0 ˜2 G .. . 0 .. . 0

··· ··· ··· ···

0 0 .. . ˜R G .. . 0

··· ··· ··· ···

 0 0  ..  .  BTN . 0  ..  .

(38)

0

Therefore rank(AN ) = rank(AR ) = R and we can compute the phases shifts φ1 , . . . , φR by resolving the system FR = −AR y. (39)

I. S, T. A, C. M,  E. B/P. SPIE, 5022:75–86, M 2004.

E   :    

8

5.1.1. Examples for only two motions We now illustrate the above results for the special case of only two overlaid motions. 1. Rank(A2 ) = 2: the estimation of the phase φ1 , φ2 is possible. First we solve for a1 , a2 the system Ft2 = −A2 y, that is      a2 F t0 F t1 F t2 (40) =− a1 F t1 F t2 F t3 and obtain a1 =

F t1 F t2 − F t0 F t3 Ft0 Ft2 − Ft21

a2 =

Ft21 − Ft0 Ft2 Ft0 Ft2 − Ft21

(41)

The phases φ1 , φ2 are then computed as the roots of p(z) = z 2 + a1 z + a2

(42)

2. Rank(A2 ) = 1: the possible cases are G1 = 0, G2 6= 0; G1 6= 0, G2 = 0 or φ1 = φ2 , G1 + G2 6= 0 and we can compute the double phase or one of the two distinct phases from Ft1 = Ft0 φ.

(43)

3. Rank(A2 ) = 0: in this case G1 = G2 = 0 or φ1 = φ2 , G1 +G2 = 0 and all equations in (27) degenerate to Ftk = 0, k = 0, . . . (44)

6. RESULTS 6.1. Derivatives and filters We computed the derivatives in the spatial domain. The first-order derivatives are Gaussian derivative filter with a sigma of one and size of 7 pixels. The filters have been applied twice to obtain the second-order derivatives.

6.2. Motion estimation We tested the regularized motion estimation algorithm on real and synthetic images for the case of two motion. In Figure 1 we analyze robustness against noise. We simulated natural images by generating 1/f spectral noise sequences to which we added Gaussian distributed noise at different signal-to-noise ratios (SNR). The errors of the estimated motion vectors are shown as a function of the SNR and the number of iterations. In the first row of Figure 1 we show the absolute mean angular errors and the angular standard derivation (STD) of the estimated motion vectors. The second row displays the errors in relative magnitude and the corresponding STD. Note that we obtain very good result with a SNR of up to 20dB. In all examples the regularization parameter λ had a value of one.

I. S, T. A, C. M,  E. B/P. SPIE, 5022:75–86, M 2004.

9

E   :    

angular error in degree

angular STD in degree

10

6

8

5 4

6 3 4 2 2 0 10

1 20

30 SNR in dB

40

50

relative length error in percent 20

15

0 10 100 150 200 300 450 10 600 8

20

30 SNR in dB

40

50

relative length STD in percent

6 10 4 5

0 10

2

20

30 SNR in dB

40

0 10

50

20

30 SNR in dB

40

50

(a) Figure 1. Performance as a function of noise level and the number of iterations. For details see text.

6.3. Separation of layers Figure 2 shows an example were the layer separation works very well. In (a) and (b) the original layers are shown, which are then overlaid in (c). In the image sequence the layers were moving with velocities (1,0) and (0,1), respectively. In (d) we show the separation result for the layer (a) and in (e) the result for layer (b). Note that both layers are well detected and that the interpolation errors are not visible in the reconstructed images. However, the errors become visible when subtracting the original in (a) from the separated layer in (d) as shown in (f). In Figure 3, however, the interpolation errors are evident. In (a) we show the first image of the sequence that is a clown image moving with velocity (1,0) and a cameraman image moving with velocity (0,1). Note that both separated images (c) and (d) are overlaid with an oriented pattern. After subtracting the original image (b) from the separation layer (e) we obtained the difference image (f) that shows the oriented error patterns. Obviously, the visibility of the errors is due to missing texture. However, the errors could be reduces with a better interpolation method. We have computed the missing frequency values as the average of the neighboring values. In Figure 4 we show the results of motion estimation with a realistic sequence and demonstrate the performance of our layer-separation algorithm for reflection removal. In (a) we show one frame of the original Mona Lisa sequence. We used the estimated motion vectors for nulling one of the two motions out of the sequence by computing α(u)f and α(v)f . The results are shown in (c) and (d) respectively. In (b) we show the Mona Lisa layer only as the result of the separation algorithm. Note that the reflections are removed.

I. S, T. A, C. M,  E. B/P. SPIE, 5022:75–86, M 2004.

10

E   :    

(a)

(b)

(c)

(d)

(e)

(f)

Figure 2. Separation of layers for high-textured images. For details see text.

7. DISCUSSION We have presented a novel method for the estimation of multiple transparent motions that is based on an iterative solution of a linear system of equations. The system is obtained by introducing a regularization term for the mixed-motion parameters. The motion-vector components are then obtained from the mixedmotion parameters by solving for the roots of a complex polynomial. Alternative regularization procedures could be used, since we have succeeded to linearize the problem of determining multiple overlaid motions. Due to the linearization, it is possible to incorporate regularization and deal with more that two motions. We have presented good results on synthetic and real images. However, the way we have computed the partial derivatives still needs to be optimized and filters other than derivatives could be used, as outlined in [5], to increase robustness. Possible extensions to cope with occluded motions have been proposed in [8]. In addition to an regularized estimation of multiple motions we have generalized a method based on the phase in the Fourier transform. We have used this method to separate the motion layers and have shown some inherent limits of the method. Further research will deal with alternative methods for the separation of the motion layers that avoid these problems.

I. S, T. A, C. M,  E. B/P. SPIE, 5022:75–86, M 2004.

11

E   :    

(a)

(b)

(c)

(d)

(e)

(f)

Figure 3. Separation of layers for low-textured images. For details see text.

ACKNOWLEDGMENTS Work is supported by the Deutsche Forschungsgemeinschaft under Ba 1176/7-1.

REFERENCES 1. M. J. Black and P. Anandan, “The robust estimation of multiple motions: parametric and piecewisesmooth flow fields,” Computer Vision and Image Understanding 63, pp. 75–104, Jan. 1996. 2. M. Shizawa and K. Mase, “Simultaneous multiple optical flow estimation,” in IEEE Conf. on Computer Vision and Pattern Recognition, I, pp. 274–8, IEEE Computer Soc., (Atlantic City, NJ), June 1990. 3. D. Vernon, “Decoupling Fourier components of dynamic image sequences: a theory of signal separation, image segmentation and optical flow estimation,” in Computer Vision - ECCV’98: 5th European Conf. on Computer Vision, H. Burkhardt and B. Neumann, eds., LNCS 1407/II, pp. 68–85, Springer Verlag, Jan. 1998. 4. J. Y. A. Wang and E. H. Adelson, “Representing moving images with layers,” IEEE Transactions on Image Processing 3(5), pp. 625–38, 1994.

I. S, T. A, C. M,  E. B/P. SPIE, 5022:75–86, M 2004.

12

E   :    

(a)

(b)

(c)

(d)

Figure 4. Separation of reflectance components. For details see text.

5. C. Mota, I. Stuke, and E. Barth, “Analytic solutions for multiple motions,” in Proc. IEEE Int. Conf. on Image Processing, II, pp. 917–20, IEEE Signal Processing Soc., (Thessaloniki, Greece), Oct. 7–10, 2001. 6. B. Horn and B. Schunck, “Determining optical flow,” Artificial Intelligence 17, pp. 185–203, Aug. 1981. 7. D. J. Fleet and A. D. Jepson, “Computation of component image velocity from local phase information,” International Journal of Computer Vision 5(1), pp. 77–104, 1990. 8. E. Barth, I. Stuke, and C. Mota, “Analysis of motion and curvature in image sequences,” in Proc. 5th Southwest Symp. on Image Analysis and Interpretation, pp. 206–10, IEEE Computer Soc., (Santa Fe, NM), Apr. 7–9, 2002.

I. S, T. A, C. M,  E. B/P. SPIE, 5022:75–86, M 2004.