Document 11.

8 downloads 53 Views 543KB Size Report
Bernd Girod: EE368 Digital Image Processing. Scale-Space Feature Detection no. 1. Scale-space feature detection. ▫ Image features can appear similarly on all  ...
Scale-space feature detection  

Chapter overview: scale-space feature detection

Image features can appear similarly on all scales

 

Scale space representation of images      

   

 

 

 

In addition to shift-invariance, scale-invariance is often a desirable property of feature detection Scale-space representation of an image is useful to detect features which are both shift-invariant and scale-invariant Bernd Girod: EE368 Digital Image Processing

 

Scale-Space Feature Detection no. 1

 

Family of signals generated by successive smoothing with a Gaussian filter

 

( )

( )

 

Harris-Laplacian SIFT SURF

Scale-Space Feature Detection no. 2

Bernd Girod: EE368 Digital Image Processing

[Witkin 1983]

 

Shift-invariance

 

Rotation-invariance

Bernd Girod: EE368 Digital Image Processing

Scale-Space Feature Detection no. 4

Bernd Girod: EE368 Digital Image Processing

Scale space: Laplacian images

Scale-Space Feature Detection no. 7

 

( )

t ⋅∇ 2 f t x, y

( )

f t x, y

t=4

( )

t ⋅∇ 2 f t x, y

t=4

t = 16

t = 64

2

)

(

)

H ω x ,ω y =

1

(k −1)σ 2

[G (ω ,ω ) − G (ω ,ω )] k 2σ 2

σ2

x

y

x

y

4. 

( )

f t x, y Bernd Girod: EE368 Digital Image Processing

Scale-Space Feature Detection no. 9

Scale space: Binarized Laplacian images

t = 16

(

) (

) (

)

)

Scale-Space Feature Detection no. 6

Difference of Gaussians

( )

t ⋅∇ 2 f t x, y

Bernd Girod: EE368 Digital Image Processing

( )

f t x, y

t = 64

x 2 + y2

DoG ( x, y ) =

2σ 2

1



1

( k − 1) σ 2 ⎜⎝ 2π k 2σ 2

e



x 2 + y2 2 k 2σ 2



− 1 e 2πσ 2

x 2 + y2

(

(

Harris

))

y

Harris

Apply Harris detector in a spatial neighborhood at scale th to refine keypoint location xh , yh Repeat 2. and 3. until convergence

x

( )

t ⋅∇ 2 f t x, y

Scale-Space Feature Detection no. 10

Non-creation of local extrema

Bernd Girod: EE368 Digital Image Processing

Scale-Space Feature Detection no. 13

Bernd Girod: EE368 Digital Image Processing

Keypoint detection with automatic scale selection (cont.)  

t = 16

t=8

   

t=4 2

t=4

t=4

Scale-Space Feature Detection no. 14

SIFT blob detection  

Harris-Laplacian example (200 strongest peaks)

t=8 2

 

t=4

SIFT - Scale-Invariant Feature Transform Decompose image into DoG scale-space representation Detect minima and maxima locally and across scales Fit 3-d quadratic function to 9 values to localize extrema with sub-pixel/sub-scale accuracy

t=2 2

t=2

t= 2 t =1

t = 16

t = 64

[Brown, Lowe, 2002]

[Lindeberg 1996] Bernd Girod: EE368 Digital Image Processing

Scale-Space Feature Detection no. 11

Bernd Girod: EE368 Digital Image Processing

Scale-Space Feature Detection no. 12

Bernd Girod: EE368 Digital Image Processing

2σ 2

⎞ ⎟ ⎠

Scale-Space Feature Detection no. 8

some initial scale For each Harris corner xh , yh detect characteristic scale

Laplacian zero-crossings

t=1

(

t = σ2 = 1, k = 1.1

th = arg max t ⋅∇ 2 f t xh , yh 3. 

) (

1 2 ω + ω 2y F t ω x ,ω y 2 x

)

Scale-space representation provides all scales; which scale is best for keypoint detection? Harris-Laplacian Harris 1.  Detect Harris corners at scale t 2. 

t=1

1 ⎛ x 2 + y2 ⎞ − e 1− πσ 4 ⎜⎝ 2σ 2 ⎟⎠

t

) (

=−

(

Keypoint detection with automatic scale selection

Low-gradient-magnitude edges removed

(

⎛ t ⎞ 1 2 ω + ω 2y exp ⎜ − ω x2 + ω 2y ⎟ F ω x ,ω y 2 x ⎝ 2 ⎠

Bernd Girod: EE368 Digital Image Processing

 

H ω x ,ω y = − ω x2 + ω y2 Gσ ω x ,ω y

)

=−

t = σ2 = 1

t = σ2 = 1, k = 1.1

t=1

) ) (

LoG vs. DoG

Zero crossings of Laplacian images

( )

) (

Laplacian of Gaussian

( )

Scale space: edge detection

Difference of Gaussians

f t x, y

(

Bernd Girod: EE368 Digital Image Processing

LoG ( x, y ) = −

LoG vs. DoG (cont.)

)

Original image f (x,y)

Zero-crossings of 2nd derivative: Fewer features at coarser scales

t = σ2 = 1

Scale-Space Feature Detection no. 5

Scale space as heat diffusion ( )

Parametric family of images smoothed by Gaussian filter

Laplacian of Gaussian

(

∂ t 1 f x, y = ∇ 2 f t x, y ∂t 2

 

Scale-Space Feature Detection no. 3

( )

∂ ∂ t F ω x ,ω y = G t ω x ,ω y F ω x ,ω y ∂t ∂t ⎛ t ⎞ ∂ = exp ⎜ − ω x2 + ω 2y ⎟ F ω x ,ω y ∂t ⎝ 2 ⎠

(

Coarser scales

Bernd Girod: EE368 Digital Image Processing

Separability

Increasing t

scale t

Non-creation of local extrema (for f (x,y) and all of its partial derivatives) since g t x, y ≥ 0 and unimodal. Solution to diffusion equation (heat equation) ∂ t 1 f x, y = ∇ 2 f t x, y ∂t 2

Scale-space representation of images

From an original signal f(x) generate a parametric family of signals f t (x) where fine-scale information is successively attenuated

Scale-space representation of images (cont.)  

 

Bernd Girod: EE368 Digital Image Processing

Scale-space representation of a signal  

Commutative semigroup property

Definition and useful properties Scale space and diffusion equation LoG vs DoG

Scale-space edge detection Scale-space keypoint detectors  

Scale-space representation of images (cont.)  

Scale-Space Feature Detection no. 15

Bernd Girod: EE368 Digital Image Processing

Scale-Space Feature Detection no. 16

SIFT scale space pyramid: octave 1

SIFT scale space pyramid: octave 2

SIFT scale space pyramid: octave 5

-

-

-

-

-

-

-

-

-

Bernd Girod: EE368 Digital Image Processing

Scale-Space Feature Detection no. 17

Bernd Girod: EE368 Digital Image Processing

Scale-Space Feature Detection no. 18

SIFT scale space pyramid: octave 3

SIFT scale space pyramid: octave 4

-

-

Bernd Girod: EE368 Digital Image Processing

Scale-Space Feature Detection no. 21

Robustness against scaling

SIFT blobs

Bernd Girod: EE368 Digital Image Processing

Scale-Space Feature Detection no. 22

Hessian keypoints in scale space

−g(x, y)

-

-

-

-

[Mikolajczyk, Schmid, 2001]  Bernd Girod: EE368 Digital Image Processing

Scale-Space Feature Detection no. 19

SURF blob detection      

Scale-Space Feature Detection no. 20

SURF blob detection (cont.)

SURF – Speeded Up Robust Features [Bay, Tuytelaars, Van Gool, ECCV 2006] No subsampling – all resolution levels at full spatial resolution Simple approximation of scale space Gaussian derivatives using integral images

D tyy  

Bernd Girod: EE368 Digital Image Processing

D txy

Determinant of Hessian

( ) t

t xx

t yy

(

det H ≈ D D − 0.9D

t xy

)

2

 

Non-maximum suppression in 3x3x3 [x,y,t] neighborhood

 

Interpolation of maximum of det(H) in image space x,y and scale t

Bernd Girod: EE368 Digital Image Processing

Scale-Space Feature Detection no. 25

SURF blob detection (cont.)

Bernd Girod: EE368 Digital Image Processing

Scale-Space Feature Detection no. 27

Bernd Girod: EE368 Digital Image Processing

Scale-Space Feature Detection no. 26

SIFT blobs

Bernd Girod: EE368 Digital Image Processing

Scale-Space Feature Detection no. 28

Bernd Girod: EE368 Digital Image Processing

Scale-Space Feature Detection no. 23

Bernd Girod: EE368 Digital Image Processing

Scale-Space Feature Detection no. 24