Image Processing

6 downloads 0 Views 2MB Size Report
images as partial functions assuming a slightly more abstract point of view, g[x,y] can also be thought of as a partial function g : R2. {. 0,...,255. } ...
Image Processing Prof. Christian Bauckhage

outline lecture 03 recap representation of digital images (part 2) images as n-way arrays images as functions derivatives of images emboss effect summary

recap

intensity images are recorded using CCD chips which consist of many photosensitive elements

CCD chip

schematic CCD matrix

recap

color images are recorded by means of splitting light beams into basic components . . .

trichroic prism

three separate CCD arrays

recap

or by means of filtering wavelengths prior to recording

Bayer filter

single CCD array

recap

in memory, digital images are stored as pixel arrays their spatial resolution is characterized by width and height of these arrays their color resolution is characterized by the number of bits per color channel typically, images are stored using 256 = 28 intensity levels per channel, this requires 8 bit (= 1 byte) per 3 pixel per channel and allows for 28 = 16, 777, 216 colors per pixel

⇔ digital images are digitized and quantized

representation of digital images (part 2)

images as n-way pixel arrays

in computer memory, digital images are usually represented as n-way arrays of integer values

n-way arrays 1-way array (“vector”)

2-way array (“matrix”)

3-way array (“tensor”)

n-way pixel arrays

intensity image (“matrix”)

color image (“tensor”)

2-way pixel arrays

a pixel p is addressed using its coordinates i and j

i

pij

we also write pij

j

2-way pixel arrays

a pixel p is addressed using its coordinates i and j

i

pij

we also write pij

j

0 6 i < m is the row index and 0 6 j < n is the column index following the convention for matrices, pixel p00 is located in the upper left corner of the image

2-way pixel arrays ⇔ 1-way pixel arrays

pixel p can either be addressed trough two indices i and j denoting its row and column coordinates and we write pij . . . or pixel p can be addressed trough a single index k and we write pk

2-way pixel arrays ⇔ 1-way pixel arrays

pixel p can either be addressed trough two indices i and j denoting its row and column coordinates and we write pij . . . or pixel p can be addressed trough a single index k and we write pk

for an image of resolution m × n, we have the following relationship pij ⇔ p(i·n+j) = pk

pixel neighborhoods

in image processing, we often consider the 4-neighborhood and the 8-neighborhood of a pixel p

p

p

4-neighborhood

8-neighborhood

pixel distances

generally, the spatial distance between two pixels p and   q with coordinates i, j and k, l is measured using the Euclidean distance between their coordinates

    q

i 2 2 k

d p, q = i−k + j−l

j − l = 

note

for the pixels q in the 4-neighborhood of p, the Euclidean distance between p and q amounts to  d p, q = 1

note

for the pixels q in the 4-neighborhood of p, the Euclidean distance between p and q amounts to  d p, q = 1

for the pixels in the 8-neighborhood of a pixel p, this does not always hold, for instance

 d pij , pi+1 j+1 =

q i − (i + 1)

p 12 + 12 √ = 2 =

2

+ j − ( j + 1)

2

question can’t we consider or define a distance that would avoid such “strange behavior” ?

question can’t we consider or define a distance that would avoid such “strange behavior” ?

answer yes, we can (more on this later) but we rather shouldn’t (more on this later)

question why does image processing resorts to pixel arrays where some immediate neighbors are closer than others ? is it because it is impossible to find an arrangement where pixels cover the image plane and have an equal distance to all their immediate neighbors ?

question why does image processing resorts to pixel arrays where some immediate neighbors are closer than others ? is it because it is impossible to find an arrangement where pixels cover the image plane and have an equal distance to all their immediate neighbors ?

answer no, in principle, there are many possibilities to achieve this!

tessellation / tiling

a tessellation is a collection of planar figures that cover the entire plane without overlaps and gaps

tessellation / tiling

interesting tessellations have certain symmetry properties, they are periodic or a-periodic, . . . most notably, there are tessellations where the Euclidean distance between the centers of adjacent tiles is always the same

question then why aren’t these used in digital photography ?

question then why aren’t these used in digital photography ? answer as of today, the memory of computers is typically conceptualized as a (very long) linear sequence of storage cells

question then why aren’t these used in digital photography ? answer as of today, the memory of computers is typically conceptualized as a (very long) linear sequence of storage cells therefore, the simplest approach is to represent the rows of an image as a sequence of storage cells and to represent the image as a sequence of such rows, i.e. as a 1-way pixel array

images as discrete functions

in a gray value image, discrete coordinates are mapped to discrete intensity values a gray value image may therefore also be thought of as a bi-variate discrete function   g x, y where    g : 0, . . . , n − 1 × 0, . . . , m − 1 → 0, . . . , 255

images as discrete functions

in a gray value image, discrete coordinates are mapped to discrete intensity values a gray value image may therefore also be thought of as a bi-variate discrete function   g x, y where    g : 0, . . . , n − 1 × 0, . . . , m − 1 → 0, . . . , 255 in contrast to the matrix interpretation of an image, here, index x denotes image columns, y denotes image rows, and coordinate [0, 0] is located in the lower left corner

example

150 0

5

10

15

20

25

30

35

40

45

50

50 45 40 35 30 25 20 15 10 5 0

images as partial functions

  assuming a slightly more abstract point of view, g x, y can also be thought of as a partial function  g : R2 ; 0, . . . , 255

images as partial functions

  assuming a slightly more abstract point of view, g x, y can also be thought of as a partial function  g : R2 ; 0, . . . , 255  i.e. while certain  tuples x, y ∈ R2 are mapped to gray values in the set 0, . . . , 255 many (or most) possible tuples will not be assigned a value

images as partial functions

  assuming a slightly more abstract point of view, g x, y can also be thought of as a partial function  g : R2 ; 0, . . . , 255  i.e. while certain  tuples x, y ∈ R2 are mapped to gray values in the set 0, . . . , 255 many (or most) possible tuples will not be assigned a value

    instead of g x, y we also write g x , x ∈ R2 to express that we understand an image as a partial 2D function

color images as vector fields

in a color image, discrete coordinates are mapped to certain color values which are represented  by triples of numbers taken from the set 0, . . . , 255 a color image therefore is a vector valued function a color image can thus be seen as a vector field   c x where  3 c : R2 ; 0, . . . , 255

question what do we gain from raising the level of abstraction ?

question what do we gain from raising the level of abstraction ? answer if we are dealing with mathematical functions, we may apply the whole toolbox of mathematical techniques which have been developed over centuries we can apply functions or operators to images, i.e. we can compute with images

question what do we gain from raising the level of abstraction ? answer if we are dealing with mathematical functions, we may apply the whole toolbox of mathematical techniques which have been developed over centuries we can apply functions or operators to images, i.e. we can compute with images

this is the basis of image processing

let’s look at an example

derivative of a continuous uni-variate function f (x)

derivative from above f+0 (x) = lim

ε→0

f (x + ε) − f (x) ε

derivative from below f−0 (x) = lim

ε→0

f (x) − f (x − ε) ε

derivative of a continuous uni-variate function f (x)

derivative from above f+0 (x) = lim

ε→0

f (x + ε) − f (x) ε

derivative from below f−0 (x) = lim

ε→0

f (x) − f (x − ε) ε

f (x) is differentiable in x if f+0 (x) = f−0 (x) = f 0 (x) =

d f (x) dx

example derivatives from above and from below of f (x) = x2 x=2

f+0 :

16

16

16

16

12

12

12

12

8

8

8

4

4

4

0 2

4

0

ε=2

f−0 :

4

0

0 0

8

...

2

0 0

4

2

4

0

16

16

16

16

12

12

12

12

8

8

8

4

4

4

0 2

ε=2

4

8

...

0

0 0

0

2

ε=1

4

4

lim ε → 0

ε = 0.5

ε=1

2

4

0 0

2

ε = 0.5

4

0

2

4

lim ε → 0

note

f (x) may be continuous in x and f+0 (x) and f−0 (x) may exists, but they need not be identical ⇔ f (x) may not be differentiable in x

f (x)

x0

x

derivative of a discrete uni-variate function f [x]

derivative from above f+0 [x] =

f [x + 1] − f [x] = f [x + 1] − f [x] (x + 1) − x

derivative from below f−0 [x] =

f [x] − f [x − 1] = f [x] − f [x − 1] x − (x − 1)

derivative of a discrete uni-variate function f [x]

derivative from above f+0 [x] =

f [x + 1] − f [x] = f [x + 1] − f [x] (x + 1) − x

derivative from below f−0 [x] =

f [x] − f [x − 1] = f [x] − f [x − 1] x − (x − 1)

due to the discrete nature of f [x], it is impossible to consider a limit process ⇒ the minimum of the denominator will always be 1 ⇒ the derivative from above and the derivative from below will generally differ

example derivatives from above and from below of f [x] = x2 x=2

16

16

12

12

8

8

4

4

0

0 0

1

2

f+0 [2]

3

4

0

1

2

f−0 [2]

3

4

“compromise”

when computing the derivative of a discrete function, consider information from below and from above

dx f [x] =

f [x + 1] − f [x − 1] 2

example discrete derivative of f [x] = x2 x=2

16

12

8

4

0 0

1

2

dx f [2]

3

4

gradients

the gradient of a continuous multivariate function f (x1 , x2 , . . . , xn ) is the vector of partial derivatives  ∂f  ∂x1

  ∇f (x1 , . . . , xn ) =  ...  ∂f ∂xn

gradients

the gradient of a continuous multivariate function f (x1 , x2 , . . . , xn ) is the vector of partial derivatives  ∂f  ∂x1

  ∇f (x1 , . . . , xn ) =  ...  ∂f ∂xn

accordingly, we can define “a gradient” of an intensity image g[x, y] as follows     1 g[x + 1, y] − g[x − 1, y] dx g[x, y] ∇g[x, y] = = · dy g[x, y] 2 g[x, y + 1] − g[x, y − 1]

note

the gradient is an operator   ∇ : Rn → R → Rn → Rn

note

the gradient is an operator   ∇ : Rn → R → Rn → Rn that is, it is a function that operates on functions

note

the gradient is an operator   ∇ : Rn → R → Rn → Rn that is, it is a function that operates on functions that is, it takes a function and produces another function

note

the gradient is an operator   ∇ : Rn → R → Rn → Rn that is, it is a function that operates on functions that is, it takes a function and produces another function contrast this with functions which take a (tuple of) number(s) and produce another (tuple of) number(s)

note

∇g[x, y] is a discrete, vector valued function, i.e. a discrete vector field

note

∇g[x, y] is a discrete, vector valued function, i.e. a discrete vector field for visualization, we usually consider only its magnitude q

q

2

∇g[x, y] = ∇g[x, y], ∇g[x, y] = ∇g[x, y]

note

∇g[x, y] is a discrete, vector valued function, i.e. a discrete vector field for visualization, we usually consider only its magnitude q

q

2

∇g[x, y] = ∇g[x, y], ∇g[x, y] = ∇g[x, y]

and values exceeding 255 are typically rounded off to 255

example

g[x, y]

dx g[x, y]

dy g[x, y]



∇g[x, y]

example

450 400 350 300 250 200 150 100 50

0

100

200

300

400

500

∇g[x, y] shown as a vector field

600

700

question what is this good for ?

question what is this good for ? answer it detects edges!

question what is this good for ? answer it detects edges! question what is this good for ?

question what is this good for ? answer it detects edges! question what is this good for ? answer we will study numerous applications of this later for now, we consider an artistic image processing example

emboss effect idea transform an input image such that it looks like a copper engraving or an etching

emboss effect idea transform an input image such that it looks like a copper engraving or an etching

approach similar to the computation of derivatives transform given image g into new image ˜ g using

˜g[x, y] = 128 + g[x + 1, y + 1] − g[x − 1, y − 1]



examples

summary

we now know about

mathematical representations of digital images continuous derivatives and gradients discrete derivatives and gradients

exercises

the divergence of a continuous multivariate function f (x1 , x2 , . . . , xn ) is defined as ∆ f = ∇2 f = ∇ · ∇f =

n X ∂2 f i=1

∂xi2

where ∆ is called the Laplace operator write down / devise an expression for the divergence of a discrete 2D function g[x, y]

exercises

assume a discrete 1D function g[x]; what does the following operator do ? AM g[x] =

 1 g[x − 1] + g[x] + g[x + 1] 3

what about this one ? WM g[x] =

 1 g[x − 1] + 2g[x] + g[x + 1] 4