Improved watermark synchronization based on local autocorrelation ...

12 downloads 0 Views 957KB Size Report
Improved watermark synchronization based on local autocorrelation function. Min-Jeong Lee. Kyung-Su Kim. Tae-Woo Oh. Korea Advanced Institute of Science ...
Journal of Electronic Imaging 18(2), 023008 (Apr–Jun 2009)

Improved watermark synchronization based on local autocorrelation function Min-Jeong Lee Kyung-Su Kim Tae-Woo Oh Korea Advanced Institute of Science and Technology Department of Electrical Engineering and Computer Science Guseong-dong, Yuseong-gu Daejeon, Korea E-mail: [email protected] Hae-Yeoun Lee Kumoh National Institute of Technology School of Computer and Software Engineering Yangho-dong, Gumi Gyeongbuk, Korea Heung-Kyu Lee Korea Advanced Institute of Science and Technology Department of Electrical Engineering and Computer Science Guseong-dong, Yuseong-gu Daejeon, Korea

Abstract. An autocorrelation function (ACF) to synchronize watermarks has been adopted in practical applications because of its robustness against affine transforms. However, ACFs are vulnerable to projective transform, which commonly occurs during the illegal copying of cinema footage due to the angle of the camcorder relative to the screen. The cinema footage that is captured by camcorders both is projected and has undergone digital-to-analog and analog-to-digital conversion (D-A/A-D conversion). We present a novel watermarking scheme that uses a local autocorrelation function (LACF) that can resist projective transforms as well as affine transforms. A watermark also used for synchronization is designed and additively embedded in the spatial domain. The embedded watermark is extracted in a blind way after recovering from distortions. The LACF scheme with a mathematical model is proposed to synchronize the watermark against distortions. On various video clips, experimental results show that the presented scheme is robust against projective distortions as well as D-A/A-D conversion. © 2009 SPIE and IS&T. 关DOI: 10.1117/1.3134121兴

1 Introduction With the rapid spread of digital content, copyright protection has become a critical issue. Many illegal copies of digital video productions for cinema release can be found on the Internet or on the black market before their official release. These copies were made by recording the projected movie with a camcorder at various angles, according to the Paper 08122R received Aug. 7, 2008; revised manuscript received Mar. 10, 2009; accepted for publication Apr. 3, 2009; published online May 29, 2009. 1017-9909/2009/18共2兲/023008/11/$25.00 © 2009 SPIE and IS&T.

Journal of Electronic Imaging

location of the pirate. Therefore, they are translated, rotated, scaled, and projected during camcording. Digital watermarking technology seems to match the requirements of digital cinema perfectly in terms of copyright protection. However, due to the fact that the illegal copies undergo severe degradation in quality and geometrical distortions, watermark detection is drastically impeded.1,2 Therefore, watermarking techniques that are used for digital cinema must survive geometrical distortions, such as rotation, scaling, translation 共RST兲 and projection. In addition, the following requirements should be satisfied: 共1兲 a payload size of over 35 bits in each 5-min movie segment; 共2兲 real-time embedding; 共3兲 imperceptibility in the high-definition motion picture source 共fidelity兲; 共4兲 robustness, even in degraded copies; and 共5兲 security against unauthorized removal.3,4 Several papers have addressed watermarking for digital cinema. Leest et al.2 proposed a video watermarking scheme that exploits the temporal axis to embed the watermark by changing the luminance value of each frame, thereby achieving robustness against geometrical distortions. However, due to the fact that flickering that is caused by the luminance change between frames is visible to the human eye, the viewing quality is not satisfactory. Delannay et al.5 investigated the restoration of geometrically distorted images that are generated by the camera’s angle of acquisition. However, their method is impractical because it uses unmodified content for detection. Moreover, the method requires that the original video footage and the cap-

023008-1

Downloaded from SPIE Digital Library on 19 Nov 2009 to 143.248.135.186. Terms of Use: http://spiedl.org/terms

Apr–Jun 2009/Vol. 18(2)

Lee et al.: Improved watermark synchronization based on local…

tured footage be time-synchronized, which is a complex task. Lubin et al.4 embedded the watermark into a low spatial-temporal frequency domain for invisible, robust, and secure watermarking. To determine spatial-temporal regions of video sequences in the embedding procedure, a vision model–based masking computation is used. However, this method cannot satisfy the requirement for realtime embedding. In fact, none of the mentioned mathods can satisfy the real-time embedding requirement, nor do they consider the projective distortions generated by camcording. Many watermarking schemes have been designed to resist geometric distortion in still images, using such methods as invariant transforms,6,7 image features,8,9 template insertion,10 and periodical sequences.11–13 These approaches are robust against RST distortions, which are called affine transforms. However, these approaches cannot estimate or recover from projective distortions. To satisfy the requirements for real-time embedding and blind detection, an autocorrelation function 共ACF兲, which is able to self-synchronize,11 has been utilized. However, it can recover the original images only from affine transforms, not from projective transforms. In digital cinema applications, the watermark has to be robust against projective transform as well as affine transform. Given that the autocorrelation peaks are used for synchronization to recover undistorted images from geometrically distorted images, the overall performance depends on the accuracy with which the correlation peaks are extracted from various geometric distortions. Moreover, because the ACF uses whole images, it cannot be used for HD-video frames or digital cinema while satisfying the real-time embedding requirement. In this paper, we propose a watermarking scheme that is robust to both geometric distortions, such as projective transforms and affine transforms, and signal-processing distortions. To resist geometric distortions, the proposed scheme uses a local autocorrelation function 共LACF兲 and establishes a related mathematical model. To satisfy the real-time embedding requirement and imperceptibility, an optimized human visual system 共HVS兲 function is used.14 The scheme does not need to use original content during detection. The remainder of the paper is organized as follows. Section 2 addresses the projective transform and the problem of ACF-based watermarking. Section 3 describes the watermark embedding process. Section 4 explains the proposed watermark detection process with the LACF to restore geometrical distortions. Experimental results are presented in Section 5. Section 6 presents our conclusions. 2 Statement of Problem 2.1 Projective Transformation Cinematic footage that is captured by a camcorder undergoes perspective–projective distortion when the position and/or viewing angle of the camcorder is changed. In this situation, distances and angles are not preserved and parallel lines do not project to parallel lines unless they are parallel to the image plane. Let x = 共x1 , x2 , x3兲T be the homogeneous vector that represents a point in the original frame and x⬘ = 共x1⬘ , x2⬘ , x3⬘兲T be the homogeneous vector that represents a point in the geometrically distorted frame. The Journal of Electronic Imaging

projective transformation is a linear transformation on homogeneous 3-vectors represented by a nonsingular 3 ⫻ 3 matrix:15

x⬘ = Hx,

冢 冣

a b c where H = d e f . g h i

共1兲

To estimate a projective transform, all nine parameters 共a ⬃ i兲 of the matrix H should be obtained. Given that the matrix H is homogeneous, the degree of freedom 共DOF兲 of H is 8, even though H consists of nine constants and i is equal to 1 unless point 共0,0兲 corresponds to the point at infinity by the matrix transform. If the host image is projected horizontally, b and h are zero, while if the host image is projected vertically, d and g are zero. Let the inhomogeneous coordinates of a pair of matching points x and x⬘ be 共x , y兲 and 共x⬘ , y ⬘兲, respectively. 共x , y兲 is calculated by 共x1 / x3 , x2 / x3兲, and 共x⬘ , y ⬘兲 is obtained by 共x1⬘ / x3⬘ , x2⬘ / x3⬘兲. Therefore, at least the original 共x , y兲 and transformed 共x⬘ , y ⬘兲 locations of four points are required. Any set of three of them should be collinear. In this paper, the set of four points is obtained by the local autocorrelation function 共LACF兲. Details of the LACF are presented in Sec. 4.1.

2.2 Autocorrelation Function Watermarking schemes that are based on an autocorrelation function 共ACF兲 have commonly been employed to resist affine transform and signal-processing distortions.11,16 The ACF handles affine transforms by embedding periodic watermark patterns. Using the periodicity, periodic peaks are found in the ACF of the watermark. By considering the peaks, the watermark detector estimates what kinds of geometric distortions occurred and how much the image was altered by calculating intervals and angles between the peak points. The hidden message is extracted after inverting the geometric distortion. Hence, correct detection of the periodic peaks is decisive for watermark detection. Due to the fact that affine transform preserves collinearity between points, ratios of distances along a line, and parallelism of lines, the overall pattern of the peaks does not change, even though the original position of the periodic peaks moves. Therefore, the ACF can estimate affine transform. However, in the case of projective transform, the ACF cannot extract the periodic peaks because projective transform does not preserve ratios of distances along a line and parallelism of lines. The projected peaks move, and the overall pattern of the peaks changes. Figure 1 shows the results of applying the ACF to images that have been subjected to geometric attack and a plot that depicts the peaks that were extracted by the ACF. The ACF correctly estimates the peaks from the watermarked image that has not been attacked and the rotated image, as shown in Figs. 1共a兲 and 1共b兲, respectively. However, it cannot identify projective transform because it calculates the autocorrelation of the entire watermark, the pattern of which is not regular. Figure 1共c兲 shows that the ACF fails to extract the correct peaks of the watermark.

023008-2

Downloaded from SPIE Digital Library on 19 Nov 2009 to 143.248.135.186. Terms of Use: http://spiedl.org/terms

Apr–Jun 2009/Vol. 18(2)

Lee et al.: Improved watermark synchronization based on local…

Fig. 1 Watermarked image and extracted peaks by ACF: 共a兲 no attack, 共b兲 rotation 20 deg, 共c兲 horizontal projection 20 deg.

3 Watermark Embedding In this section, we describe how the watermark is embedded in the host video frame. Figure 2 shows the embedding procedure, which is designed to satisfy the requirements for Journal of Electronic Imaging

digital cinema.3 First, the watermark pattern is generated and then inserted into part of the video frames, taking the human visual system 共HVS兲 into account. In the presented scheme, the watermark pattern is used

023008-3

Downloaded from SPIE Digital Library on 19 Nov 2009 to 143.248.135.186. Terms of Use: http://spiedl.org/terms

Apr–Jun 2009/Vol. 18(2)

Lee et al.: Improved watermark synchronization based on local…

Perceptual model

decode Original video frame (I)

Video sequence

⎡ 1 ⎢ 0 ⎢ ⎢⎣ −1

Display Screen

1 0 −1

⎤ ⎥ ⎥ ⎥⎦

⎡ 1 ⎢ 1 ⎢ ⎢⎣ 0

1 0 −1

N

Watermarked video frame (Iw)

⎡ −1 ⎢ 0 ⎢ ⎢⎣ 1

Local weighting factor

−1 0 1

0 −1 −1

⎤ ⎥ ⎥ ⎥⎦

⎡ 1 ⎢ 1 ⎢ ⎢⎣ 1

NW

−1 0 1

⎤ ⎥ ⎥ ⎥⎦

⎡ −1 ⎢ −1 ⎢ ⎢⎣ 0

−1 0 1

S

Watermark generator

0 0 0

−1 −1 −1

⎤ ⎥ ⎥ ⎥⎦

⎡ 0 ⎢ 1 ⎢ ⎢⎣ 1

W

0 1 1

⎤ ⎥ ⎥ ⎥⎦

⎡ −1 ⎢ −1 ⎢ ⎢⎣ −1

SE

0 0 0

−1 0 1

−1 −1 0

⎤ ⎥ ⎥ ⎥⎦

1 1 0

⎤ ⎥ ⎥ ⎥⎦

SW

1 ⎤ 1 ⎥ ⎥ 1 ⎥⎦

⎡ 0 ⎢ −1 ⎢ ⎣⎢ −1

E

1 0 −1 NE

Fig. 3 Compass operators.

Basic pattern generation

Wb Wb Wb Wb Wb Wb Wb Wb

Low-pass filtering

Key & Message

1 0 −1

Wb Wb Wb Wb

2

Wb Wb Wb Wb

HVS共x,y兲 =

Tiling Periodic watermark(W)

共3兲

Fig. 2 Watermark embedding procedure.

in two ways: 共1兲 to estimate and recover geometric distortions, and 共2兲 to extract the embedded message. In order for the LACF to estimate geometric distortions, the watermark pattern should have periodicity. The basic pattern for a periodic watermark is generated using a secret key and a message 共e.g., copyright information, fingerprint information, etc.兲 and consists of a 2-D random sequence of size 共M / n ⫻ M / n兲 that follows a Gaussian distribution with zero mean and unit variance. M denotes the final size of the watermark pattern in each direction, and n denotes the number of repetitions in each direction. To enable the LACF to perform better, the basic pattern is low-pass filtered before repetition because low-frequency components are affected little by common signal processing, especially digital-to-analog and analog-to-digital 共D-A/A-D兲 conversion.17 The filtered basic pattern Wb is repeated n times on the vertical and horizontal axes, respectively, to get the periodicity. After a periodic watermark pattern of size M ⫻ M has been obtained, the pattern is embedded using an additive spread-spectrum method with perceptual scaling. The watermark W is embedded in a video frame F as follows:

Fw共x,y兲 = F共x,y兲 + ␣␭共x,y兲 ⫻ W共x,y兲,





M M = Wb x mod ,y mod , n N

where F = 兵N , NW , W , SW , S , SE , E , NE其 in Fig. 3. The compass operator can reduce computational costs by utilizing its separable property. The separable 3 ⫻ 3 filter requires only 6 共=3 + 3兲, rather than 9 共=3 ⫻ 3兲, multiplications. Last, the local weighting factor on ␭共x , y兲 is obtained by: ␭共x,y兲 = S0 ⫻ 关C − HVS共x,y兲兴 + S1 ⫻ HVS共x,y兲,

共4兲

where C is a constant value that limits the upper bound of HVS energies, and S0 and S1 are the user-defined weighting factors for plain and textured regions, respectively. HVS共x , y兲 has a high value in textured regions and a low value in plain regions. Therefore, S0 affects the strength in plain regions more than S1, whereas S1 affects the embedding strength in textured regions. By trading off watermark visibility against robustness, the watermark strength could be controlled by adjusting S0 and S1. 4 Watermark Detection The watermark is detected as follows 共Fig. 4兲: 共1兲 estimate the watermark using a whitening filter, 共2兲 find geometric distortions using the LACF on the estimated watermark pattern, 共3兲 recover the watermark from the distortions, and

decode

Denoising filter

where W共x,y兲

Geometric distortion restoration

Folding

共2兲

Accumulation

Watermarked video frame (Iw)

Video sequence

where ␣ is a global weighting factor and ␭共x , y兲 is a local weighting factor of the pixel 共x , y兲 from HVS. We employ an HVS function that has been optimized for real-time embedding14 that adopts eight compass operators as a local weighting function, while preserving the noise visibility function 共NVF兲.18 The compass operators measure gradients in a selected number of directions. An anticlockwise circular shift of the eight boundaries gives a 45-deg rotation of the gradient direction. Our HVS function is defined as follows: Journal of Electronic Imaging

2

1 1 兺 兺 兺 兩F共i, j兲 ⫻ F共x + i − 1,y + j − 1兲兩, 8 F 9 i=0 j=0

Wb'

Cross-correlation based detector correlation > threshold ?

Estimated watermark

Geometric distortion estimation using LACF

if yes extract

Decoded Message

Watermark generator

Key

Basic pattern generation Low-pass filtering

Wb Reference watermark

Fig. 4 Watermark detection procedure.

023008-4

Downloaded from SPIE Digital Library on 19 Nov 2009 to 143.248.135.186. Terms of Use: http://spiedl.org/terms

Apr–Jun 2009/Vol. 18(2)

Lee et al.: Improved watermark synchronization based on local…

共4兲 extract the embedded message. The watermark is estimated and the message is extracted in a manner similar to the way in which they are estimated and extracted in the ACF-based schemes.11,16 Our major contributions are discussed in the following subsections. Estimating the projection parameters by extracting the peaks using the LACF is described in Sec. 4.1, and recovering the distortions using a mathematical model is explained in Sec. 4.2. Due to the fact that a blind detector is used, the embedded watermark is estimated by employing Wiener filtering19 as a denoising filter. The goal of using a denoising filter is to decorrelate the watermarked video frame Fw⬘ to obtain an approximately spectrally white version of Fw⬘ . The Wiener filter estimates the original signal from the watermarked frame:19

␴2共x,y兲 − ␯2 关Fw⬘ 共x,y兲 − ␮共x,y兲兴, F⬘共x,y兲 = ␮共x,y兲 + ␴2共x,y兲

共6兲

¯ ⬘兲 · FFT共W 兲*兴 IFFT关FFT共W b b . ¯ 兩W⬘兩 · 兩W 兩 b

共7兲

b

If the normalized cross-correlation C exceeds an adaptive threshold, the hidden message is extracted successfully. The decision D to verify the existence of the watermark is made by D = max关C共x,y兲兴 ⬎ T,

共8兲

x,y

where T is the detection threshold defined by T = ␮ c + ␣ c␴ c ,

共9兲

where ␮c is the average, and ␴c is the standard deviation of the normalized cross-correlation. ␣c is a predefined value that is related to the false positive error rate. Journal of Electronic Imaging

(a) dx(RB) dx(RA)

M

dy(RA)

wR

M (=wR)

dy(RB)

RA

共5兲

For the purpose of enhancing the energy of the estimated watermark, we sum the values of each pixel of the estimated watermark in each frame in the series of frames for t seconds. With this accumulated version of W⬘, geometric distortion is estimated using the LACF and recovered to synchronize the watermark between embedding and detection. The basic pattern of size 共M / n ⫻ M / n兲 is generated using a secret key as a reference watermark. Before the message is extracted, the periodic watermark is folded so that it is the same size as the basic pattern. Normalized ¯ ⬘ and the cross-correlation between the folded watermark W b reference watermark pattern Wb can be calculated so that it can be performed in less time with fast Fourier transform 共FFT兲 by C=

M

M

M (=hR)

where ␮共·兲 and ␴2共·兲 are the local mean and local variance of the Fw⬘ , respectively. ␯2 is the noise variance. The average of the local variances is chosen as ␯2, because the detector has no knowledge about the probability distribution of the noise. The estimated watermark W⬘ that is yielded by the Wiener filter is given by W⬘ = Fw⬘ − F⬘ .

M

RB

RA

hR

RB

(b)

(c)

Fig. 5 Examples of projection attacks: M denotes the size of the periodic watermark pattern in each direction. Two parallel local areas to which the LACF will be applied are denoted by RA and RB. wR and hR stand for the width and the height of the regions RA and RB. dx共R兲 and dy共R兲 are the distances of the x axis and y axis from 共0,0兲 for the region R. 共a兲 Original watermark; 共b兲 horizontal projection example; 共c兲 vertical projection example.

4.1 Local Autocorrelation Function As mentioned in Sec. 2.1, four pairs of points in the original and transformed images are required to calculate the parameter in matrix H in Eq. 共1兲. Four corner points of the original watermark pattern are the original points, and four corner points of the distorted watermark pattern are the corresponding points. Given that both the embedder and the detector know the locations of four original points, it is necessary to know the locations of only four distorted points. The periodicity of the embedded watermark is exploited to find these four distorted points. On the assumption that projection occurs, the watermark pattern is arranged relative to the direction and the degree of the projection. When horizontal projection occurs, the height of the basic pattern on the same vertical line is regular, but different from the original height. The change in the height is proportional to the degree of horizontal projection. When vertical projection occurs, the width of the basic pattern on the same horizontal line is regular, but different from the original width. The change in the width is proportional to the degree of vertical projection. Along the direction that preserves the periodicity, the degree of the projection is estimated by comparing two measured periods on two different lines. We calculate the ACF locally on the region, along the direction that preserves the periodicity, to get the direction and the periods. In other words, the LACF is employed instead of the ACF, which uses two parallel local areas far from a distance. As shown in Fig. 5, two horizontally parallel local areas are needed for vertical projection, while two vertically parallel local areas are needed for horizontal

023008-5

Downloaded from SPIE Digital Library on 19 Nov 2009 to 143.248.135.186. Terms of Use: http://spiedl.org/terms

Apr–Jun 2009/Vol. 18(2)

Lee et al.: Improved watermark synchronization based on local…

projection. The two parallel local areas are denoted by RA and RB, respectively. It is crucial to set the size of each region and the distance between RA and RB. To detect horizontal projection, the height of a region should be as great as the height of one basic pattern, which guarantees that at least one peak on the horizontal axis will be extracted. The height of the region should also be less than twice the height of one basic pattern to maintain the normal ratio of the basic pattern. The width of the region has to be the same as the width of the periodic watermark pattern, as shown in Fig. 5共b兲. By contrast, to detect vertical projection, the region should be as wide as the width of one basic pattern and narrower than twice the width of one basic pattern. The region should have the same height as the periodic watermark pattern, as shown in Fig. 5共c兲. When the size of the extracted watermark is M ⫻ M, these relations are as follows: hR = M and

M M 艋 wR ⬍ 2 ⫻ n n

共horizontal projection兲, 共10兲

wR = M and

M M 艋 hR ⬍ 2 ⫻ n n

共vertical projection兲, 共11兲

where wR and hR stand for the width and the height of a region, respectively. 共M / n兲 stands for the width and the height of a basic pattern. Two regions should be far enough apart to allow the difference in the intervals between local autocorrelation peaks to be measured. Equations 共12兲 and 共13兲 present the distance between the two parallel regions: 0 艋 dx共A兲 ⬍ M − dx共B兲 dx共A兲 − dx共B兲 ⬎

M n

共horizontal projection兲,

dy共A兲 = dy共B兲 = 0

共12兲

0 艋 dy共A兲 ⬍ M − dy共B兲 dy共A兲 − dy共B兲 ⬎

M n

共vertical projection兲,

dx共A兲 = dx共B兲 = 0

共13兲

where dx共R兲 and dy共R兲 are the distances of the x axis and y axis, respectively, from 共0, 0兲 for the selecting region R for the LACF. The distance from RA to RB and the size of regions are selected adaptively according to the size of the used basic pattern and the bounds of the projective distortion. Using predefined distances and sizes that were determined on the basis of extensive experimentation, the application of the LACF to 共x , y兲 of region R on the estimated watermark pattern W⬘ is modeled as Journal of Electronic Imaging

wR−1

LACFR共x,y兲 =

hR−1

兺 兺 W⬘ i=−w +1 j=−h +1



R

R

+

j 2



i x + dx共R兲 + ,y + dy共R兲 2

2

.

共14兲

Similar to the ACF, the LACF is calculated by FFTbased fast equation as follows: LACFR =

IFFT关FFT共R兲 · FFT共R兲*兴 , 兩R兩2

共15兲

where the * operator denotes complex conjugation, and R is the selecting local parallel region RA or RB. If the watermark is embedded in the host frame, the result yielded by the LACF shows a periodic peak pattern. Then, the geometric distortions are estimated and reversed by using a local autocorrelation peak 共LACP兲. The LACP is detected from the results of the LACF by applying an adaptive threshold as follows: LACP ⬎ ␮LACF + ␣LACF␴LACF ,

共16兲

where ␮LACF and ␴LACF denote the average and standard deviation of the LACF, respectively, ␣LACF is a value that is related to the false positive error rate. Presetting the maximum false positive error rate, we calculate ␣LACF and obtain the threshold. Figure 6 shows the results of applying the LACF results to a watermarked image that had been subjected to rotation and projective distortion. The LACF extracts LACPs from all distorted images. In Fig. 6共c兲 in particular, two LACF results show the different intervals between each autocorrelation peak. Using the LACF results as a basis, we can calculate intervals and angles for estimating geometric distortions. 4.2 Estimation of Distortion and Restoration Due to the fact that the watermark is embedded in only part of the frames, projective distortion makes the watermark not only projected, but also translated 共see Fig. 7兲. It is necessary to construct a mathematical model that takes these distortions into account. In the mathematical model, the intervals and angles between LACPs are used as parameters. For simplicity, it is assumed that the watermark pattern is embedded in the center of the host frame, as shown in Fig. 8. The application of the LACF yields LACPs on the line HN and IO. Let the four corner points of the original watermark pattern be A, B, C, and D, respectively. E is the center point of AD, and P is an intersection point of the extensions of AB⬘ and DC⬘. We let K denote the point at which HN and EP intersect and G denote the point at which IO and EP intersect. J denotes the point at which AP and the extension line of HN intersect, and F denotes the point at which AP and the extension line of IO intersect. From the interval of the LACPs, KG is obtained. Our goal is to obtain the coordinates of projectivedistorted points A, B⬘, C⬘, and D. In this geometry, only the coordinates of B⬘ and C⬘ need to be computed because the positions of A and D do not change. Therefore, it is neces-

023008-6

Downloaded from SPIE Digital Library on 19 Nov 2009 to 143.248.135.186. Terms of Use: http://spiedl.org/terms

Apr–Jun 2009/Vol. 18(2)

Lee et al.: Improved watermark synchronization based on local…

sary to know the length of B⬘L共=LC⬘兲. First, the lengths of the two parallel lines HN and IO are calculated: n HK = IHN ⫻ , 2

共17兲

n IG = IIO ⫻ . 2

共18兲

IXY stands for the interval between peaks on the line XY, and n is the number of times that the basic watermark pattern is repeated. Next, the similarity of triangles ⌬HKP and ⌬IGP is employed to obtain the length of GP. In addition, the length of JK is calculated using the ratio of JK to HK, which is the same as the ratio of the height of the frame hF to the height of the original watermark M: KG ⫻ IG

GP =

JK =

HK − IG

共19兲

,

hF ⫻ HK . M

共20兲

The length of EK is obtained using KP, JK, and the similarity of triangles ⌬JKP and ⌬AEP. EK = AE ⫻

KP JK

共21兲

− KP,

where AE = hF / 2. Using Eqs. 共19兲–共21兲, B⬘L共=LC⬘兲 is defined as follows: B ⬘L =

hF ⫻ 共EP − wF兲 2 ⫻ EP

共22兲

,

where wF stands for the width of the frame. Four pairs of points in Fig. 8 are transformed as follows: A共0,0兲 ⇒ A共0,0兲, B共wF,0兲 ⇒ B⬘共wF,hF/2 − B⬘L兲, C共wF,hF兲 ⇒ C⬘共wF,hF/2 + B⬘L兲, D共0,hF兲 ⇒ D共0,hF兲.

共23兲

Nine coefficients of projection matrix H in Eq. 共1兲 are obtained by substituting the preceding coordinates as follows: a=

hF 2B⬘L

Fig. 6 Watermarked image and extracted peaks by LACF: 共a兲 no attack, 共b兲 rotation 20 deg 共c兲 horizontal projection 20 deg.

,

b = c = f = h = 0,

d=

e = i = 1,

共hF兲2 − 2 ⫻ B⬘L ⫻ hF 4 ⫻ B ⬘L ⫻ w F

,

Journal of Electronic Imaging

g=

h F − 2 ⫻ B ⬘L 2 ⫻ B ⬘L ⫻ w F

.

023008-7

Downloaded from SPIE Digital Library on 19 Nov 2009 to 143.248.135.186. Terms of Use: http://spiedl.org/terms

共24兲 Apr–Jun 2009/Vol. 18(2)

Lee et al.: Improved watermark synchronization based on local…

(a)

(b) Fig. 9 Snapshot examples of test videos.

(c)

(d)

Fig. 7 Change in the position of the watermark due to the projection. 共a兲 Location of the original watermark. 共b兲 Location of the watermark after horizontal projection 10 deg. 共c兲 Comparison of the positions of the watermark, prior to and after projection. 共d兲 Scaled version of 共c兲.

Last, the watermark pattern is recovered from the geometric distortion using the inverse matrix H−1. Due to the fact that the matrix H is a nonsingular matrix by the definition of projective transformation,15 the inverse of H is always obtained. 5 Experimental Results On the HD-resolution clips of digital cinema shown in Fig. 9, the fidelity and robustness are measured against both various geometric distortions and D-A/A-D conversion attack. The LACF-based scheme and the ACF-based scheme were compared in the same environment. A 40-bit payload was embedded into each 5-min clip to adhere to digital cinema initiatives 共DCI兲.3 To enhance performance with respect to robustness, the same watermark bit is embedded for 2 s. As explained in Sec. 3, the 2-D basic pattern whose size is 96⫻ 96 is tiled eight times 共n = 8兲 to the vertical and horizontal axes, respectively. Thus, the watermark pattern is formed in dimensions of 768⫻ 768 and is embedded in the middle of the frame. The factors S0 and S1 in Eq. 共4兲 are set to 7.0 and 1.0, respectively. For the LACF, the parameters in Eq. 共14兲 are set for both horizontal and vertical projection. For horizontal projection, wR is set to 96 共=M / n兲, and hR is set to 768 共=M兲. The dx共A兲 for region RA is set to 144, and the dx共B兲 for region RB is set to 432. Both dy共A兲 and dy共B兲 are set to zero. For vertical projection, wR is set to 768 共=M兲, and hR is set to A

E

B J H

F I

K

G

N

O

D

B’ L C’ C

Fig. 8 Geometry of a horizontally projected image. Journal of Electronic Imaging

P

96 共=M / n兲. The dy共A兲 is set to 144, and the dy共B兲 is set to 432. Both dx共A兲 and dx共B兲 are set to zero. Both ␣c in Eq. 共9兲 and ␣LACF in Eq. 共16兲 are set to 4.67 by setting the false positive error rate to 10−6. After embedding, the average peak signal-to-noise ratio 共PSNR兲 was 44.1 dB for the test videos. Fidelity was tested as described in Ref. 4. Clips were projected onto a wide screen using an EPSON EMP-TW1000 projector. The projected clips were about 2.20 m and 1.24 m in the horizontal and vertical directions, respectively. Four expert observers participated in a two-alternative, forced-choice experiment in which each trial consisted of two presentations of the same clip, once with and once without the watermark present. Observers viewed the screen from two picture heights and were asked to indicate which clips contained the watermark. Each source clip was played four times in each trial. Each trial lasted five minutes. No observer could determine the identity of the watermarked clip with certainty in any case. Robustness against projection transform and D-A/A-D conversion attacks is analyzed mainly because our previous works14,16 addressed robustness against signal-processing distortions and geometric distortions in Stirmark 4.1.20 To assess the performance of our scheme, the concept of “extraction rate” is introduced as a measure of performance. From the next section on, the performance is measured using the extraction rates. 5.1 Projection Attack Videos that were watermarked using the LACF-based and ACF-based methods were projected onto the screen and camcorded from different angles to make horizontally projected, vertically projected, and combined projected videos. Comparison of the two methods’ performance against projective distortions is plotted in Fig. 10. Figures 10共a兲 and 10共c兲 show the extraction rate for each projection attack, which is given by the percentage of the total number of video frames to watermark extracted frames correctly. Figures 10共b兲 and 10共d兲 show the normalized crosscorrelation value when the watermark is extracted. When the angle of projection exceeded 40 deg, more than half the area of the embedded watermark was translated outside the original embedding position. Due to the fact that we were concerned only about the original embedding position, the watermark outside the specific window was lost during watermark detection. In horizontal projection, the presented LACF-based scheme extracted the message until the angle of projection reached 8 deg, while the ACF-based scheme failed to extract the message in any case except that there is no attack. Due to the fact that the ratio of the width to the

023008-8

Downloaded from SPIE Digital Library on 19 Nov 2009 to 143.248.135.186. Terms of Use: http://spiedl.org/terms

Apr–Jun 2009/Vol. 18(2)

Lee et al.: Improved watermark synchronization based on local…

Fig. 10 Comparison of the performance of the ACF and LACF schemes against projective distortions. 共a兲 Vertical projection; 共b兲 vertical projection; 共c兲 horizontal projection; 共d兲 horizontal projection.

height of video frames is different according to the resolution of video format, the range of angles within which the LACF-based scheme can extract the message is also different in horizontal and vertical projection. In the HDresolution video, the frame width is almost twice the frame height 共e.g., 1920⫻ 1080兲. Hence, at the same angle of projection, horizontal projection causes significantly more distortion than vertical projection. In practice, combined horizontal and vertical projection occurs. However, because we exploit the fact that the change in the length of lines that are orthogonal to the projection axis is regular, the presented LACF-based scheme is sensitive to the combined projective attacks. Table 1 summarizes the results for the performance of the LACF-based scheme against combined projection. When the combined projection was slight, the watermark was extracted correctly. However, when the angle of vertical projection exceeded 6 deg or the angle of horizontal projection exceeded 8 deg, the LACF-based scheme failed. Table 2 summarizes the results for the performance of the two schemes against geometrical distortions, including Journal of Electronic Imaging

affine transform. As shown in Fig. 10, the LACF-based scheme correctly extracted the embedded 40-bit payload. The ACF-based scheme could not extract the watermark when any projective distortion occurred. In all cases, the correlation value of the LACF-based scheme exceeded the threshold of 10−6 for the probability that a false positive would occur. In addition, the extraction rate of the LACFbased scheme was higher than that of the ACF-based scheme, except for rotation. When the angle of rotation exceeded 17 deg, the LACF-based scheme failed to extract the payload because the LACF region could not cover the distorted peaks. We limited the width of the LACF region to a comparatively small size in order to recover from projective distortions accurately. However, large distortions can be restored by using larger LACF regions before extracting LACF peaks. 5.2 D-A/A-D Conversion Robustness against D-A/A-D conversion, which included projective and affine transforms and signal-processing dis-

023008-9

Downloaded from SPIE Digital Library on 19 Nov 2009 to 143.248.135.186. Terms of Use: http://spiedl.org/terms

Apr–Jun 2009/Vol. 18(2)

Lee et al.: Improved watermark synchronization based on local… Table 1 Snapshot, extraction rate, and normalized cross-correlation of the proposed scheme against combined projective attacks 共fp = 10−6兲.

Table 3 Extraction rate and normalized cross-correlation of the proposed scheme against D-A/A-D conversion 关fp = 10−6兴. Attack

Extraction rate

Correlation/threshold

D-A/A-D conversion

0.70

0.33/ 0.11

D-A/A-D conversion with MPEG-1 conversion

0.76

0.31/ 0.11

D-A/A-D conversion with MPEG-4 conversion

0.72

0.31/ 0.11

D-A/A-D conversion with frame rate 30 fps to 24 fps

0.67

0.31/ 0.11

D-A/A-D conversion with scaling to 640⫻ 480

0.62

0.31/ 0.13

Vertical projection 2◦

4◦

6◦

Snapshot 2◦ Extraction rate

1.00

0.58

0.17

Correlation / Threshold

0.42 / 0.13

0.25 / 0.12

0.17 / 0.10

Horizontal Snapshot

projection 4◦

Extraction rate

0.97

0.58

0.00

Correlation / Threshold

0.46 / 0.13

0.25 / 0.12

0.14 / 0.12

Snapshot 6◦ Extraction rate

0.61

0.23

0.00

Correlation / Threshold

0.35 / 0.12

0.20 / 0.12

0.10 / 0.08

Snapshot 8◦ Extraction rate

0.58

0.18

0.00

Correlation / Threshold

0.25 / 0.12

0.17 / 0.10

0.10 / 0.08

tortions, was tested. Clips were projected on a screen with the same environment as the fidelity test and captured with a tripod-mounted SONY HDR-FX1 camcorder. The camcorder was located approximately three picture heights away from the screen. The results for the extraction rate and normalized crosscorrelation are summarized in Table 3. In all cases, the 40-bit payloads were extracted with the 100% reliability, where average correlation values were larger than the threshold for the false positive probability. Through experiments, the presented scheme was robust against D-A/A-D conversion and other attacks that are sometimes carried out following D-A/A-D conversion in real video processing.

6 Conclusion Given that many pirated copies of digital video content are geometrically distorted, a watermark for application in digital cinema should survive such distortions. We proposed a watermarking scheme that is robust against geometric distortions, including projective transform and affine transform. We showed by experiment that our proposed LACF is robust against projective distortions 共vertical, horizontal, and combined projection兲, as well as frame-rate change, format conversion, and D-A/A-D conversion for video applications. Moreover, the proposed scheme was designed to embed the watermark in real time for application in digital cinema. Our work can be extended to focus on combined projection attacks and can be improved to be suitable for use in various illumination environments. Acknowledgements This work was supported by the Korea Science and Engineering Foundation 共KOSEF兲 grant funded by the Korea government 共MEST兲 共No. R0A-2007-000-20023-0兲, and Ministry of Culture, Sports and Tourism 共MCST兲 and Korea Culture Content Agency 共KOCCA兲 in the Culture Technology 共CT兲 Research & Development Program 2009. References

Table 2 Extraction rate and normalized cross-correlation of the ACF and LACF scheme against various geometric distortions 关fp = 10−6兴. Extraction rate

Correlation/threshold

Attack

ACF

LACF

Original 共1920⫻ 1080兲

1.00

0.99

0.67/ 0.11 0.68/ 0.11

Rotation 17 deg

0.98

0.47

0.71/ 0.13 0.73/ 0.14

Scaling to 480⫻ 270

0.28

0.63

0.30/ 0.14 0.27/ 0.14

Horizontal projection 8 deg

0.00

1.00

0.12/ 0.10 0.64/ 0.14

Vertical projection 40 deg

0.00

0.74

0.12/ 0.10 0.58/ 0.14

Translation 30%

0.54

0.83

0.17/ 0.08 0.17/ 0.08

Journal of Electronic Imaging

ACF

LACF

1. S. Vural, H. Tomii, and H. Yamauchi, “Robust digital cinema watermarking,” Int. J. Information Technol. 1, 255–259 共2005兲. 2. A. Leest, J. Haitsma, and T. Kalker, “On digital cinema and watermarking,” Proc. SPIE 5020, 526–535 共2001兲. 3. Digital Cinema Initiatives, LLC, Digital Cinema System Specification Version 1.2 共2008兲, http://www.dcimovies.com/ DCIDigitalCinemaSystemSpecificationv1_2.pdf. 4. J. Lubin, J. Bloom, and H. Cheng, “Robust, content-dependent, highfidelity watermark for tracking in digital cinema,” Proc. SPIE 5020, 536–545 共2003兲. 5. D. Delannay, J. Delaigle, B. Macq, and M. Barlaud, “Compensation of geometrical deformations for watermark extraction in the digital cinema application,” Proc. SPIE 4314, 149–157 共2003兲. 6. J. O. Ruanaidh and T. Pun, “Rotation, scale, and translation invariant spread spectrum digital image watermarking,” Signal Process. 66, 303–317 共1998兲. 7. C. Lin, J. Bloom, I. Cox, M. Miller, and Y. Lui, “Rotation, scale, and translation-resilient watermarking for images,” IEEE Trans. Image Process. 10, 767–782 共2001兲. 8. M. Alghoniemy and A. Tewfik, “Geometric distortion correction in image watermarking,” Proc. SPIE 3971, 82–89 共2000兲. 9. P. Bas, J. Chassery, and B. Macq, “Geometrically invariant watermarking using feature points,” IEEE Trans. Image Process. 11, 1014– 1028 共2002兲.

023008-10

Downloaded from SPIE Digital Library on 19 Nov 2009 to 143.248.135.186. Terms of Use: http://spiedl.org/terms

Apr–Jun 2009/Vol. 18(2)

Lee et al.: Improved watermark synchronization based on local… 10. S. Pereira and T. Pun, “Robust template matching for affine resistant image watermarks,” IEEE Trans. Image Process. 9, 1123–1129 共2000兲. 11. M. Kutter, “Watermarking resisting to translation, rotation, and scaling,” Proc. SPIE 3528, 423–431 共1998兲. 12. S. Voloshynovskiy, R. Deguillaume, and T. Pun, “Multibit digital watermarking robust against local nonlinear geometrical distortions,” Proc. ICIP 3, 999–1002 共2001兲. 13. D. Bogumil, “Reversing global and local geometrical distortions in image watermarking,” in Int. Workshop on Information Hiding, Lecture Notes in Computer Science vol. 3200, pp. 25–37, Springer, New York 共2004兲. 14. K.-S. Kim, H.-Y. Lee, D.-H. Im, and H.-K. Lee, “Practical, real-time, and robust watermarking on the spatial domain for high-definition video contents,” IEICE Trans. Inf. Syst. E91-D, 1359–1368 共2008兲. 15. R. I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, Cambridge, UK 共2004兲. 16. C. H. Lee and H. K. Lee, “Improved autocorrelation function based watermarking with side information,” J. Electron. Imaging 14, 1–13 共2005兲. 17. J. Huang, Y. Q. Shi, and Y. Shi, “Embedding image watermarks in dc components,” IEEE Trans. Circuits Syst. Video Technol. 10, 974–979 共2000兲. 18. S. Voloshynovskiy, A. Herrigel, N. Baumgartner, and T. Pun, “A stochastic approach to content adaptive digital image watermarking,” in Proc 3rd Int. Workshop on Information Hiding, Lecture Notes in Computer Science vol. 1768, pp. 212–236, Springer, Berlin 共1999兲. 19. I. G. Karybali and K. Berberidis, “Efficient spatial image watermarking via new perceptual masking and blind detection schemes,” IEEE Trans. Inf. Forensics Security 1, 256–274 共2006兲. 20. F. A. P. Petitcolas, “Watermarking schemes evaluation,” Signal Process. 17, 58–64 共2000兲.

Tae-Woo Oh received his BS degree in computer engineering from Ajou University, Korea, in 2007, and his MS degree in computer science from Korea Advanced Institute of Science and Technology 共KAIST兲 in 2009. He is currently working toward his PhD degree in the Multimedia Computing Laboratory, Department of Electrical Engineering and Computer Science, KAIST. His research interests include image/video watermarking, fingerprinting, and multimedia signal processing. Hae-Yeoun Lee received his MS and PhD degrees in computer science from Korea Advanced Institute of Science and Technology 共KAIST兲 in 1997 and 2006, respectively. From 2001 to 2006, he was with Satrec Initiative, Korea. From 2006 to 2007, he was a postdoctoral researcher at Weill Medical College, Cornell University. He is now with Kumoh National Institute of Technology, Korea. His major interests are digital watermarking, image processing, remote sensing, and digital rights management.

Min-Jeong Lee received a BS degree in computer engineering from Kyungpook National University, Korea, in 2006, and an MS degree in computer science from Korea Advanced Institute of Science and Technology 共KAIST兲 in 2008. She is currently pursuing a PhD degree in the Multimedia Computing Laboratory, Department of Electrical Engineering and Computer Science, KAIST. Her research interests are focused on image/video watermarking, with particular attention to multimedia forensics, and information security.

Heung-Kyu Lee received his BS degree in electronics engineering from Seoul National University, Korea, in 1978, and MS and PhD degrees in computer science from Korea Advanced Institute of Science and Technology 共KAIST兲 in 1981 and 1984, respectively. Since 1986, he has been a professor in the Department of Computer Science, KAIST. His major interests are digital watermarking, digital fingerprinting, and digital rights management.

Kyung-Su Kim received his BS degree in computer engineering from Inha University, Korea, in 2005, and his MS degree in computer science from Korea Advanced Institute of Science and Technology 共KAIST兲 in 2007. He is currently working toward his PhD degree in the Multimedia Computing Laboratory, Department of Electrical Engineering and Computer Science, KAIST. His research interests include image/video watermarking and fingerprinting, error concealment methods, information security, multimedia signal processing, and multimedia communications.

Journal of Electronic Imaging

023008-11

Downloaded from SPIE Digital Library on 19 Nov 2009 to 143.248.135.186. Terms of Use: http://spiedl.org/terms

Apr–Jun 2009/Vol. 18(2)