IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 2, FEBRUARY 2007

391

Optimal Signature Design for Spread-Spectrum Steganography Maria Gkizeli, Member, IEEE, Dimitris A. Pados, Member, IEEE, and Michael J. Medley, Senior Member, IEEE

Abstract—For any given host image or group of host images and any (block) transform domain of interest, we find the signature vector that when used for spread-spectrum (SS) message embedding maximizes the signal-to-interference-plus-noise ratio (SINR) at the output of the corresponding maximum-SINR linear filter. We establish that, under a (colored) Gaussian assumption on the transform domain host data, the same derived signature minimizes host distortion for any target message recovery error rate and maximizes the Shannon capacity of the covert steganographic link. Then, we derive jointly optimal signature and linear processor designs for SS embedding in linearly modified transform domain host data and demonstrate orders of magnitude improvement over current SS steganographic practices. Optimized multisignature/multimessage embedding in the same host data is studied as well. Index Terms—Authentication, covert communications, data hiding, distortion, linear filters, signal-to-interference-plus-noise ratio (SINR), spread spectrum, steganography, watermarking.

I. INTRODUCTION TEGANOGRAPHY is the process of embedding a “secret” digital signal (hidden message) in another digital signal (image or audio) called “cover” or “host.” Unlike general digital watermarking applications, steganography attempts to establish covert communication between trusting parties and imposes the requirement of concealing the existence of the embedded message. As in any watermarking operation, the first step in the design of a steganographic system is to determine the embedding process. This is a crucial task since host-carrier properties, message detector design and performance depend directly on the way the message is inserted in the host data. While each watermarking application has its own individual requirements

S

Manuscript received December 21, 2004; revised July 2, 2006. This work was supported by the U.S. Air Force Research Laboratory under Agreements F30602-03-2-0023 and FA8750-04-1-0091. This paper was presented in part at the IEEE International Conference on Image Processing (ICIP), Singapore, October 2004. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Benoit Macq. M. Gkizeli was with the Department of Electrical Engineering, State University of New York at Buffalo, Buffalo, NY 14260 USA. She is now with the Department of Electronics, Technological Education Institute of Crete, Chania, 73133 Greece (e-mail: [email protected]). D. A. Pados is with the Department of Electrical Engineering, State University of New York at Buffalo, Buffalo, NY 14260 USA (e-mail: [email protected] eng.buffalo.edu). M. J. Medley is with the Department of Electrical Engineering, State University of New York Institute of Technology, Utica, NY 13504 USA (e-mail: [email protected]). Digital Object Identifier 10.1109/TIP.2006.888345

[1], [2], the broad common objective of most steganographic applications is a satisfactory tradeoff between hidden message resistance to noise/disturbance, information delivery rate, and host distortion for concealment purposes. Message embedding can be performed either directly in the time (audio) or spatial (image) domain [3]–[7] or in a transform domain (for example, for images we may consider full-frame discrete Fourier transform (DFT) [8]–[12], full-frame discrete cosine transform (DCT) [13], block DFT or DCT [14]–[18], or wavelet transforms [19]–[21]). Direct embedding in the original host signal domain may be desirable for system complexity purposes, while embedding in a transform domain may take advantage of the particular transform domain properties [23]. In this paper, we focus our attention on transform domain spread-spectrum (SS) embedding methods for image steganography. In a broad sense, any steganographic system for which the secret signal is spread over a wide range of host image frequencies can be referred to as an SS embedding system. Once the transform embedding domain is selected, the hidden message can be applied to the host data through an additive [8], [12], [14]–[17], [22] or multiplicative [9]–[11] rule. In the literature, additive SS embedding methods either directly apply (add) the message/watermark to several host coefficients [17] or in direct analogy to SS digital communications systems use an amplitude modulated signature to deposit one information symbol across a group of host data coefficients [8], [12], [14], [16] or a linearly transformed version of the host data coefficients [22]. SS embedding algorithms for blind image steganography (that is, hidden message recovery without knowledge of the original image) have been based on the understanding that the host signal acts as a source of interference to the secret message of interest. Yet, it should also be understood that this interference is known to the message embedder. Such knowledge can be exploited appropriately to facilitate the task of the blind receiver at the other end and minimize the recovery error rate for a given host distortion level, minimize host distortion for a given target recovery error rate, maximize the Shannon capacity of the covert steganographic channel, etc. Indeed, if we were to place the steganography application in an information theoretic context, it could be viewed as a communications-with-side-information problem [24]–[26]. Optimized embedding methods can facilitate host interference suppression at the receiver side when knowledge of the host signal is adequately exploited in system design. In this paper, for any given image (or set of images), (block) transform domain, and host bins, we derive the additive embedding signature that maximizes the output signal-to-interference-plus-noise ratio (SINR) of the linear maximum-SINR re-

1057-7149/$25.00 © 2007 IEEE

392

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 2, FEBRUARY 2007

ceiver filter.1 We show that under a (colored) Gaussian assumption on the host bins, this same signature minimizes the receiver bit-error rate (BER) at any mean-square (MS) image distortion level, minimizes—conversely—the MS image distortion at any target BER, and maximizes the Shannon capacity of the covert link. We then generalize our findings and present a novel joint signature and linear host data projection optimization scheme. In this present work, we consider only scalar parametrized host data projection as in [22]. Finally, we extend signature-only as well as joint signature and host-projection optimization to multiuser (multiple-signature) embedding. Our emphasis is directed primarily toward low complexity, sequential, conditional multiuser optimization. The rest of the paper is organized as follows. Section II presents the core signature and embedding optimization results. These results are generalized to multiple signature embedding in Section III. Section IV is devoted to experimental studies and comparisons. A few concluding remarks are drawn in Section V. II. SIGNATURE OPTIMIZATION FOR SS EMBEDDING In this section, we develop an optimized SS steganographic system. To draw a parallelism with conventional SS communications systems, in SS watermark (message) embedding the watermark can be regarded as the SS signal of interest transmitted through a noisy “channel” (the host). The disturbance to the SS signal of interest is the host data themselves plus potential external noise due to physical transmission of the watermarked data and/or processing/attacking. The purpose of the watermark detector is to withstand the influence of the total end-to-end disturbance and recover the original hidden message.

H2f

g

Fig. 1. (a) Baboon image example 0; 1; . . . ; 255 data autocorrelation matrix (8 8 DCT, 63-bin host).

2

. (b) Host

A. Signal Model and Notation Consider a host image that is to be wais the image alphabet and termarked where is the image size in pixels. Fig. 1(a) shows a grayscale baboon image example in . is partitioned into Without loss of generality, the image local blocks of size pixels. Each block is to carry one hidden information bit , respectively. Embedding is performed in a real 2-D transform domain . After transform calculation and conventional zig-zag scanning vectorization, . From the we obtain transform domain vectors we choose a fixed subset coefficients (bins) to form the final host of (for example, it is common vectors , from and appropriate to exclude the dc coefficient, the host ). The autocorrelation matrix of the host data is an important statistical quantity for our developments and is defined as follows: 1The problem—and solution—parallels the eigen-signature design for code-division multiple-access (CDMA) wireless communications [28]. In SS steganography, however, the disturbance (transform-domain host) is readily available at the embedder for manipulation and statistical exploitation and in contrast to the CDMA problem may be plausibly characterized as Gaussian.

(1) denotes statistical expectation (here, with respect where to over the given image ) and is the transpose operator. It is easy to verify that in general , where is the size-L identity matrix; that is, is not constant-value diagonal or “white” in field language. For example, 8 8 DCT with 63-bin host data formation (exclude only the dc coefficient) for the baboon image in Fig. 1(a) gives the host autocorrelation in Fig. 1(b). matrix B. Signature Optimization Consider direct additive SS embedding of the form (2) where

is the message bit embedded in the host is the bit amplitude, , is the (normalized) embedding signature to be designed, and represents potential external white Gaussian noise2 of variance 2Additive white Gaussian noise is frequently viewed as a suitable model for quantization errors, channel transmission disturbances, and/or image processing attacks.

GKIZELI et al.: OPTIMAL SIGNATURE DESIGN

393

. The mean-squared distortion of the original image due to the embedded data only is

is the optimum (minimum probability of error) bit detector [30] with probability of error

(3) and total disturbance in (2), With signal of interest the linear filter that operates on and offers maximum SINR at its output [27] is

(4)

(10) . We see that is a where . If we now view monotonically decreasing function of as a function of the embedding signature , then from Proposition 1 and (10), when the probability of error of the optimum detector is minimized to

The exact maximum output SINR value attained is

(11) and optimum detection reduces to (12) (5)

as a function of the embedding We propose to view , and identify the signature that maxisignature mizes the SINR at the output of the maximum SINR filter. Our findings are presented in the form of a proposition below that parallels the developments for code-division-multiple-access (CDMA) codeword optimization in [28] and [29]. The proof is straightforward and omitted. Proposition 1: Consider additive SS embedding according to be eigenvectors of in (1) with cor(2). Let . For any hidden responding eigenvalues message-induced distortion level , a signature that maximizes the output SINR of the maximum SINR filter is (6) When

, the output SINR is maximized to

(7) and maximum SINR data filtering simplifies to (8)

Conversely, for any preset probability of error level minimizes the host distortion due to the hidden message to (13) We conclude that under a Gaussian transform-domain host data assumption, the “minimum” eigenvector of the host data autocorrelation matrix, when used as the embedding signature, allows message recovery with the minimum possible and it does so by trivial signature BER (eigenvector) matched filter detection. Conversely, the mesis minimized for any given sage-induced image distortion target BER. If necessary, further BER improvements below for any fixed distortion can be attained via error correcting coding of the information bits at the expense of reduced information bit payload. The maximum possible host payload in bits that still allows—theoretically for asymptotically large number of image blocks —message where recovery with arbitrarily small probability of error is is the Shannon capacity of the covert link identifies in bits per embedding attempt. We recall that the information conveyed about the input variable by the received vector and denotes the input variable probability distribution function. For Gaussian host data and average image distortion constraint , we can calculate [31] (14)

In summary, Proposition 1 says that the “minimum” eigenvector of the host data autocorrelation matrix, when used as the embedding signature, sends the output SINR to its max. At the same time, maximum imum possible value SINR filtering becomes plain signature (eigenvector) matched filtering. If, in addition, we are allowed to assume that is Gaussian, , then (9)

where is the determinant operator. We can show that the signature choice is also optimal in terms of capacity, of the covert link, and, therei.e., maximizes the capacity for the host vectors fore, the maximum allowable payload . The result is presented in the form of Proposition 2 below whose proof is given in the Appendix. Proposition 2: Consider additive SS embedding by (2) with . Let be eigenvectors of with . For any corresponding eigenvalues

394

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 2, FEBRUARY 2007

hidden message-induced distortion level , the signature maximizes the covert channel capacity to

bits per symbol embedding

(signature , parameter ) that maximizes the output SINR of the maximum SINR filter is

(20)

(15) and

C. Signature Optimization for Linearly Transformed Host Data

(21)

In an effort to further reduce host distortion for a given target probability of error during message recovery, we modify the previous developments and attempt to steer the host data vectors operator of away from the embedding signature using an [22] where both the parameter and the form the signature , are to be designed.3 In parallel to (2), the composite signal now becomes

When

and

, the output SINR is maximized to (22)

and maximum SINR data filtering simplifies to

(16) and the mean-squared distortion due to the embedding operation only is (17) We observe that, in contrast to (3), the distortion level is controlled not only by but by and , as well. With signal of and total disturbance in (16), the interest linear filter that operates on and offers maximum SINR at its output is

(18) The exact maximum output SINR value attained is

(23) The target distortion is achieved when the bit amplitude is set . at Proposition 3 shows that the optimum signature assignand maximum ment is still the “minimum” eigenvector of SINR filtering still reduces to plain signature (eigenvector) matched filtering. The optimum selection of depends on the , the noise variance , and minimum eigenvalue of the target distortion level . The optimum pair allows the output SINR to attain its maximum possible value . From (21), it is interesting and to note that as the allowed distortion the embedding scheme converges to the conventional scheme , the linear operator of Section II-B. As converges to the orthogonal projector and the allowable distortion is budgeted more toward host-data modification and less toward the embedding bit amplitude. , then If we assume that is Gaussian, is the optimum bit detector with probability of error

(19) In the following, we look at as a function of both the , and embedding signature and the parameter identify the signature and parameter values that jointly maximize the SINR at the output of the maximum SINR filter. Our findings are presented in the form of a proposition below. The proof is given in the Appendix. Proposition 3: Consider additive SS embedding in linearly be eigenvectransformed host data by (16). Let with corresponding eigenvalues tors of . For any hidden message-induced distortion level , a pair 3If

c

= 0, we revert to the developments of Section II-B. If c = 1; I

(24) As in the plain additive SS embedding scenario, if is Gaussian then the probability of error is a monotonically decreasing func. The pair which maximizes tion of the output SINR of the maximum-SINR filter is also minimizing the probability of error of the optimum detector to

(25)

0

becomes the projector orthogonal to s. In this work, as in [22], we confine css ourselves within this class of scalar parametrized linear operators merely for mathematical convenience and tractability.

and optimum detection reduces to

.

GKIZELI et al.: OPTIMAL SIGNATURE DESIGN

395

We can show that for Gaussian transform-domain host data , the covert channel capacity is given by

denotes the “exclude- ” data autocorrelation matrix, where that is the autocorrelation matrix of the disturbance to message defined as

(26) assignment of ProposiThen, we can prove that the tion 3 is also optimal in terms of capacity. This result is summarized in the following proposition whose proof is given in the Appendix. Proposition 4: Consider additive SS embedding by (16) with . Let be eigenvectors of with corresponding eigenvalues . For any hidden message-induced distortion level , the pair of (20) and (21) maximizes the covert channel capacity to

(32) The exact maximum output SINR value attained is (33)

(27) bits per symbol embedding. In the following section, we consider the problem of data embedding in one host with multiple signatures.

III. MULTISIGNATURE EMBEDDING We may generalize the signal model in (2) to cover multisignature/multimessage embedding of the form

As in Section II for single-message embedding, we propose as a function of the embedding signature to view , and identify the signature vector that maximizes the SINR value. Our findings are presented in the form of Proposition 5 below and parallel the developments of Proposition 1 for the single-message case. Proposition 5: Consider additive SS embedding by (28). be eigenvectors of in (32) with corLet . For any responding eigenvalues , a signature that maximessage-induced distortion level mizes the output SINR of the maximum SINR filter is (34)

(28) When coming potentially from distinct where bits messages, are embedded simultaneously in with correand embedding signatures sponding amplitudes . Thus, the contribution of each individual embedded message bit to the composite watermarked signal is and the mean-squared distortion to the original host data due to the embedded message alone is

, the output SINR is maximized to (35)

and total disturbance With signal of interest in (28), the linear filter that operates on and offers maximum SINR at its output is

and maximum SINR data filtering simplifies to . In summary, Proposition 5 says that the “minimum” eigenvector of the disturbance autocorrelation matrix when used as the embedding signature allows the output SINR to attain its . At the same time, maximum possible value maximum SINR filtering becomes plain signature (eigenvector) matched filtering. In the context of multisignature optimization, for fixed-bit and arbitrary signature initializaamplitude values , consider repeated tion . In such an applications of Proposition 5 for eigen-update signature cycle each signature is replaced by the minimum-eigenvalue eigenvector of the disturbance autocorrelation matrix as seen by the message corresponding to that signature. Once all signatures are updated, a second update cycle may begin. The whole procedure may continue for a predetermined number of cycles or until convergence

(31)

(36)

(29) Under a statistical independence assumption across message bits, the mean-squared distortion of the original image due to the total multimessage insertion is (30)

396

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 2, FEBRUARY 2007

It can be proven that convergence of (36) is guaranteed and, as shown in a CDMA literature context [28], at each cycle the genof the signature set eralized total-squared correlation

and maximum SINR data filtering simplifies to

where , is nonincreasing.4 If we consider channel coding the message bits before embedding and assume that the host in (28) is Gaussian, the steganographic system determined by the signature matrix is a special case of the -user Gaussian multiple access channel with average input distortion constraints [31], [33]. For such a channel, (defined as the maximum sum of mesthe sum capacity sage coding rates at which messages can be recovered reliably [33], [34]) is a reasonable criterion of quality for the signature set and equals

If, in addition to the assumptions of Proposition 6, we are al, then the optimum detector lowed to assume that for the bit of interest is with probability of error . As a simple illustration of the use of Proposition 6 for conditionally optimal multisignature design, suppose that we want to embed in the message bits host data vector with fixed corresponding amplitudes (mean) and squared distortions to be chosen. By Proposition 1 of Secsignatures equal to the bottom eigenvector of tion II, we set . Given and under the constraint that we search for an orthogonal to , by Proposition 6, we design which is the next available eigenvector of from the bottom. and under the constraint that we Given search for an orthogonal to both and , by Proposition . We continue this way until the final 6, we assign . Once again, a welcome side efassignment fect of this conditionally SINR optimal signature design procedure is that the maximum SINR receiver for each message bit , simplifies to a matched filter and requires no knowledge of other system parameters. For (fixed) unequal embedding amplitude values , the exact order by which the eigenvectors of are drawn to become signatures is important if we consider the sum capacity of the steganographic system. We can show that a necessary condition for a maximum sum capacity solution under the constraint of eigenvector assignment is that the ordering of the bit amplitudes be inversely proportional to the ordering of the eigenvalues of the corresponding signature eigenvectors. This statement is given below in the form of a lemma. The proof can be found in the Appendix. Lemma 1: Consider additive SS embedding by (28) with and let be eigenvectors of with cor. Without loss of responding eigenvalues generality assume that . Then

(37) Minimization of the metric translates to maximization of [34]. However, decrease in at each cycle of the algorithm in (36) does not necessarily imply increase of as seen for instance in [35]–[38] via binary signature examples. Apart from global optimality limitations, the multicycle multisignature optimization procedure in (36) requires recalculation of the disturbance autocorrelation matrix and eigen decomposition at each step of each cycle. A simple low-cost alternative to (36) could be a conditionally optimal single-cycle design method based on the following proposition. Proposition 6: Consider additive SS embedding according to (38) be eigenvectors of in (1) with correLet sponding eigenvalues . If the signatures are eigenvectors of , then for , a signaany message-induced distortion level that maximizes the output SINR of the maximum ture SINR filter for the bit of interest subject to the constraint is

(41)

(39) where is the minimum-eigenvalue eigenvector of available. , the output SINR is (conditionally) maxiWhen mized to (40) 4Yet, there is no guarantee that TSC (S) will converge to its minimum possible value (global minimum) [29], [32].

If we generalize our approach and view the individual amplitudes/distortions as design parameters themselves, then we can search for the optimal amplitude assignment that maximizes sum capacity subject to a total allowable distortion constraint . We derive the optimal amplitude values in the lemma below. The proof is given in the Appendix. Lemma 2: Consider additive SS embedding by (28). Let be eigenvectors of with corresponding . If the signatures aseigenvalues are distinct eigenvectors of sociated with the message bits then the sum

GKIZELI et al.: OPTIMAL SIGNATURE DESIGN

397

capacity is maximized subject to an expected total distortion if constraint (42) where and is the Kuhn–Tucker coefficient [31] chosen such that the distortion constraint is met. To find the necessary parameter value in (42), we suggest in to arrange the participating eigenvalues . Then ascending order

be eigenvectors of in (1) with correLet . Assume that sponding eigenvalues , are all eigenvectors of and is the index of the minimum-eigenvalue eigenvector of available. For any , an given message-induced distortion level pair that maximizes the output SINR of the maximum SINR is filter subject to the constraint

(43) where the cutoff index is the greatest integer in and . The optimal message amplitude/distortion allocation solution of Lemma 2 can be viewed as a power waterfilling procedure [31] in the eigen domain of the host. Finally, as the last technical development in this paper, we examine the possibility of carrying out multisignature embedding in linearly transformed host data. We assume that the host data vector is linearly transformed by an operator of the form where and , are the parameters and signatures to be designed. The final composite signal is

(47) and

(48) and When tionally) maximized to

, the output SINR is (condi-

(49) (44) and the mean-squared distortion induced by each individual message , is

(45) With signal of interest trix of the disturbance is

and maximum SINR data filtering simplifies to . If in addition to the assumptions of Proposition 7, we are al, then the optimum detector lowed to assume that for the bit of interest is with probability of error (50)

, the autocorrelation ma-

. Unfortunately, in contrast to (38) for multisignature embedding in nontransremains a function of (as well as ). In formed data, this context, unconditionally optimal multisignature multicycle optimization along the lines of (36) is practically an unrealistic objective. Instead, we suggest to design sequentially the amplitudes , parameters , and signatures of the embedded such that the output SINR messages is conditionally maximized given all past fixed embeddings (single-cycle optimization). Our developments are presented in the form of Proposition 7 below whose proof is given in the Appendix. Proposition 7: Consider additive SS embedding according to

(46)

As an illustrative example of the use of Proposition 7, suppose that we would like to embed in the host data vector message bits with individual corresponding . We first design mean-squared host distortion the system parameters , and for message bit in the absence of any other message in the host image. According to , the optimal parameter selection is Proposition 7

and by (45) . Next, we proceed with the and optimize and subject to the second message bit and the constraint . Since desired distortion level is already an eigenvector of , Proposition 7 offers the equation shown at the bottom of the next page, and by (45) . We continue calculating signatures , parameters , and amplitudes as above, always subject to the desired distortion and orthogonality between the signature to , be designed and all other prior signatures. Provided that

398

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 2, FEBRUARY 2007

the final set of designed parameters is shown in the equation at . the bottom of the page, and Optimal distortion allocation for sum capacity maximization in multimessage embedding in linearly transformed host data subject to the constraint that all messages are assigned distinct , and subject to a total distoreigenvectors of tion constraint is a joint optimization problem with respect to and . We suggest an iterative solution approach based on Proposition 7 above and Lemma 2 presented earlier in this section. We initially fix the distortions induced by each message and find the optimum parameters according to Proposition 7. Then, we perform optimum amplitude allocation according to Lemma 2. Based on this allocation, we reevaluate all by Proposition 7. We continue until convergence is observed. In the following, we present extensive experimental results that we obtained from the implementation of the developed steganographic algorithms.

IV. EXPERIMENTAL STUDIES To carry out an experimental study of the developments presented in the previous sections, we consider first as a host example the familiar grayscale 512 512 “Boat” image in Fig. 2(a) that has been widely used in the pertinent literature. We perform 8 8 block DCT single-signature embedding over all bins except the dc coefficient. Hence, our signature and we embed bits. For length is the sake of generality, we also incorporate white Gaussian dB. Fig. 3 shows the recovery BER noise of variance under signature matched filter detection as a function of the distortion created by the embedded message over the 0- to 20-dB range for four different embedding schemes: a) SS embedding with an arbitrary signature, b) SS embedding with an arbitrary signature and optimized selection of the host data transformation parameter as in [22] (known as “improved spread-spectrum” or ISS), c) SS embedding with an optimal signature according to Proposition 1, and d) SS embedding with a jointly optimal signature and host data transformaaccording to Proposition 3. The tion parameter demonstrated BER improvement of our joint signature and parameter optimization procedure, in particular, measures in orders of magnitude. Fig. 2(b) shows the Boat image after 20 -embedding of the 4 Kbit message and 3-dB dB

2

Fig. 2. (a) Boat image example (512 512 grayscale). (b) Boat image after 20-dB (s ; c ) embedding of 4 Kbits and additive white Gaussian noise of variance 3 dB.

additive white Gaussian noise disturbance in the block DCT domain. In Fig. 4, we repeat the same experiment of Fig. 3 on the 256 256 grayscale Baboon image in Fig. 1(a) (signature , hidden message of bits, and length additive white Gaussian noise disturbance of variance 3 dB). Comparatively speaking (Boat versus Baboon host or Fig. 3 versus Fig. 4 results), message recovery from the Baboon host appears to be a somewhat more difficult problem. Yet, the

GKIZELI et al.: OPTIMAL SIGNATURE DESIGN

399

Fig. 3. Bit-error rate versus host distortion (Boat image, = 3 dB).

Fig. 5. Bit-error rate versus host distortion (average findings over USC-SIPI image database [41], = 3 dB).

Fig. 4. Bit-error rate versus host distortion (Baboon image, = 3 dB).

Fig. 6. Capacity versus distortion (Baboon image, = 3 dB).

proposed joint signature and host transformation parameter opBER at 20-dB timization scheme maintains a better than host distortion and outperforms the proposed signature-only optimization scheme by about eight orders of magnitude. To address the need for experimental verification of highest credibility, we carried out the experiments of Figs. 3 and 4 over the whole USC-SIPI database [41] of 44 miscellaneous images. Fig. 5 shows the average BER versus distortion for the database. The average database findings are quite similar to the individual Baboon (or Boat) results. In Fig. 6, we return and continue the work with the Baboon host and plot the capacity versus distortion performance curves for the four embedders under consideration. We see, for example, that at 20–dB host distortion the jointly optimized embedder offers 2.7 information bits payload per embedded symbol (suggesting implicitly the suitability of a higher than binary message alphabet). The bit payload number goes down to 1.2 for signature-only optimization, 0.6 for ISS embedding [22], and 0.5 for arbitrary signature embedding.

Next, we consider the problem of multisignature embedding. We keep the Baboon image as the host and wish to hide data blocks/messages of length 1024 bits each with each block/message having its own individual embedding signature. Each message is allowed to cause the same expected distor. Therefore, for station to the host tistically independent messages, the total distortion to the host . As before, for the sake of generality, we is add to the host white Gaussian noise of variance 3 dB. We study five different multisignature embedding schemes: a) Embedding with arbitrary signatures, b) ISS embedding [22], c) multicycle eigen-signature design by (36), d) conditional optiassignment, mization by Proposition 6 (sequential ), and e) conditional optimization by Proposition assignment, ). The results 7 (sequential in Fig. 7 reiterate the importance of optimized host-data manipulation in conjunction with signature optimization. In fact, optimization even the least faunder joint sequential outperforms in recovery BER the most vored message

400

Fig. 7. BER as a function of the per-message distortion K = 15 messages of size 1024 bits each, = 3 dB).

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 2, FEBRUARY 2007

D (Baboon image,

Fig. 9. Sum capacity versus distortion (Baboon image, K = 15; = 3 dB).

Fig. 8. Baboon image after multisignature embedding via Proposition 7 (K = 15 messages of size 1 024 bits each, per-message distortion 20 dB, = 3 dB).

favored

message under sequential signature-only optimization or multicycle signature-only design for per-message distortion values above 18 dB.5 Fig. 8 shows the Baboon image after embedding the fifteen messages (15 1024 , optimizabits) via joint sequential tion with 20-dB per-message distortion (31.8-dB total distortion) and 3-dB variance additive white Gaussian noise. Finally, in Fig. 9, we present sum capacity results when the design and sequential two proposed schemes, sequential design, , employ waterfilling power joint allocation (use of Lemma 2 alone or coupled use of Proposition 7 and Lemma 2, correspondingly). We see, for example, that design offers at 32-dB total distortion the waterfilled information bit payload of about 36 bits per 15 symbols emdesign offers only bedded, while the waterfilled about 12 bits per 15 symbols embedded. In Fig. 10, we repeat the experiment of Fig. 9 over the 44 images of the USC-SIPI database [41]. The relative average sum capacity behavior of the embedders remains the same over the database. In absolute

5The performance of the most favored message (i = 1) for both proposed conditional schemes, s ; c and s ; c = 0, remains the same as in the single message case (see Fig. 4) since the (orthogonal) eigenvector signature assignment completely avoids multimessage interference.

Fig. 10. Sum capacity versus distortion (average findings over USC-SIPI image database [41], K = 15; = 3 dB).

numbers, for all embedders the database images have on the average about ten more information bits payload per 15-symbol embedding than the Baboon image at 32-dB total distortion. V. CONCLUSION We considered the problem of hiding digital data in a digital host image via SS embedding in an arbitrary transform domain. We showed that use of the minimum-eigenvalue eigenvector of the transform domain host data autocorrelation matrix as the embedding signature offers the maximum possible SINR under linear filter message recovery and, conveniently, does so under plain signature correlation (signature matched filtering). If we allow ourselves the added assumption of (colored) Gaussian transform-domain host data, then we see that the above described system as a whole becomes minimum probability of error and maximum Shannon capacity optimal, as well. To take these findings one step further, we examined SS embedding in transform-domain host data that are modified by a

GKIZELI et al.: OPTIMAL SIGNATURE DESIGN

401

parametrized projection-like linear operator. We found the joint signature and parameter values under the optimality scenaria mentioned above. Conveniently, the jointly optimal signature is still the minimum-eigenvalue eigenvector and the SINR optimal linear filter at the receiver side is still the signature correlator. Yet, joint signature and parameter optimization was seen to offer dramatic improvements in SINR, probability of error, and capacity. Finally, we extended our effort to cover multisignature/multimessage embedding. First, under signature-only optimization we developed a computationally costly multicycle eigen-signature design scheme based on the disturbance autocorrelation matrices. The alternative suggestion based on the host data autocorrelation matrix alone and sequential (conditional) eigen-signature optimization is practically much more appealing. A waterfilling amplitude assignment algorithm was developed as well to maximize sum capacity under eigen-signature designs. All multisignature findings were generalized to cover parametrized projection-like modification of the host data with, once again, dramatic improvements in probabilty of error or sum capacity. As a brief concluding remark, image-adaptive signature(s) or signature(s)/parameter(s) optimization as described in this work can be carried out over a set of host images (frames) if desired. The only technical difference is the calculation of the host data autocorrelation matrix which now has to extend over the whole host set. As long as the cumulative host autocorrelation matrix is , significant gains are to be not constant-value diagonal collected over standard nonadaptive SS embedding techniques.

Proof of Proposition 3: For a target distortion value , the in (19) equals and is maximized for . We will show that the second term in (19), , is maximized by , as well. By the matrix inversion lemma term

and (53)

(54) Combining (53) and (54), we obtain

(55)

APPENDIX Proof of Proposition 2: We need to find that maximizes in (14) subject to . Since is a strictly monotonic function

The derivative of the righthandside of (55) with respect to gives

(56)

(51)

Hence, a decreasing function of

is . Yet,

. Therefore

Using the rank-one update rule [40]

(57) Since

for any

Therefore

(58) (52) By direct differentiation of the final expression in (58) and root selection, we obtain in (21).

402

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 2, FEBRUARY 2007

Proof of Proposition 4: For the signal model in (16), the channel capacity is (59), shown at the bottom of the page. Applying a rank-one update [40] to the determinant in the numerator, we obtain

(60) The result follows from the proof of Proposition 3. Proof of Lemma 1: The sum capacity of the channel in associated with the mes(28) provided that all signatures are distinct eigenvectors of sage bits is given by

with respect to

and obtain (63)

is the Kuhn–Tucker coefficient [31] chosen where . Substitution of the optimal amplisuch that tude allocation of (63) in (61) gives the maximum attainable sum capacity value where , and are as in (43). Proof of Proposition 7: The output SINR of the maximum SINR filter for the signal model in (46) is

(61) Then, (64) Let be the matrix with columns the eigenthe diagonal matrix vectors of , and the number of available eigenwith the eigenvalues of vectors of that do not correspond to any . , are eigenvectors of we can write Since but

and (65) for

and

Proof of Lemma 2: To identify the set of amplitudes (or equivalently distortions) that maximizes the concave sum ca, pacity function in (61) subject to the distortion constraint we differentiate the Lagrange functional

(62)

where and are diagonal matrices of dimension . which partitions into Consider the permutation matrix that contains the available eigenvectors of with the corresponding eigenvalues in descending order that contains all used eigenvectors: and . Let and be the diagonal matrices that contain the eigenvalues of the eigenvectors in and , respectively. Then

and

(66)

(59)

GKIZELI et al.: OPTIMAL SIGNATURE DESIGN

where

and . Since

403

are diagonal matrices of dimension is an orthonormal basis of , we can write

subject to

as

. We will show that both terms in (73), and , are maximized by the same . For the first term we

have (67) minimum eigenvector of where

and is equivalent to

. The constraint . Hence

(74)

Consider now the second term. By the matrix inversion lemma (68)

The unit norm requirement,

, implies that or

.

subject to We want to find and . Substituting from (68) to the first term in (64), we obtain

(75) Using the matrix inversion lemma again

(69) Using (65) and (68), the second term in (64) becomes

(70) which, using (66), reduces to

(76) Combining (75) and (76), we have

(77) (71) We know that if a matrix

is invertible and

exist, then see the equation shown at the bottom of the page [27]. Hence, (71) reduces further to

Differentiation of the righthandside of (77) with respect to gives

(72) Using (69) and (72), the initial optimization problem can be written equivalently as Hence

(73)

404

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 2, FEBRUARY 2007

is a decreasing function of ever, where and and maximum eigenvalues in . Therefore

minimum eigenvector of

. Howare the minimum

(78)

We conclude [cf. (73), (74), (78), and (68)] that the SINR exis the minimum availpression in (64) is maximized when for any . Hence able eigenvector of

(79) where is the minimum available eigenvalue of (equivalently is the bottom element of ). The optimum value can be computed by setting the derivative of the last expression in (79) equal to zero. The latter gives two candidate . We select the value that maximizes (79), which values for is the one in (48). REFERENCES [1] F. Hartung and M. Kutter, “Multimedia watermarking techniques,” Proc. IEEE, vol. 87, no. 7, pp. 1079–1107, Jul. 1999. [2] G. C. Langelaar, I. Setyawan, and R. L. Lagendijk, “Watermarking digital image and video data: A state-of-the-art overview,” IEEE Signal Process. Mag., vol. 17, no. 9, pp. 20–46, Sep. 2000. [3] L. Marvel and C. G. Boncelet, “Spread spectrum image steganography,” IEEE Trans. Image Process., vol. 8, no. 8, pp. 1075–1083, Aug. 1999. [4] M. Kutter and S. Winkler, “A vision-based masking model for spreadspectrum image watermarking,” IEEE Trans. Image Process., vol. 11, no. 1, pp. 16–25, Jan. 2002. [5] J. R. Smith and B. O. Comiskey, “Modulation and information hiding in images,” Lecture Notes Comput. Sci., vol. 1174, pp. 207–226, 1996. [6] M. Wu and B. Liu, “Data hiding in binary image for authentication and annotation,” IEEE Trans. Multimedia, vol. 6, no. 4, pp. 528–538, Aug. 2004. [7] N. Nikolaidis and I. Pitas, “Robust image watermarking in the spatial domain,” Signal Process., vol. 66, pp. 385–403, May 1998. [8] I. J. Cox, J. Kilian, F. T. Leighton, and T. Shannon, “Secure spread spectrum watermarking for multimedia,” IEEE Trans. Image Process., vol. 6, no. 12, pp. 1673–1687, Dec. 1997. [9] M. Barni, F. Bartolini, A. De Rosa, and A. Piva, “Optimum decoding and detection of multiplicative watermarks,” IEEE Trans. Signal Process., vol. 51, no. 4, pp. 1118–1123, Apr. 2003. [10] ——, “A new decoder for the optimum recovery of nonadditive watermarks,” IEEE Trans. Image Process., vol. 10, no. 5, pp. 755–766, May 2001. [11] C. Qiang and T. S. Huang, “Robust optimum detection of transform domain multiplicative watermarks,” IEEE Trans. Signal Process., vol. 51, no. 4, pp. 906–924, Apr. 2003. [12] G. Csurka, F. Deguillaume, J. J. K. O’Ruanaidh, and T. Pun, “A Bayesian approach to affine transformation resistant image and video watermarking,” Lecture Notes Comput. Sci., vol. 1768, pp. 270–285, 2000. [13] M. Barni, F. Bartolini, A. De Rosa, and A. Piva, “Capacity of full frame DCT image watermarks,” IEEE Trans. Image Process., vol. 9, no. 8, pp. 1450–1455, Aug. 2000.

[14] J. Hernandez, M. Amado, and F. Perez-Gonzalez, “DCT-domain watermarking techniques for still images: Detector performance analysis and a new structure,” IEEE Trans. Image Process., vol. 9, no. 1, pp. 55–68, Jan. 2000. [15] C. Qiang and T. S. Huang, “An additive approach to transform-domain information hiding and optimum detection structure,” IEEE Trans. Multimedia, vol. 3, no. 3, pp. 273–284, Sep. 2001. [16] C. B. Adsumilli, M. C. Q. Farias, S. K. Mitra, and M. Carli, “A robust error concealment technique using data hiding for image and video transmission over lossy channels,” IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 11, pp. 1394–1406, Nov. 2005. [17] J. J. Eggers and B. Girod, “Quantization effects on digital watermarks,” in Signal Process., Feb. 2001, vol. 81, pp. 239–263. [18] P. Moulin and M. K. Mihçak, “A framework for evaluating the datahiding capacity of image sources,” IEEE Trans. Image Process., vol. 11, no. 9, pp. 1029–1042, Sep. 2002. [19] S. Pereira, S. Voloshynovskiy, and T. Pun, “Optimized wavelet domain watermark embedding strategy using linear programming,” in Proc. SPIE Wavelet Applications Conf., Orlando, FL, Apr. 2000, vol. 4056, pp. 490–498. [20] P. Moulin and A. Ivanovic´ , “The zero-rate spread-spectrum watermarking game,” IEEE Trans. Signal Process., vol. 51, no. 4, pp. 1098–1117, Apr. 2003. [21] X. G. Xia, C. G. Boncelet, and G. R. Arce, “A multiresolution watermark for digital images,” in Proc. IEEE Int. Conf. Image Processing, Nov. 1998, vol. 1, pp. 548–551. [22] H. S. Malvar and D. A. Florêncio, “Improved spread spectrum: A new modulation technique for robust watermarking,” IEEE Trans. Signal Process., vol. 51, no. 4, pp. 898–905, Apr. 2003. [23] C. Fei, D. Kundur, and R. H. Kwong, “Analysis and design of watermarking algorithms for improved resistance to compression,” IEEE Trans. Image Process., vol. 13, no. 2, pp. 126–144, Feb. 2004. [24] M. H. M. Costa, “Writing on dirty paper,” IEEE Trans. Inf. Theory, vol. IT-29, no. 3, pp. 439–441, May 1983. [25] B. Chen and G. Wornell, “Quantization index modulation: A class of provably good methods for digital watermarking and information embedding,” IEEE Trans. Inf. Theory, vol. 47, no. 5, pp. 1423–1443, May 2001. [26] P. Moulin and J. A. O’Sullivan, “Information-theoretic analysis of information hiding,” IEEE Trans. Inf. Theory, vol. 49, no. 3, pp. 563–593, Mar. 2003. [27] D. G. Manolakis, V. K. Ingle, and S. M. Kogon, Statistical and Adaptive Signal Processing. New York: McGraw-Hill, 2000. [28] C. Rose, S. Ulukus, and R. D. Yates, “Wireless systems and interference avoidance,” IEEE Trans. Wireless Commun., vol. 1, no. 7, pp. 415–428, Jul. 2002. [29] C. Rose, “CDMA codeword optimization: Interference avoidance and convergence via class warfare,” IEEE Trans. Inf. Theory, vol. 47, no. 9, pp. 2368–2382, Sep. 2001. [30] H. L. Van Trees, Detection Estimation and Modulation Theory, Part I. New York: Wiley, 2001. [31] T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991. [32] P. Anigstein and V. Anantharam, “Ensuring convergence of the MMSE iteration for interference avoidance to the global optimum,” IEEE Trans. Inf. Theory, vol. 49, no. 4, pp. 873–885, Apr. 2003. [33] M. Rupf and J. L. Massey, “Optimum sequence multisets for syncronous code-division-multiple-access channels,” IEEE Trans. Inf. Theory, vol. 40, no. 4, pp. 1261–1266, Jul. 1994. [34] P. Viswanath and V. Anantharam, “Optimal sequences for CDMA under colored noise: A Schur-Saddle function property,” IEEE Trans. Inf. Theory, vol. 48, no. 6, pp. 1295–1318, Jun. 2002. [35] G. N. Karystinos and D. A. Pados, “New bounds on the total squared correlation and optimum design of DS-CDMA binary signature sets,” IEEE Trans. Commun., vol. 51, no. 1, pp. 48–51, Jan. 2003. [36] C. Ding, M. Golin, and T. Kløve, “Meeting the Welch and KarystinosPados bounds on DS-CDMA binary signature sets,” Des., Codes Cryptogr., vol. 30, pp. 73–84, Aug. 2003. [37] P. Ipatov, “On the Karystinos-Pados bounds and optimal binary DS-CDMA signature ensembles,” IEEE Commun. Lett., vol. 8, no. 2, pp. 81–83, Feb. 2004. [38] G. N. Karystinos and D. A. Pados, “The maximum squared correlation, sum capacity, and total asymptotic efficiency of minimum totalsquared-correlation binary signature sets,” IEEE Trans. Inf. Theory, vol. 51, no. 1, pp. 348–355, Jan. 2005.

GKIZELI et al.: OPTIMAL SIGNATURE DESIGN

[39] P. Viswanath, D. N. C. Tse, and V. Anantharam, “Asymptotically optimal waterfilling in multiple antenna multiple access channels,” IEEE Trans. Inf. Theory, vol. 47, no. 1, pp. 241–267, Jan. 2001. [40] C. D. Meyer, Matrix Analysis and Applied Linear Algebra.. Philadelphia, PA: SIAM, 2000. [41] USC-SIPI Image Database, [Online]. Available: http://sipi.usc.edu/ database/database.cgi?volume=misc

Maria Gkizeli (S’00–M’03) received the Diploma degree in electrical engineering from the Democritus University of Thrace, Xanthi, Greece, in 1997, and the M.Sc. and Ph.D. degrees in mobile and satellite communications from the University of Surrey, Guildford, U.K., in 1998 and 2002, respectively. From September 2002 to September 2004, she was a Postdoctoral Fellow with the Communications and Signals Laboratory, Department of Electrical Engineering, State University of New York at Buffalo, where she also held an appointment as an Adjunct Instructor from August 2003 to May 2004. Since September 2004, she has been a Visiting Lecturer with the Department of Electronics, Technological Education Institute of Crete, Chania, Greece. Her research interests lie in the areas of communication theory and systems, covert communications, wireless networks, satellite communications, and image processing. Dr. Gkizeli is a member of the IEEE Signal Processing Society and the Technical Chamber of Greece.

405

Dimitris A. Pados (M’95) was born in Athens, Greece, on October 22, 1966. He received the Diploma degree in computer science and engineering (five-year program) from the University of Patras, Patras, Greece, in 1989, and the Ph.D. degree in electrical engineering from the University of Virginia, Charlottesville, VA, in 1994. From 1994 to 1997, he held an Assistant Professor position in the Department of Electrical and Computer Engineering and the Center for Telecommunications Studies, University of Louisiana, Lafayette. Since August 1997, he has been with the Department of Electrical Engineering, State University of New York at Buffalo, where he is presently an Associate Professor. His research interests are in the general areas of communication theory and adaptive signal processing with an emphasis on wireless multiple access communications, spread-spectrum theory and applications, coding, and sequences. Dr. Pados is a member of the IEEE Communications, Information Theory, Signal Processing, and Computational Intelligence Societies. He served as an Associate Editor for the IEEE SIGNAL PROCESSING LETTERS from 2001 to 2004 and the IEEE TRANSACTIONS ON NEURAL NETWORKS from 2001 to 2005. He received a 2001 IEEE International Conference on Telecommunications best paper award and the 2003 IEEE Transactions on Neural Networks Outstanding Paper Award for articles that he coauthored with his students.

Michael J. Medley (S’91–M’95–SM’02) received the B.S., M.S., and Ph.D. degrees in electrical engineering from Rensselaer Polytechnic Institute, Troy, NY, in 1990, 1991, and 1995, respectively. Since 1991, he has been a Research Engineer for the United States Air Force at the Air Force Research Laboratory, Rome, NY, where he has been involved in adaptive interference suppression and spread-spectrum waveform design. In 2002, he joined the State University of New York Institute of Technology, Utica, as an Assistant Professor. His research interests include transform domain signal processing, adaptive filtering, steganography/steganalysis, and spread-spectrum communications.

391

Optimal Signature Design for Spread-Spectrum Steganography Maria Gkizeli, Member, IEEE, Dimitris A. Pados, Member, IEEE, and Michael J. Medley, Senior Member, IEEE

Abstract—For any given host image or group of host images and any (block) transform domain of interest, we find the signature vector that when used for spread-spectrum (SS) message embedding maximizes the signal-to-interference-plus-noise ratio (SINR) at the output of the corresponding maximum-SINR linear filter. We establish that, under a (colored) Gaussian assumption on the transform domain host data, the same derived signature minimizes host distortion for any target message recovery error rate and maximizes the Shannon capacity of the covert steganographic link. Then, we derive jointly optimal signature and linear processor designs for SS embedding in linearly modified transform domain host data and demonstrate orders of magnitude improvement over current SS steganographic practices. Optimized multisignature/multimessage embedding in the same host data is studied as well. Index Terms—Authentication, covert communications, data hiding, distortion, linear filters, signal-to-interference-plus-noise ratio (SINR), spread spectrum, steganography, watermarking.

I. INTRODUCTION TEGANOGRAPHY is the process of embedding a “secret” digital signal (hidden message) in another digital signal (image or audio) called “cover” or “host.” Unlike general digital watermarking applications, steganography attempts to establish covert communication between trusting parties and imposes the requirement of concealing the existence of the embedded message. As in any watermarking operation, the first step in the design of a steganographic system is to determine the embedding process. This is a crucial task since host-carrier properties, message detector design and performance depend directly on the way the message is inserted in the host data. While each watermarking application has its own individual requirements

S

Manuscript received December 21, 2004; revised July 2, 2006. This work was supported by the U.S. Air Force Research Laboratory under Agreements F30602-03-2-0023 and FA8750-04-1-0091. This paper was presented in part at the IEEE International Conference on Image Processing (ICIP), Singapore, October 2004. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Benoit Macq. M. Gkizeli was with the Department of Electrical Engineering, State University of New York at Buffalo, Buffalo, NY 14260 USA. She is now with the Department of Electronics, Technological Education Institute of Crete, Chania, 73133 Greece (e-mail: [email protected]). D. A. Pados is with the Department of Electrical Engineering, State University of New York at Buffalo, Buffalo, NY 14260 USA (e-mail: [email protected] eng.buffalo.edu). M. J. Medley is with the Department of Electrical Engineering, State University of New York Institute of Technology, Utica, NY 13504 USA (e-mail: [email protected]). Digital Object Identifier 10.1109/TIP.2006.888345

[1], [2], the broad common objective of most steganographic applications is a satisfactory tradeoff between hidden message resistance to noise/disturbance, information delivery rate, and host distortion for concealment purposes. Message embedding can be performed either directly in the time (audio) or spatial (image) domain [3]–[7] or in a transform domain (for example, for images we may consider full-frame discrete Fourier transform (DFT) [8]–[12], full-frame discrete cosine transform (DCT) [13], block DFT or DCT [14]–[18], or wavelet transforms [19]–[21]). Direct embedding in the original host signal domain may be desirable for system complexity purposes, while embedding in a transform domain may take advantage of the particular transform domain properties [23]. In this paper, we focus our attention on transform domain spread-spectrum (SS) embedding methods for image steganography. In a broad sense, any steganographic system for which the secret signal is spread over a wide range of host image frequencies can be referred to as an SS embedding system. Once the transform embedding domain is selected, the hidden message can be applied to the host data through an additive [8], [12], [14]–[17], [22] or multiplicative [9]–[11] rule. In the literature, additive SS embedding methods either directly apply (add) the message/watermark to several host coefficients [17] or in direct analogy to SS digital communications systems use an amplitude modulated signature to deposit one information symbol across a group of host data coefficients [8], [12], [14], [16] or a linearly transformed version of the host data coefficients [22]. SS embedding algorithms for blind image steganography (that is, hidden message recovery without knowledge of the original image) have been based on the understanding that the host signal acts as a source of interference to the secret message of interest. Yet, it should also be understood that this interference is known to the message embedder. Such knowledge can be exploited appropriately to facilitate the task of the blind receiver at the other end and minimize the recovery error rate for a given host distortion level, minimize host distortion for a given target recovery error rate, maximize the Shannon capacity of the covert steganographic channel, etc. Indeed, if we were to place the steganography application in an information theoretic context, it could be viewed as a communications-with-side-information problem [24]–[26]. Optimized embedding methods can facilitate host interference suppression at the receiver side when knowledge of the host signal is adequately exploited in system design. In this paper, for any given image (or set of images), (block) transform domain, and host bins, we derive the additive embedding signature that maximizes the output signal-to-interference-plus-noise ratio (SINR) of the linear maximum-SINR re-

1057-7149/$25.00 © 2007 IEEE

392

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 2, FEBRUARY 2007

ceiver filter.1 We show that under a (colored) Gaussian assumption on the host bins, this same signature minimizes the receiver bit-error rate (BER) at any mean-square (MS) image distortion level, minimizes—conversely—the MS image distortion at any target BER, and maximizes the Shannon capacity of the covert link. We then generalize our findings and present a novel joint signature and linear host data projection optimization scheme. In this present work, we consider only scalar parametrized host data projection as in [22]. Finally, we extend signature-only as well as joint signature and host-projection optimization to multiuser (multiple-signature) embedding. Our emphasis is directed primarily toward low complexity, sequential, conditional multiuser optimization. The rest of the paper is organized as follows. Section II presents the core signature and embedding optimization results. These results are generalized to multiple signature embedding in Section III. Section IV is devoted to experimental studies and comparisons. A few concluding remarks are drawn in Section V. II. SIGNATURE OPTIMIZATION FOR SS EMBEDDING In this section, we develop an optimized SS steganographic system. To draw a parallelism with conventional SS communications systems, in SS watermark (message) embedding the watermark can be regarded as the SS signal of interest transmitted through a noisy “channel” (the host). The disturbance to the SS signal of interest is the host data themselves plus potential external noise due to physical transmission of the watermarked data and/or processing/attacking. The purpose of the watermark detector is to withstand the influence of the total end-to-end disturbance and recover the original hidden message.

H2f

g

Fig. 1. (a) Baboon image example 0; 1; . . . ; 255 data autocorrelation matrix (8 8 DCT, 63-bin host).

2

. (b) Host

A. Signal Model and Notation Consider a host image that is to be wais the image alphabet and termarked where is the image size in pixels. Fig. 1(a) shows a grayscale baboon image example in . is partitioned into Without loss of generality, the image local blocks of size pixels. Each block is to carry one hidden information bit , respectively. Embedding is performed in a real 2-D transform domain . After transform calculation and conventional zig-zag scanning vectorization, . From the we obtain transform domain vectors we choose a fixed subset coefficients (bins) to form the final host of (for example, it is common vectors , from and appropriate to exclude the dc coefficient, the host ). The autocorrelation matrix of the host data is an important statistical quantity for our developments and is defined as follows: 1The problem—and solution—parallels the eigen-signature design for code-division multiple-access (CDMA) wireless communications [28]. In SS steganography, however, the disturbance (transform-domain host) is readily available at the embedder for manipulation and statistical exploitation and in contrast to the CDMA problem may be plausibly characterized as Gaussian.

(1) denotes statistical expectation (here, with respect where to over the given image ) and is the transpose operator. It is easy to verify that in general , where is the size-L identity matrix; that is, is not constant-value diagonal or “white” in field language. For example, 8 8 DCT with 63-bin host data formation (exclude only the dc coefficient) for the baboon image in Fig. 1(a) gives the host autocorrelation in Fig. 1(b). matrix B. Signature Optimization Consider direct additive SS embedding of the form (2) where

is the message bit embedded in the host is the bit amplitude, , is the (normalized) embedding signature to be designed, and represents potential external white Gaussian noise2 of variance 2Additive white Gaussian noise is frequently viewed as a suitable model for quantization errors, channel transmission disturbances, and/or image processing attacks.

GKIZELI et al.: OPTIMAL SIGNATURE DESIGN

393

. The mean-squared distortion of the original image due to the embedded data only is

is the optimum (minimum probability of error) bit detector [30] with probability of error

(3) and total disturbance in (2), With signal of interest the linear filter that operates on and offers maximum SINR at its output [27] is

(4)

(10) . We see that is a where . If we now view monotonically decreasing function of as a function of the embedding signature , then from Proposition 1 and (10), when the probability of error of the optimum detector is minimized to

The exact maximum output SINR value attained is

(11) and optimum detection reduces to (12) (5)

as a function of the embedding We propose to view , and identify the signature that maxisignature mizes the SINR at the output of the maximum SINR filter. Our findings are presented in the form of a proposition below that parallels the developments for code-division-multiple-access (CDMA) codeword optimization in [28] and [29]. The proof is straightforward and omitted. Proposition 1: Consider additive SS embedding according to be eigenvectors of in (1) with cor(2). Let . For any hidden responding eigenvalues message-induced distortion level , a signature that maximizes the output SINR of the maximum SINR filter is (6) When

, the output SINR is maximized to

(7) and maximum SINR data filtering simplifies to (8)

Conversely, for any preset probability of error level minimizes the host distortion due to the hidden message to (13) We conclude that under a Gaussian transform-domain host data assumption, the “minimum” eigenvector of the host data autocorrelation matrix, when used as the embedding signature, allows message recovery with the minimum possible and it does so by trivial signature BER (eigenvector) matched filter detection. Conversely, the mesis minimized for any given sage-induced image distortion target BER. If necessary, further BER improvements below for any fixed distortion can be attained via error correcting coding of the information bits at the expense of reduced information bit payload. The maximum possible host payload in bits that still allows—theoretically for asymptotically large number of image blocks —message where recovery with arbitrarily small probability of error is is the Shannon capacity of the covert link identifies in bits per embedding attempt. We recall that the information conveyed about the input variable by the received vector and denotes the input variable probability distribution function. For Gaussian host data and average image distortion constraint , we can calculate [31] (14)

In summary, Proposition 1 says that the “minimum” eigenvector of the host data autocorrelation matrix, when used as the embedding signature, sends the output SINR to its max. At the same time, maximum imum possible value SINR filtering becomes plain signature (eigenvector) matched filtering. If, in addition, we are allowed to assume that is Gaussian, , then (9)

where is the determinant operator. We can show that the signature choice is also optimal in terms of capacity, of the covert link, and, therei.e., maximizes the capacity for the host vectors fore, the maximum allowable payload . The result is presented in the form of Proposition 2 below whose proof is given in the Appendix. Proposition 2: Consider additive SS embedding by (2) with . Let be eigenvectors of with . For any corresponding eigenvalues

394

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 2, FEBRUARY 2007

hidden message-induced distortion level , the signature maximizes the covert channel capacity to

bits per symbol embedding

(signature , parameter ) that maximizes the output SINR of the maximum SINR filter is

(20)

(15) and

C. Signature Optimization for Linearly Transformed Host Data

(21)

In an effort to further reduce host distortion for a given target probability of error during message recovery, we modify the previous developments and attempt to steer the host data vectors operator of away from the embedding signature using an [22] where both the parameter and the form the signature , are to be designed.3 In parallel to (2), the composite signal now becomes

When

and

, the output SINR is maximized to (22)

and maximum SINR data filtering simplifies to

(16) and the mean-squared distortion due to the embedding operation only is (17) We observe that, in contrast to (3), the distortion level is controlled not only by but by and , as well. With signal of and total disturbance in (16), the interest linear filter that operates on and offers maximum SINR at its output is

(18) The exact maximum output SINR value attained is

(23) The target distortion is achieved when the bit amplitude is set . at Proposition 3 shows that the optimum signature assignand maximum ment is still the “minimum” eigenvector of SINR filtering still reduces to plain signature (eigenvector) matched filtering. The optimum selection of depends on the , the noise variance , and minimum eigenvalue of the target distortion level . The optimum pair allows the output SINR to attain its maximum possible value . From (21), it is interesting and to note that as the allowed distortion the embedding scheme converges to the conventional scheme , the linear operator of Section II-B. As converges to the orthogonal projector and the allowable distortion is budgeted more toward host-data modification and less toward the embedding bit amplitude. , then If we assume that is Gaussian, is the optimum bit detector with probability of error

(19) In the following, we look at as a function of both the , and embedding signature and the parameter identify the signature and parameter values that jointly maximize the SINR at the output of the maximum SINR filter. Our findings are presented in the form of a proposition below. The proof is given in the Appendix. Proposition 3: Consider additive SS embedding in linearly be eigenvectransformed host data by (16). Let with corresponding eigenvalues tors of . For any hidden message-induced distortion level , a pair 3If

c

= 0, we revert to the developments of Section II-B. If c = 1; I

(24) As in the plain additive SS embedding scenario, if is Gaussian then the probability of error is a monotonically decreasing func. The pair which maximizes tion of the output SINR of the maximum-SINR filter is also minimizing the probability of error of the optimum detector to

(25)

0

becomes the projector orthogonal to s. In this work, as in [22], we confine css ourselves within this class of scalar parametrized linear operators merely for mathematical convenience and tractability.

and optimum detection reduces to

.

GKIZELI et al.: OPTIMAL SIGNATURE DESIGN

395

We can show that for Gaussian transform-domain host data , the covert channel capacity is given by

denotes the “exclude- ” data autocorrelation matrix, where that is the autocorrelation matrix of the disturbance to message defined as

(26) assignment of ProposiThen, we can prove that the tion 3 is also optimal in terms of capacity. This result is summarized in the following proposition whose proof is given in the Appendix. Proposition 4: Consider additive SS embedding by (16) with . Let be eigenvectors of with corresponding eigenvalues . For any hidden message-induced distortion level , the pair of (20) and (21) maximizes the covert channel capacity to

(32) The exact maximum output SINR value attained is (33)

(27) bits per symbol embedding. In the following section, we consider the problem of data embedding in one host with multiple signatures.

III. MULTISIGNATURE EMBEDDING We may generalize the signal model in (2) to cover multisignature/multimessage embedding of the form

As in Section II for single-message embedding, we propose as a function of the embedding signature to view , and identify the signature vector that maximizes the SINR value. Our findings are presented in the form of Proposition 5 below and parallel the developments of Proposition 1 for the single-message case. Proposition 5: Consider additive SS embedding by (28). be eigenvectors of in (32) with corLet . For any responding eigenvalues , a signature that maximessage-induced distortion level mizes the output SINR of the maximum SINR filter is (34)

(28) When coming potentially from distinct where bits messages, are embedded simultaneously in with correand embedding signatures sponding amplitudes . Thus, the contribution of each individual embedded message bit to the composite watermarked signal is and the mean-squared distortion to the original host data due to the embedded message alone is

, the output SINR is maximized to (35)

and total disturbance With signal of interest in (28), the linear filter that operates on and offers maximum SINR at its output is

and maximum SINR data filtering simplifies to . In summary, Proposition 5 says that the “minimum” eigenvector of the disturbance autocorrelation matrix when used as the embedding signature allows the output SINR to attain its . At the same time, maximum possible value maximum SINR filtering becomes plain signature (eigenvector) matched filtering. In the context of multisignature optimization, for fixed-bit and arbitrary signature initializaamplitude values , consider repeated tion . In such an applications of Proposition 5 for eigen-update signature cycle each signature is replaced by the minimum-eigenvalue eigenvector of the disturbance autocorrelation matrix as seen by the message corresponding to that signature. Once all signatures are updated, a second update cycle may begin. The whole procedure may continue for a predetermined number of cycles or until convergence

(31)

(36)

(29) Under a statistical independence assumption across message bits, the mean-squared distortion of the original image due to the total multimessage insertion is (30)

396

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 2, FEBRUARY 2007

It can be proven that convergence of (36) is guaranteed and, as shown in a CDMA literature context [28], at each cycle the genof the signature set eralized total-squared correlation

and maximum SINR data filtering simplifies to

where , is nonincreasing.4 If we consider channel coding the message bits before embedding and assume that the host in (28) is Gaussian, the steganographic system determined by the signature matrix is a special case of the -user Gaussian multiple access channel with average input distortion constraints [31], [33]. For such a channel, (defined as the maximum sum of mesthe sum capacity sage coding rates at which messages can be recovered reliably [33], [34]) is a reasonable criterion of quality for the signature set and equals

If, in addition to the assumptions of Proposition 6, we are al, then the optimum detector lowed to assume that for the bit of interest is with probability of error . As a simple illustration of the use of Proposition 6 for conditionally optimal multisignature design, suppose that we want to embed in the message bits host data vector with fixed corresponding amplitudes (mean) and squared distortions to be chosen. By Proposition 1 of Secsignatures equal to the bottom eigenvector of tion II, we set . Given and under the constraint that we search for an orthogonal to , by Proposition 6, we design which is the next available eigenvector of from the bottom. and under the constraint that we Given search for an orthogonal to both and , by Proposition . We continue this way until the final 6, we assign . Once again, a welcome side efassignment fect of this conditionally SINR optimal signature design procedure is that the maximum SINR receiver for each message bit , simplifies to a matched filter and requires no knowledge of other system parameters. For (fixed) unequal embedding amplitude values , the exact order by which the eigenvectors of are drawn to become signatures is important if we consider the sum capacity of the steganographic system. We can show that a necessary condition for a maximum sum capacity solution under the constraint of eigenvector assignment is that the ordering of the bit amplitudes be inversely proportional to the ordering of the eigenvalues of the corresponding signature eigenvectors. This statement is given below in the form of a lemma. The proof can be found in the Appendix. Lemma 1: Consider additive SS embedding by (28) with and let be eigenvectors of with cor. Without loss of responding eigenvalues generality assume that . Then

(37) Minimization of the metric translates to maximization of [34]. However, decrease in at each cycle of the algorithm in (36) does not necessarily imply increase of as seen for instance in [35]–[38] via binary signature examples. Apart from global optimality limitations, the multicycle multisignature optimization procedure in (36) requires recalculation of the disturbance autocorrelation matrix and eigen decomposition at each step of each cycle. A simple low-cost alternative to (36) could be a conditionally optimal single-cycle design method based on the following proposition. Proposition 6: Consider additive SS embedding according to (38) be eigenvectors of in (1) with correLet sponding eigenvalues . If the signatures are eigenvectors of , then for , a signaany message-induced distortion level that maximizes the output SINR of the maximum ture SINR filter for the bit of interest subject to the constraint is

(41)

(39) where is the minimum-eigenvalue eigenvector of available. , the output SINR is (conditionally) maxiWhen mized to (40) 4Yet, there is no guarantee that TSC (S) will converge to its minimum possible value (global minimum) [29], [32].

If we generalize our approach and view the individual amplitudes/distortions as design parameters themselves, then we can search for the optimal amplitude assignment that maximizes sum capacity subject to a total allowable distortion constraint . We derive the optimal amplitude values in the lemma below. The proof is given in the Appendix. Lemma 2: Consider additive SS embedding by (28). Let be eigenvectors of with corresponding . If the signatures aseigenvalues are distinct eigenvectors of sociated with the message bits then the sum

GKIZELI et al.: OPTIMAL SIGNATURE DESIGN

397

capacity is maximized subject to an expected total distortion if constraint (42) where and is the Kuhn–Tucker coefficient [31] chosen such that the distortion constraint is met. To find the necessary parameter value in (42), we suggest in to arrange the participating eigenvalues . Then ascending order

be eigenvectors of in (1) with correLet . Assume that sponding eigenvalues , are all eigenvectors of and is the index of the minimum-eigenvalue eigenvector of available. For any , an given message-induced distortion level pair that maximizes the output SINR of the maximum SINR is filter subject to the constraint

(43) where the cutoff index is the greatest integer in and . The optimal message amplitude/distortion allocation solution of Lemma 2 can be viewed as a power waterfilling procedure [31] in the eigen domain of the host. Finally, as the last technical development in this paper, we examine the possibility of carrying out multisignature embedding in linearly transformed host data. We assume that the host data vector is linearly transformed by an operator of the form where and , are the parameters and signatures to be designed. The final composite signal is

(47) and

(48) and When tionally) maximized to

, the output SINR is (condi-

(49) (44) and the mean-squared distortion induced by each individual message , is

(45) With signal of interest trix of the disturbance is

and maximum SINR data filtering simplifies to . If in addition to the assumptions of Proposition 7, we are al, then the optimum detector lowed to assume that for the bit of interest is with probability of error (50)

, the autocorrelation ma-

. Unfortunately, in contrast to (38) for multisignature embedding in nontransremains a function of (as well as ). In formed data, this context, unconditionally optimal multisignature multicycle optimization along the lines of (36) is practically an unrealistic objective. Instead, we suggest to design sequentially the amplitudes , parameters , and signatures of the embedded such that the output SINR messages is conditionally maximized given all past fixed embeddings (single-cycle optimization). Our developments are presented in the form of Proposition 7 below whose proof is given in the Appendix. Proposition 7: Consider additive SS embedding according to

(46)

As an illustrative example of the use of Proposition 7, suppose that we would like to embed in the host data vector message bits with individual corresponding . We first design mean-squared host distortion the system parameters , and for message bit in the absence of any other message in the host image. According to , the optimal parameter selection is Proposition 7

and by (45) . Next, we proceed with the and optimize and subject to the second message bit and the constraint . Since desired distortion level is already an eigenvector of , Proposition 7 offers the equation shown at the bottom of the next page, and by (45) . We continue calculating signatures , parameters , and amplitudes as above, always subject to the desired distortion and orthogonality between the signature to , be designed and all other prior signatures. Provided that

398

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 2, FEBRUARY 2007

the final set of designed parameters is shown in the equation at . the bottom of the page, and Optimal distortion allocation for sum capacity maximization in multimessage embedding in linearly transformed host data subject to the constraint that all messages are assigned distinct , and subject to a total distoreigenvectors of tion constraint is a joint optimization problem with respect to and . We suggest an iterative solution approach based on Proposition 7 above and Lemma 2 presented earlier in this section. We initially fix the distortions induced by each message and find the optimum parameters according to Proposition 7. Then, we perform optimum amplitude allocation according to Lemma 2. Based on this allocation, we reevaluate all by Proposition 7. We continue until convergence is observed. In the following, we present extensive experimental results that we obtained from the implementation of the developed steganographic algorithms.

IV. EXPERIMENTAL STUDIES To carry out an experimental study of the developments presented in the previous sections, we consider first as a host example the familiar grayscale 512 512 “Boat” image in Fig. 2(a) that has been widely used in the pertinent literature. We perform 8 8 block DCT single-signature embedding over all bins except the dc coefficient. Hence, our signature and we embed bits. For length is the sake of generality, we also incorporate white Gaussian dB. Fig. 3 shows the recovery BER noise of variance under signature matched filter detection as a function of the distortion created by the embedded message over the 0- to 20-dB range for four different embedding schemes: a) SS embedding with an arbitrary signature, b) SS embedding with an arbitrary signature and optimized selection of the host data transformation parameter as in [22] (known as “improved spread-spectrum” or ISS), c) SS embedding with an optimal signature according to Proposition 1, and d) SS embedding with a jointly optimal signature and host data transformaaccording to Proposition 3. The tion parameter demonstrated BER improvement of our joint signature and parameter optimization procedure, in particular, measures in orders of magnitude. Fig. 2(b) shows the Boat image after 20 -embedding of the 4 Kbit message and 3-dB dB

2

Fig. 2. (a) Boat image example (512 512 grayscale). (b) Boat image after 20-dB (s ; c ) embedding of 4 Kbits and additive white Gaussian noise of variance 3 dB.

additive white Gaussian noise disturbance in the block DCT domain. In Fig. 4, we repeat the same experiment of Fig. 3 on the 256 256 grayscale Baboon image in Fig. 1(a) (signature , hidden message of bits, and length additive white Gaussian noise disturbance of variance 3 dB). Comparatively speaking (Boat versus Baboon host or Fig. 3 versus Fig. 4 results), message recovery from the Baboon host appears to be a somewhat more difficult problem. Yet, the

GKIZELI et al.: OPTIMAL SIGNATURE DESIGN

399

Fig. 3. Bit-error rate versus host distortion (Boat image, = 3 dB).

Fig. 5. Bit-error rate versus host distortion (average findings over USC-SIPI image database [41], = 3 dB).

Fig. 4. Bit-error rate versus host distortion (Baboon image, = 3 dB).

Fig. 6. Capacity versus distortion (Baboon image, = 3 dB).

proposed joint signature and host transformation parameter opBER at 20-dB timization scheme maintains a better than host distortion and outperforms the proposed signature-only optimization scheme by about eight orders of magnitude. To address the need for experimental verification of highest credibility, we carried out the experiments of Figs. 3 and 4 over the whole USC-SIPI database [41] of 44 miscellaneous images. Fig. 5 shows the average BER versus distortion for the database. The average database findings are quite similar to the individual Baboon (or Boat) results. In Fig. 6, we return and continue the work with the Baboon host and plot the capacity versus distortion performance curves for the four embedders under consideration. We see, for example, that at 20–dB host distortion the jointly optimized embedder offers 2.7 information bits payload per embedded symbol (suggesting implicitly the suitability of a higher than binary message alphabet). The bit payload number goes down to 1.2 for signature-only optimization, 0.6 for ISS embedding [22], and 0.5 for arbitrary signature embedding.

Next, we consider the problem of multisignature embedding. We keep the Baboon image as the host and wish to hide data blocks/messages of length 1024 bits each with each block/message having its own individual embedding signature. Each message is allowed to cause the same expected distor. Therefore, for station to the host tistically independent messages, the total distortion to the host . As before, for the sake of generality, we is add to the host white Gaussian noise of variance 3 dB. We study five different multisignature embedding schemes: a) Embedding with arbitrary signatures, b) ISS embedding [22], c) multicycle eigen-signature design by (36), d) conditional optiassignment, mization by Proposition 6 (sequential ), and e) conditional optimization by Proposition assignment, ). The results 7 (sequential in Fig. 7 reiterate the importance of optimized host-data manipulation in conjunction with signature optimization. In fact, optimization even the least faunder joint sequential outperforms in recovery BER the most vored message

400

Fig. 7. BER as a function of the per-message distortion K = 15 messages of size 1024 bits each, = 3 dB).

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 2, FEBRUARY 2007

D (Baboon image,

Fig. 9. Sum capacity versus distortion (Baboon image, K = 15; = 3 dB).

Fig. 8. Baboon image after multisignature embedding via Proposition 7 (K = 15 messages of size 1 024 bits each, per-message distortion 20 dB, = 3 dB).

favored

message under sequential signature-only optimization or multicycle signature-only design for per-message distortion values above 18 dB.5 Fig. 8 shows the Baboon image after embedding the fifteen messages (15 1024 , optimizabits) via joint sequential tion with 20-dB per-message distortion (31.8-dB total distortion) and 3-dB variance additive white Gaussian noise. Finally, in Fig. 9, we present sum capacity results when the design and sequential two proposed schemes, sequential design, , employ waterfilling power joint allocation (use of Lemma 2 alone or coupled use of Proposition 7 and Lemma 2, correspondingly). We see, for example, that design offers at 32-dB total distortion the waterfilled information bit payload of about 36 bits per 15 symbols emdesign offers only bedded, while the waterfilled about 12 bits per 15 symbols embedded. In Fig. 10, we repeat the experiment of Fig. 9 over the 44 images of the USC-SIPI database [41]. The relative average sum capacity behavior of the embedders remains the same over the database. In absolute

5The performance of the most favored message (i = 1) for both proposed conditional schemes, s ; c and s ; c = 0, remains the same as in the single message case (see Fig. 4) since the (orthogonal) eigenvector signature assignment completely avoids multimessage interference.

Fig. 10. Sum capacity versus distortion (average findings over USC-SIPI image database [41], K = 15; = 3 dB).

numbers, for all embedders the database images have on the average about ten more information bits payload per 15-symbol embedding than the Baboon image at 32-dB total distortion. V. CONCLUSION We considered the problem of hiding digital data in a digital host image via SS embedding in an arbitrary transform domain. We showed that use of the minimum-eigenvalue eigenvector of the transform domain host data autocorrelation matrix as the embedding signature offers the maximum possible SINR under linear filter message recovery and, conveniently, does so under plain signature correlation (signature matched filtering). If we allow ourselves the added assumption of (colored) Gaussian transform-domain host data, then we see that the above described system as a whole becomes minimum probability of error and maximum Shannon capacity optimal, as well. To take these findings one step further, we examined SS embedding in transform-domain host data that are modified by a

GKIZELI et al.: OPTIMAL SIGNATURE DESIGN

401

parametrized projection-like linear operator. We found the joint signature and parameter values under the optimality scenaria mentioned above. Conveniently, the jointly optimal signature is still the minimum-eigenvalue eigenvector and the SINR optimal linear filter at the receiver side is still the signature correlator. Yet, joint signature and parameter optimization was seen to offer dramatic improvements in SINR, probability of error, and capacity. Finally, we extended our effort to cover multisignature/multimessage embedding. First, under signature-only optimization we developed a computationally costly multicycle eigen-signature design scheme based on the disturbance autocorrelation matrices. The alternative suggestion based on the host data autocorrelation matrix alone and sequential (conditional) eigen-signature optimization is practically much more appealing. A waterfilling amplitude assignment algorithm was developed as well to maximize sum capacity under eigen-signature designs. All multisignature findings were generalized to cover parametrized projection-like modification of the host data with, once again, dramatic improvements in probabilty of error or sum capacity. As a brief concluding remark, image-adaptive signature(s) or signature(s)/parameter(s) optimization as described in this work can be carried out over a set of host images (frames) if desired. The only technical difference is the calculation of the host data autocorrelation matrix which now has to extend over the whole host set. As long as the cumulative host autocorrelation matrix is , significant gains are to be not constant-value diagonal collected over standard nonadaptive SS embedding techniques.

Proof of Proposition 3: For a target distortion value , the in (19) equals and is maximized for . We will show that the second term in (19), , is maximized by , as well. By the matrix inversion lemma term

and (53)

(54) Combining (53) and (54), we obtain

(55)

APPENDIX Proof of Proposition 2: We need to find that maximizes in (14) subject to . Since is a strictly monotonic function

The derivative of the righthandside of (55) with respect to gives

(56)

(51)

Hence, a decreasing function of

is . Yet,

. Therefore

Using the rank-one update rule [40]

(57) Since

for any

Therefore

(58) (52) By direct differentiation of the final expression in (58) and root selection, we obtain in (21).

402

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 2, FEBRUARY 2007

Proof of Proposition 4: For the signal model in (16), the channel capacity is (59), shown at the bottom of the page. Applying a rank-one update [40] to the determinant in the numerator, we obtain

(60) The result follows from the proof of Proposition 3. Proof of Lemma 1: The sum capacity of the channel in associated with the mes(28) provided that all signatures are distinct eigenvectors of sage bits is given by

with respect to

and obtain (63)

is the Kuhn–Tucker coefficient [31] chosen where . Substitution of the optimal amplisuch that tude allocation of (63) in (61) gives the maximum attainable sum capacity value where , and are as in (43). Proof of Proposition 7: The output SINR of the maximum SINR filter for the signal model in (46) is

(61) Then, (64) Let be the matrix with columns the eigenthe diagonal matrix vectors of , and the number of available eigenwith the eigenvalues of vectors of that do not correspond to any . , are eigenvectors of we can write Since but

and (65) for

and

Proof of Lemma 2: To identify the set of amplitudes (or equivalently distortions) that maximizes the concave sum ca, pacity function in (61) subject to the distortion constraint we differentiate the Lagrange functional

(62)

where and are diagonal matrices of dimension . which partitions into Consider the permutation matrix that contains the available eigenvectors of with the corresponding eigenvalues in descending order that contains all used eigenvectors: and . Let and be the diagonal matrices that contain the eigenvalues of the eigenvectors in and , respectively. Then

and

(66)

(59)

GKIZELI et al.: OPTIMAL SIGNATURE DESIGN

where

and . Since

403

are diagonal matrices of dimension is an orthonormal basis of , we can write

subject to

as

. We will show that both terms in (73), and , are maximized by the same . For the first term we

have (67) minimum eigenvector of where

and is equivalent to

. The constraint . Hence

(74)

Consider now the second term. By the matrix inversion lemma (68)

The unit norm requirement,

, implies that or

.

subject to We want to find and . Substituting from (68) to the first term in (64), we obtain

(75) Using the matrix inversion lemma again

(69) Using (65) and (68), the second term in (64) becomes

(70) which, using (66), reduces to

(76) Combining (75) and (76), we have

(77) (71) We know that if a matrix

is invertible and

exist, then see the equation shown at the bottom of the page [27]. Hence, (71) reduces further to

Differentiation of the righthandside of (77) with respect to gives

(72) Using (69) and (72), the initial optimization problem can be written equivalently as Hence

(73)

404

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 2, FEBRUARY 2007

is a decreasing function of ever, where and and maximum eigenvalues in . Therefore

minimum eigenvector of

. Howare the minimum

(78)

We conclude [cf. (73), (74), (78), and (68)] that the SINR exis the minimum availpression in (64) is maximized when for any . Hence able eigenvector of

(79) where is the minimum available eigenvalue of (equivalently is the bottom element of ). The optimum value can be computed by setting the derivative of the last expression in (79) equal to zero. The latter gives two candidate . We select the value that maximizes (79), which values for is the one in (48). REFERENCES [1] F. Hartung and M. Kutter, “Multimedia watermarking techniques,” Proc. IEEE, vol. 87, no. 7, pp. 1079–1107, Jul. 1999. [2] G. C. Langelaar, I. Setyawan, and R. L. Lagendijk, “Watermarking digital image and video data: A state-of-the-art overview,” IEEE Signal Process. Mag., vol. 17, no. 9, pp. 20–46, Sep. 2000. [3] L. Marvel and C. G. Boncelet, “Spread spectrum image steganography,” IEEE Trans. Image Process., vol. 8, no. 8, pp. 1075–1083, Aug. 1999. [4] M. Kutter and S. Winkler, “A vision-based masking model for spreadspectrum image watermarking,” IEEE Trans. Image Process., vol. 11, no. 1, pp. 16–25, Jan. 2002. [5] J. R. Smith and B. O. Comiskey, “Modulation and information hiding in images,” Lecture Notes Comput. Sci., vol. 1174, pp. 207–226, 1996. [6] M. Wu and B. Liu, “Data hiding in binary image for authentication and annotation,” IEEE Trans. Multimedia, vol. 6, no. 4, pp. 528–538, Aug. 2004. [7] N. Nikolaidis and I. Pitas, “Robust image watermarking in the spatial domain,” Signal Process., vol. 66, pp. 385–403, May 1998. [8] I. J. Cox, J. Kilian, F. T. Leighton, and T. Shannon, “Secure spread spectrum watermarking for multimedia,” IEEE Trans. Image Process., vol. 6, no. 12, pp. 1673–1687, Dec. 1997. [9] M. Barni, F. Bartolini, A. De Rosa, and A. Piva, “Optimum decoding and detection of multiplicative watermarks,” IEEE Trans. Signal Process., vol. 51, no. 4, pp. 1118–1123, Apr. 2003. [10] ——, “A new decoder for the optimum recovery of nonadditive watermarks,” IEEE Trans. Image Process., vol. 10, no. 5, pp. 755–766, May 2001. [11] C. Qiang and T. S. Huang, “Robust optimum detection of transform domain multiplicative watermarks,” IEEE Trans. Signal Process., vol. 51, no. 4, pp. 906–924, Apr. 2003. [12] G. Csurka, F. Deguillaume, J. J. K. O’Ruanaidh, and T. Pun, “A Bayesian approach to affine transformation resistant image and video watermarking,” Lecture Notes Comput. Sci., vol. 1768, pp. 270–285, 2000. [13] M. Barni, F. Bartolini, A. De Rosa, and A. Piva, “Capacity of full frame DCT image watermarks,” IEEE Trans. Image Process., vol. 9, no. 8, pp. 1450–1455, Aug. 2000.

[14] J. Hernandez, M. Amado, and F. Perez-Gonzalez, “DCT-domain watermarking techniques for still images: Detector performance analysis and a new structure,” IEEE Trans. Image Process., vol. 9, no. 1, pp. 55–68, Jan. 2000. [15] C. Qiang and T. S. Huang, “An additive approach to transform-domain information hiding and optimum detection structure,” IEEE Trans. Multimedia, vol. 3, no. 3, pp. 273–284, Sep. 2001. [16] C. B. Adsumilli, M. C. Q. Farias, S. K. Mitra, and M. Carli, “A robust error concealment technique using data hiding for image and video transmission over lossy channels,” IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 11, pp. 1394–1406, Nov. 2005. [17] J. J. Eggers and B. Girod, “Quantization effects on digital watermarks,” in Signal Process., Feb. 2001, vol. 81, pp. 239–263. [18] P. Moulin and M. K. Mihçak, “A framework for evaluating the datahiding capacity of image sources,” IEEE Trans. Image Process., vol. 11, no. 9, pp. 1029–1042, Sep. 2002. [19] S. Pereira, S. Voloshynovskiy, and T. Pun, “Optimized wavelet domain watermark embedding strategy using linear programming,” in Proc. SPIE Wavelet Applications Conf., Orlando, FL, Apr. 2000, vol. 4056, pp. 490–498. [20] P. Moulin and A. Ivanovic´ , “The zero-rate spread-spectrum watermarking game,” IEEE Trans. Signal Process., vol. 51, no. 4, pp. 1098–1117, Apr. 2003. [21] X. G. Xia, C. G. Boncelet, and G. R. Arce, “A multiresolution watermark for digital images,” in Proc. IEEE Int. Conf. Image Processing, Nov. 1998, vol. 1, pp. 548–551. [22] H. S. Malvar and D. A. Florêncio, “Improved spread spectrum: A new modulation technique for robust watermarking,” IEEE Trans. Signal Process., vol. 51, no. 4, pp. 898–905, Apr. 2003. [23] C. Fei, D. Kundur, and R. H. Kwong, “Analysis and design of watermarking algorithms for improved resistance to compression,” IEEE Trans. Image Process., vol. 13, no. 2, pp. 126–144, Feb. 2004. [24] M. H. M. Costa, “Writing on dirty paper,” IEEE Trans. Inf. Theory, vol. IT-29, no. 3, pp. 439–441, May 1983. [25] B. Chen and G. Wornell, “Quantization index modulation: A class of provably good methods for digital watermarking and information embedding,” IEEE Trans. Inf. Theory, vol. 47, no. 5, pp. 1423–1443, May 2001. [26] P. Moulin and J. A. O’Sullivan, “Information-theoretic analysis of information hiding,” IEEE Trans. Inf. Theory, vol. 49, no. 3, pp. 563–593, Mar. 2003. [27] D. G. Manolakis, V. K. Ingle, and S. M. Kogon, Statistical and Adaptive Signal Processing. New York: McGraw-Hill, 2000. [28] C. Rose, S. Ulukus, and R. D. Yates, “Wireless systems and interference avoidance,” IEEE Trans. Wireless Commun., vol. 1, no. 7, pp. 415–428, Jul. 2002. [29] C. Rose, “CDMA codeword optimization: Interference avoidance and convergence via class warfare,” IEEE Trans. Inf. Theory, vol. 47, no. 9, pp. 2368–2382, Sep. 2001. [30] H. L. Van Trees, Detection Estimation and Modulation Theory, Part I. New York: Wiley, 2001. [31] T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991. [32] P. Anigstein and V. Anantharam, “Ensuring convergence of the MMSE iteration for interference avoidance to the global optimum,” IEEE Trans. Inf. Theory, vol. 49, no. 4, pp. 873–885, Apr. 2003. [33] M. Rupf and J. L. Massey, “Optimum sequence multisets for syncronous code-division-multiple-access channels,” IEEE Trans. Inf. Theory, vol. 40, no. 4, pp. 1261–1266, Jul. 1994. [34] P. Viswanath and V. Anantharam, “Optimal sequences for CDMA under colored noise: A Schur-Saddle function property,” IEEE Trans. Inf. Theory, vol. 48, no. 6, pp. 1295–1318, Jun. 2002. [35] G. N. Karystinos and D. A. Pados, “New bounds on the total squared correlation and optimum design of DS-CDMA binary signature sets,” IEEE Trans. Commun., vol. 51, no. 1, pp. 48–51, Jan. 2003. [36] C. Ding, M. Golin, and T. Kløve, “Meeting the Welch and KarystinosPados bounds on DS-CDMA binary signature sets,” Des., Codes Cryptogr., vol. 30, pp. 73–84, Aug. 2003. [37] P. Ipatov, “On the Karystinos-Pados bounds and optimal binary DS-CDMA signature ensembles,” IEEE Commun. Lett., vol. 8, no. 2, pp. 81–83, Feb. 2004. [38] G. N. Karystinos and D. A. Pados, “The maximum squared correlation, sum capacity, and total asymptotic efficiency of minimum totalsquared-correlation binary signature sets,” IEEE Trans. Inf. Theory, vol. 51, no. 1, pp. 348–355, Jan. 2005.

GKIZELI et al.: OPTIMAL SIGNATURE DESIGN

[39] P. Viswanath, D. N. C. Tse, and V. Anantharam, “Asymptotically optimal waterfilling in multiple antenna multiple access channels,” IEEE Trans. Inf. Theory, vol. 47, no. 1, pp. 241–267, Jan. 2001. [40] C. D. Meyer, Matrix Analysis and Applied Linear Algebra.. Philadelphia, PA: SIAM, 2000. [41] USC-SIPI Image Database, [Online]. Available: http://sipi.usc.edu/ database/database.cgi?volume=misc

Maria Gkizeli (S’00–M’03) received the Diploma degree in electrical engineering from the Democritus University of Thrace, Xanthi, Greece, in 1997, and the M.Sc. and Ph.D. degrees in mobile and satellite communications from the University of Surrey, Guildford, U.K., in 1998 and 2002, respectively. From September 2002 to September 2004, she was a Postdoctoral Fellow with the Communications and Signals Laboratory, Department of Electrical Engineering, State University of New York at Buffalo, where she also held an appointment as an Adjunct Instructor from August 2003 to May 2004. Since September 2004, she has been a Visiting Lecturer with the Department of Electronics, Technological Education Institute of Crete, Chania, Greece. Her research interests lie in the areas of communication theory and systems, covert communications, wireless networks, satellite communications, and image processing. Dr. Gkizeli is a member of the IEEE Signal Processing Society and the Technical Chamber of Greece.

405

Dimitris A. Pados (M’95) was born in Athens, Greece, on October 22, 1966. He received the Diploma degree in computer science and engineering (five-year program) from the University of Patras, Patras, Greece, in 1989, and the Ph.D. degree in electrical engineering from the University of Virginia, Charlottesville, VA, in 1994. From 1994 to 1997, he held an Assistant Professor position in the Department of Electrical and Computer Engineering and the Center for Telecommunications Studies, University of Louisiana, Lafayette. Since August 1997, he has been with the Department of Electrical Engineering, State University of New York at Buffalo, where he is presently an Associate Professor. His research interests are in the general areas of communication theory and adaptive signal processing with an emphasis on wireless multiple access communications, spread-spectrum theory and applications, coding, and sequences. Dr. Pados is a member of the IEEE Communications, Information Theory, Signal Processing, and Computational Intelligence Societies. He served as an Associate Editor for the IEEE SIGNAL PROCESSING LETTERS from 2001 to 2004 and the IEEE TRANSACTIONS ON NEURAL NETWORKS from 2001 to 2005. He received a 2001 IEEE International Conference on Telecommunications best paper award and the 2003 IEEE Transactions on Neural Networks Outstanding Paper Award for articles that he coauthored with his students.

Michael J. Medley (S’91–M’95–SM’02) received the B.S., M.S., and Ph.D. degrees in electrical engineering from Rensselaer Polytechnic Institute, Troy, NY, in 1990, 1991, and 1995, respectively. Since 1991, he has been a Research Engineer for the United States Air Force at the Air Force Research Laboratory, Rome, NY, where he has been involved in adaptive interference suppression and spread-spectrum waveform design. In 2002, he joined the State University of New York Institute of Technology, Utica, as an Assistant Professor. His research interests include transform domain signal processing, adaptive filtering, steganography/steganalysis, and spread-spectrum communications.