## Canonical Solutions to Nonconvex Minimization Problems over

Oct 19, 2012 - Sa = {Ï â R | Ï â¥ 0, det G(Ï) = 0}. ... By the standard procedure of the canonical dual transformation, we rewrite the ... x1 in the quadratic form 1 ... Since V (Ç«) is a proper closed convex function over Râ := {Ç« â R| Ç« â¤ 0}, we ...

Canonical Solutions to Nonconvex Minimization Problems over Lorentz Cone Ning Ruan1,2 and David Yang Gao1 1. Graduate School of Information Technology and Mathematical Science,

arXiv:1210.5300v1 [math.OC] 19 Oct 2012

University of Ballarat, Ballarat, Vic 3353, Australia. 2. Department of Mathematics and Statistics, Curtin University of Technology, Perth, WA 6845, Australia. Abstract This paper presents a canonical dual approach for solving nonconvex quadratic minimization problem. By using the canonical duality theory, nonconvex primal minimization problems over n-dimensional Lorentz cone can be transformed into certain canonical dual problems with only one dual variable, which can be solved by using standard convex minimization methods. Extremality conditions of these solutions are classified by the triality theory. Applications are illustrated.

Key Words: Conical optimization; nonlinear programming; constrained minimization; canonical duality; NP-hard problems; global optimization.

1

Primal Problem and It’s Canonical Dual

The primal problem (P) proposed to solve is the so-called second order cone programming: (P) :



1 min P (x) = hx, Qxi − hx, ci : x ∈ C 2



,

(1)

where Q ∈ Rn×n is a given symmetrical matrix; c ∈ Rn is a given vector; C ⊂ Rn is the so-called Lorentz cone in Rn : C = {(x1 , x2 ) ∈ R × Rn−1 | kx2 k ≤ x1 , x1 > 0},

(2)

which is a special second order cone. The problem (P) appears in many applications such as structural optimization, filter design, and grasping force optimization in robotics. Extensive research has been focused on this subject. In this paper we present a canonical dual approach for solving the second order cone optimization problem (P). By using the canonical duality theory developed in 1

[2, 4, 7, 8], the canonical dual problem (P d ) can be formulated as   1 d −1 max P (σ) = − hc, G (σ)ci : σ ∈ Sa , 2

(3)

where G(σ) = Q + σLo , Lo =

−1

0

0

In−1

!

is the Lorentz matrix, in which, In−1 is an identical matrix in R(n−1)×(n−1) . The dual feasible space Sa ⊂ R is defined by Sa = {σ ∈ R | σ ≥ 0, det G(σ) 6= 0}.

(4)

Theorem 1 The problem (P d ) is canonically dual to (P) in the sense that the vector σ ¯ ∈ Sa is a KKT point of (P d ) if and only if the vector ¯ = G−1 (¯ x σ )c

(5)

P (¯ x) = P d (¯ σ ).

(6)

is a KKT point of (P), and

Proof. By the standard procedure of the canonical dual transformation, we rewrite the cone constraint kx2 k ≤ x1 in the quadratic form 21 xT Lo x ≤ 0 and introduce a nonlinear transformation (i.e. the so-called geometrical mapping) ǫ = Λ(x) = 12 xT Lo x : Rn → R. Thus, the cone constraint x ∈ C can be replaced identically by ǫ(x) = Λ(x) ≤ 0. Let ( 0 if ǫ ≤ 0, V (ǫ) = (7) +∞ otherwise. The primal problem (P) can be written in the following canonical form :   1 T T n (Pc ) : min P (x) = V (Λ(x)) + x Qx − c x | x ∈ R . 2

(8)

According to the Fenchel transformation, the sup-conjugate V ♯ of the function V (ǫ) is defined by V ♯ (σ) = sup{ǫT σ − V (ǫ)} = ǫ∈R

(

0

if σ ≥ 0,

+∞

otherwise.

Since V (ǫ) is a proper closed convex function over R− := {ǫ ∈ R| ǫ ≤ 0}, we know that σ ∈ ∂V (ǫ) ⇔

ǫ ∈ ∂V ♯ (σ) ⇔ 2

V (ǫ) + V ♯ (σ) = ǫσ.

(9)

The pair of (ǫ, σ) is then called a generalized canonical dual pair on R− × R+ by the definition introduced in [2, 4]. Following the original idea of Gao and Strang , we replace V (Λ(x)) in equation (8) by the Fenchel-Young equality V (Λ(x)) = Λ(x)T σ − V ♯ (σ). Then the so-called total complementary function Ξ(x, σ) : Rn × R+ → R associated with the problem (Pc ) can be defined as below 1 Ξ(x, σ) = Λ(x)T σ − V ♯ (σ) + xT Qx − cT x. 2

(10)

By the definition of Λ(x) and V ♯ (σ), on Rn × R+ we have 1 Ξ(x, σ) = xT G(σ)x − xT c. 2

(11)

The criticality condition of Ξ(x, σ) leads to the equilibrium equation G(σ)x = c,

(12)

σ ≥ 0, xT Lo x ≤ 0, σxT Lo x = 0.

(13)

and the KKT conditions

Substituting (12) into the canonical dual transformation P d (σ) = sta{Ξ(x, σ) | x ∈ C}, the canonical dual function P d (σ) is then formulated. It is easy to prove that if σ ¯ ≥ 0 is a KKT point of (P d ), then we have σ ¯ ≥ 0,

¯ = 1x ¯ (σ) ¯ T L0 x ¯ (σ) ¯ ≤ 0, ∇P d (λ) 2

1 ¯ (σ) ¯ T L0 x ¯ (σ)) ¯ = 0, σ ¯·( x 2

(14) (15)

¯ (¯ ¯ (¯ where x σ ) = G−1 (¯ σ )c. This shows that x σ ) is also a KKT point of the primal problem (P). ¯ = G−1 (σ)c, we have By the complementarity condition (15), and the fact of x 1 P d(¯ σ ) = − cT G−1 (¯ σ )c 2 1 T −1 c G (¯ σ )c − cT G−1 (¯ σ )c = 2 1 −1 = (G (¯ σ )c)T G(¯ σ )G−1 (¯ σ )c − cT G−1 (¯ σ )c 2 1 −1 = (G (¯ σ )c)T (Q + σ ¯ L0 )G−1 (¯ σ )c − cT G−1 (¯ σ )c 2 3

σ ¯ 1 −1 (G (¯ σ )c)T QG−1 (¯ σ )c − cT G−1 (¯ σ )c + (G−1 (¯ σ )c)T L0 G−1 (¯ σ )c 2 2 1 −1 (G (¯ σ )c)T QG−1 (¯ σ )c − cT G−1 (¯ σ )c = 2 1 T ¯ Q¯ ¯ = x x − cT x 2 = P (¯ x) =

This proves the theorem.

2



Extremality Conditions

Let Sa+ = {σ ∈ R | σ ≥ 0, G(σ) ≻ 0}.

(16)

Theorem 2 Suppose that σ ¯ ∈ Sa is a solution to (P) and ¯ = G−1 (¯ x σ )c. ¯ is a global minimizer of P (x) on C and If σ ¯ ∈ Sa+ , then x σ ). P (¯ x) = min P (x) = max+ P d (σ) = P d (¯ x∈C

(17)

σ∈Sa

Proof. By Theorem 1 we know that the vector σ ¯ ∈ Sa is a KKT point of the ¯ = G−1 (¯ problem (P d ) if and only if x σ )c is a KKT point of the problem (P), and P (¯ x) = Ξ(¯ x, σ ¯ ) = P d (¯ σ ).

(18)

Particularly, if σ ¯ ∈ Sa+ , the canonical dual function P d (σ) is concave. In this case, the total complementary function Ξ is a saddle function, i.e., it is convex in x ∈ Rn and concave in σ ∈ Sa+ . Thus, we have P d (¯ σ ) = max+ P d (σ) σ∈Sa

= max+ minn Ξ(x, σ) = minn max+ Ξ(x, σ) x∈R σ∈Sa σ∈Sa x∈R     1 T 1 T T ♯ = minn x Qx − x c + max+ x L0 x σ − V (σ) = min P (x) x∈R x∈C 2 2 σ∈Sa due to the fact that V (Λ(x)) = max+ σ∈Sa



  ( 0 1 T x L0 x σ − V ♯ (σ) = 2 ∞ 4

if x ∈ C, otherwise.

From Theorem 1 we have (17).

 n

In a special case when Q = Diag (q) is a diagonal matrix with q = {qi } ∈ R being its diagonal elements, we have −1

G (σ) =



1 qi + σδi−

In this case,



,

(19)

n

P d (σ) = − where δi− =

1X c2i , 2 i=1 qi + σδi−

n −1 i = 1 1

i 6= 1

, δi+ =

(20)

n 1 i=1 0 i 6= 1

.

(21)

For the given {ci }, and {qi } such that −qn ≤ qn−1 ≤ . . . , ≤ q1 , the dual variable σ can be solved completely within each interval −qi+1 < σ < −qi or −q2 < σ < q1 , such that qi < qi+1 (i = 2, · · · , n).

3

Applications

We now list a few examples to illustrate the applications of the theory presented in this paper.

3.1

Two-D nonconvex minimization

First of all, let us consider two dimensional concave minimization problem:   1 2 2 2 (P) : min P (x) = (q1 x1 + q2 x2 ) − c1 x1 − c2 x2 : kx2 k ≤ x1 , (x1 , x2 ) ∈ R , 2 (22) On the dual feasible set Sa = {σ ∈ R | σ ≥ 0, (q1 − σ)(q2 + σ) 6= 0}, the canonical dual function has the form of  1 1 q1 −σ d T P (σ) = − [c1 , c2 ] 2

1 q2 +σ



c1 c2



.

(23)

(24)

Assume q1 = 0.1, q2 = −0.3, c1 = 0.5, c2 = −0.3, so we have σ = 0.45 ∈ Sa+ = {σ ∈ R| 0.3 < σ < 0.5}.

(25)

By Theorem 1, we know that x = {c1 /(q1 − σ), c2 /(q2 + σ)}= {2, −2} is a global minimizer, it is easy to verify that P (x) = P d (σ) = −0.4(see Figure 1-2). 5

3

2

1

2 0

0

2 -1

-2 0

-2

-2

0 -2

-3

2

-3

-1

-2

1

0

2

3

Figure 1: Graphs of P (x) and its contour for two dimensional problem. 3

2

1

0.2

0.4

0.6

0.8

1

-1

-2

-3

Figure 2: Graph of P d (σ) for two dimensional problem.

3.2

Two-D general nonconvex minimization 

1 (P) : min P (x) = (q1 x21 + q2 x22 + 2q3 x1 x2 ) − c1 x1 − c2 x2 : kx2 k ≤ x1 , (x1 , x2 ) ∈ R2 2 (26) On the dual feasible set Sa = {σ ∈ R2 | σ ≥ 0, (q1 − σ)(q2 + σ) − q32 6= 0}, the canonical dual function has the form of  q1 − σ 1 T d P (σ) = − [c1 , c2 ] 2 a3

q3 q2 + σ

−1 

c1 c2



(27)

(28)

If we choose q1 = 1.8, q2 = −0.6, q3 = 0.4, c1 = 0.5, c2 = 0.6, then we have σ = 1.29 ∈ Sa+ = {σ ∈ R| (q1 − σ)(q2 − σ) − a23 > 0}. (29) −1    c1 q1 − σ q3 = {0.5500, 0.5499} is a By Theorem 1, we know that x = c2 q3 q2 + σ global minimizer, and P (x) = P d (σ) = −0.3025(see Figure 3-4). 6



,

3

2

1

10 5 0 -5

0

2 -1

0

-2

-2

0

-2 -3

2

-3

-2

-1

0

1

2

3

Figure 3: Graphs of P (x) and its contour for general two dimensional problem. 6 4 2 1

0.5

1.5

2

-2 -4 -6

Figure 4: Graph of P d (σ) for general two dimensional problem.

3.3

Three-D general nonconvex minimization

  1 3 (P) : min P (x) = hx, Qxi − hx, ci : kx2 k ≤ x1 , (x1 , x2 ) ∈ R , 2   q11 q12 q13   where Q =  q12 q22 q23 , c = {c1 , c2 , c3 }. On the dual feasible set q13

where

q23

(30)

q33

Sa = {σ ∈ R | σ ≥ 0, detG(σ) 6= 0},

(31)

(32)

(33)

 G(σ) = 

q11 − σ

q12

q13

q12

q22 + σ

q23

q13

q23

q33 + σ

 ,

the canonical dual function has the form of c1

1   P d(σ) = − [c1 , c2 , c3 ]T G−1 (σ)  c2  2 c3 7

Suppose q11 = 2, q22 = −2, q33 = 1, q12 = −1, q13 = 2, q23 = 0, c1 = 1.5, c2 = −0.5, c3 = 1.5, we have σ = 0.4509 ∈ Sa+ = {σ ∈ R| σ ≥ 0, G(σ) ≻ 0}. (34)   c1   By theorem 1, we know that x = G−1 (σ)  c2 = {0.4355, 0.0416, 0.4335} is a global

c3 minimizer, and P (x) = P d (σ) = −0.6413(see Figure 5). 2 1 0.5

1

1.5

2

2.5

3

3.5

-1 -2 -3 -4

Figure 5: Graph of P d (σ) for general three dimensional problem.

4

Conclusions

We have presented a concrete application of the canonical dual transformation and triality to conic optimization problems. Results show that by the use of this method, the nonconvex cone constrained problem (P) in Rn can be reformulated as a perfect dual problem in R, also the KKT points and extremality conditions of the originally difficult problems are idetified by Theorem 1 and 2. Physically speaking, each optimal point represents a stable equilibrium state of the system. Duality theory reveals the intrinsic pattern of duality relations of these critical points, and plays an important role in nonconvex analysis, detailed study and comprehensive applications of this theory were presented in monograph .

Acknowledgment This paper was partially supported by a grant (AFOSR FA9550-10-1-0487) from the US Air Force Office of Scientific Research. Dr. Ning Ruan was supported by a funding 8

from the Australian Government under the Collaborative Research Networks (CRN) program.

References  Gao, D.Y.(1998). Duality, triality and complementary extremun principles in nonconvex parametric variational problems with applications, IMA J. Appl. Math., 61, 199-235.  Gao, D.Y.(2000). Duality Principles in Nonconvex Systems: Theory, Methods and Applications. Kluwer Academic Publishers, Dordrecht/Boston/London, xviii+454pp.  Gao, D.Y.(2000). Analytic solution and triality theory for nonconvex and nonsmooth vatiational problems with applicatons, Nonlinear Analysis, 42(7), 11611193.  Gao, D.Y. (2000). Canonical dual transformation method and generalized triality theory in nonsmooth global optimization, J. Global Optimization, 17 (1/4), pp. 127-160.  Gao, D.Y. (2001). Tri-duality in global optimization, in Encyclopedia of Optimization, C. A. Floudas and P.D. Pardalos (eds). Kluwer Academic Publishers, Dordrecht/Boston/London, Vol. 1, pp. 485-491.  Gao, D.Y. (2003). Nonconvex semi-linear problems and canonical dual solutions. Advances in Mechanics and Mathematics, Vol. II, Kluwer Academic Publishers. pp. 261-312.  Gao, D.Y. (2003). Perfect duality theory and complete solutions to a class of global optimization problems, Optimisation, 52 (4-5), pp. 467-493.  Gao, D.Y. (2004). Complete solutions to constrained quadratic optimization problems, J. Global Optimisation, special issue on Duality. 29, 377-399.  Gao, D.Y.(2005). Sufficient conditions and perfect duality in nonconvex minimization with inequality constraints, J. Industrial and Management Optimization, 1, 59-69. 9

 Gao, D.Y.(2006). Complete solutions and extremality criteria to polynomial optimization problems, J. Global Optimization, 35, 131-143.  Gao, D.Y.(2007). Solutions and optimality criteria to box constrained nonconvex minimization problem, J. Industrial and Management Optimization, 3(2), 293-304.  Gao, D.Y. and Ruan, N. (2008). Solutions and optimality criteria for nonconvex quadratic-exponential minimization problem, Mathematical Methods of Operations Research, 67(3), 479-496.  Gao, D.Y. and Ruan, N. (2010). Solutions to quadratic minimization problems with box and integer constraints, J. Global Optimization, 47(3): 463-484.  Gao, D.Y., Ruan, N., and Sherali, H.D.(2009). Solutions and optimality criteria for nonconvex constrained global optimization problems, J. Global Optimization, 45(3): 473-497.  Gao, D.Y., Ruan, N. and Sherali, H.D. (2010). Canonical dual solutions for fixed cost quadratic program, Optimization and Optimal Control: Theory and Applications, A. Chinchuluun, P.M. Pardalos, R. Enkhbat and L. Tseveendorj (eds.), Springer, 39: 139-156.  Gao, D.Y. and Strang, G.(1989). Geometric nonlinearity: Potential energy, complementary energy, and the gap function. Quart. Appl. Math., 47(3), 487-504.  Gao, D.Y. and Sherali, H.D.(2006). Canonical Duality Theory: Connections between nonconvex mechanics and global optimization Advances in Mechanics and Mathematics, Vol. III, Springer.  Hansen, P., Jaumard, B., Ruiz, M., and Xiong, J.(1991). Global minimization of indefinite quadratic functions subjects to box constraints. Technical report, Tech´ nical Report G-91-54, Gread, Ecole Polytechnique, Universit´e McGill, Montreal.  Horst, R., Pardalos, P.M., and Thoai, N.V.(2000). Introduction to Global Optimization. Kluwer Academic Publishers.  Li, S.F. and Gupta, A.(2006). On dual configuration forces, J. of Elasticity, 84, 13-31.

10

 Ruan, N., Gao, D.Y., and Jiao, Y. (2010). Canonical dual least square method for solving general nonlinear systems of equations, Computational Optimization with Applications Computational Optimization with Applications, 47(2): 335-347.

11