A Linear Programming Approach to Semidefinite ...

0 downloads 0 Views 220KB Size Report
May 3, 2001 - A semidefinite programming problem can be regarded as a convex nonsmooth optimization problem, so it can be represented as a semi-infinite ...
A Linear Programming Approach to Semidefinite Programming Problems Kartik Krishnan Department of Mathematical Sciences Rensselaer Polytechnic Institute Troy, NY 12180 USA [email protected] http://www.rpi.edu/˜kartis Joint work with John Mitchell. Work supported by NSF grant number CCR-9901822. Candidacy exam RPI, Troy, NY 12180 3rd May 2001

Solving SDP via LP

1

Abstract A semidefinite programming problem can be regarded as a convex nonsmooth optimization problem, so it can be represented as a semi-infinite linear programming problem. Thus, in principle, it can be solved using a cutting plane approach; we describe such a method. The cutting plane method uses an interior point algorithm to solve the linear programming relaxations approximately, because this resulted in the generation of better constraints than a simplex cutting plane method. Further, the bundle method of Helmberg and Rendl can be used to generate a set of linear constraints. Typically, these alone are not sufficient to give a good linear programming relaxation. Nonetheless, if they are used in conjunction with some families of problem specific constraints, tight LP relaxations can be obtained.

Solving SDP via LP

2

Contents 1. Linear algebra preliminaries. 2. The semidefinite programming (SDP) problem. 3. Similarities with linear programming (LP) problem. 4. Some duality theory and geometry in SDP. 5. Applications of semidefinite programming. 6. An interior point approach to solving SDP. 7. Limitations of the interior point approach. 8. Nonlinear formulations of the SDP. 9. SDP as an semi-infinite LP. 10. Dual vs primal. 11. Regaining a primal SDP solution. 12. The Spectral Bundle method. 13. Rationale for using the bundle constraints. 14. Computational results using the bundle LP approach. 15. Cutting plane LP approach and our refinements. 16. Cutting plane LP results. 17. Conclusions.

3

Solving SDP via LP

1

Linear algebra preliminaries • S n is the set of symmetric matrices of size n. • S n is isomorphic to R

n(n+1) 2

n+1 2

or R(

).

• A ∈ S n is symmetric positive semidefinite (psd) ⇔ dT Ad ≥ 0, ∀d.

• S+n is the set of all symmetric psd matrices of size n. • A ∈ S n is symmetric positive definite (pd) ⇔ dT Ad > 0, ∀d ∈ Rn \{0}.

n • S++ is the set of all symmetric pd matrices of size n.

• X ∈ S n is psd (pd) ⇔ λi ≥ 0(> 0), ∀i = 1, . . . , n. • For A ∈ S n , Trace(A) = tr(A) =

Pn

i=1 Aii.

• For A, C ∈ S n , C•X = Trace(CX) =

Pn

• For A ∈ S n we have A = P ΛA P T =

i=1

Pn

j=1 Cij Xij .

T i=1 λi pi pi

Pn

(Spectral decomposition). P is a matrix of eigenvectors and ΛA is a diagonal matrix containing the eigenvalues of A.

4

Solving SDP via LP

2

Linear algebra preliminaries (cont.) • Positive semidefiniteness of A ∈ S n can be checked in

n3 3

arithmetic operations. (Cholesky factorisation)

• Every A ∈ S+n has a unique kth root B i.e. B k = √ A, ∀k ≥ 1. Since A = P ΛAP T , B = P k ΛAP T . • A set C ⊆ Rn is a cone if (x, y ∈ C ⇒ λ(x + y) ∈ C, ∀λ ≥ 0).

n+1 2

• S+n is a closed cone in R(

).

n is not a cone because 0 ∈ / S++ n. • Note that S++ n • S++ is the interior of the cone S+n .

• Cones (optimisation over them) are n 1. K = R+ the positive orthant

(Linear Programming). 2. K = {(

τ

) ∈ Rn+1 : τ ≥ ||x||2}

x (Second order cone programming).

3. K = S+n (Semidefinite programming).

5

Solving SDP via LP

3

The SDP problem

Standard form: min

C•X

subject to A(X) = b

(SDP )

X  0,

with dual bT y

max

subject to AT y + S = C

(SDD)

S  0

• X and S are square matrices. • Constraints require that X and S be positive semidefinite.

• Inner products are the Frobenius inner product. 



A1 • X   Pk  . T . , A y = • AX = . i=1 yj Aj , where each Aj         

 

Ak • X is a symmetric matrix.

• C is symmetric and b is an k-vector.

6

Solving SDP via LP

4

Similarities with the LP problem Primal SDP

LP

min C • X

min cT x

A(X) = b s.t.

s.t.

Ax = b

X  0,

x ≥ 0,

Dual bT y

max s.t.

bT y

max

AT y + s = c

AT y + S = C s.t.

s ≥ 0

S  0

• Matrices C, X, S, Ai ∈ S n ⇔ Vectors c, x, s, ai ∈ Rn

• C•X =

P

i,j

Cij Xij ⇔ cT x =

Pn

i=1 ci xi

n • X  0 ∈ S+n ⇔ x ≥ 0 ∈ R+ 

A1 • X ... • AX =        

• AT y =

Ak • X

Pk

        





T  a1 x   

 ⇔ Ax =  ...

T i=1 yi Ai ⇔ A y =

 

aTk x

Pk

     

i=1 yi ai

7

Solving SDP via LP

5

Duality and geometry in SDP

5.1

Strong and weak duality

Theorem 1 (Weak Duality) If X is feasible in (SDP) and (y,S) in (SDD) then C • X − bT y = X • S ≥ 0 Strong duality holds under a strict feasibility (Slater) condition Theorem 2 (Strong duality) Suppose (SDP) and (SDD) have strictly feasible points. Then both (SDP) and (SDD) attain their optimal solutions, and these optimal values are equal 5.2

Rank of extreme matrices in SDP

Theorem 3 (i) Let F be a face of dimension m of the feasible set of (SDP). For X ∈ F the rank r of X is bounded by



r+1 2

≤ m+k

8

Solving SDP via LP

6

Applications of the SDP

6.1

Minimising the maximum eigenvalue

Given : A matrix M ∈ S n depending linearly on a vector z ∈ Rk .

Objective : Choose z so as to minimise λmax(M(z)) (the maximum eigenvalue) λmax(M(z)) ≤ η ⇔ λmax(M(z) − ηI) ≤ 0 ⇔ λmin(ηI − M(z)) ≥ 0 ⇔

ηI − M(z)  0

This gives the SDP max

−η

s.t. ηI − M(z)  0 This is an SDP in dual form with the dual variable y = (η; z)

9

Solving SDP via LP

6.2

Convex quadratically constrained programming

SDP contains nonlinear (convex) programming problems. For example max dT y s.t. fi (y) = y T Ai y − bTi y − ci ≤ 0 i = 1, . . . , k The constraints fi(y) ≤ 0 are convex. Here Ai ∈ S+n , i = 1, . . . , k. Let Ai = UiT Ui Note that fi(y) ≤ 0 ⇔ (

I

Ui y T

(Uiy)

bTi y

+ ci

)  0.

Thus we have the SDP dT y

max s.t. (

I

Ui y T

(Ui y)

bTi y

+ ci

)  0 i = 1, . . . , k

10

Solving SDP via LP

6.3

The max cut problem

Given : A graph G = (V, E) and a nonnegative weight E vector w = wij ∈ R+ . A cut δ(S) where S ⊆ V is

defined as follows

δ(S) = {(i, j) ∈ E : i ∈ S, j ∈ / S} The weight of the cut is w(δ(S)) =

P

i,j∈δ(S) wij

Objective : Find the cut δ(S) with maximum weight • x ∈ Rn with xi = 1 if i ∈ S and −1 otherwise. • xixj = −1 if (i, j) ∈ δ(S) and 1 otherwise. • Define the Laplacian matrix L ∈ S n as follows Lii = Lij =

wij j 4 w − 4ij

P

i = 1, . . . , n i 6= j

The weight of the cut is w(δ(S)) =

1P 2 i



k to

tighten our LP relaxations.

• The main computational task in the LP approach is in computing the optimal bundle P . Solving the resulting LP relaxations is relatively trivial.

Solving SDP via LP

59

References [1] B. Borchers. SDPLIB 1.2, a library of semidefinite programming test problems. Optimization Methods and Software, 11:683–690, 1999. [2] C. Helmberg. Semidefinite programming for combinatorial optimization. Technical Report ZR-00-34, TU Berlin, Konrad-Zuse-Zentrum, Berlin, October 2000. Habilitationsschrift. [3] A. Lisser and F. Rendl. Telecommunication clustering using linear and semidefinite programming. Technical report, Instit¨ ut f¨ ur Mathematik, Universit¨at Klagenfurt, A – 9020 Klagenfurt, Austria, November 2000. [4] J. E. Mitchell. Realignment in the NFL. Technical report, Mathematical Sciences, Rensselaer Polytechnic Institute, Troy, NY 12180, November 2000. [5] G. Pataki and S. H. Schmieta. The DIMACS of mixed semidefinite-quadratic-linear programs, http://dimacs.rutgers.edu/Challenges/Seventh/Instances/.

library 2000.

[6] K. C. Toh, M. J. Todd, and R. Tutuncu. SDPT3 — a Matlab software package for semidefinite programming. Optimization Methods and Software, 11:545–581, 1999. [7] Y. Ye. Approximating quadratic programming with bound and quadratic constraints. Mathematical Programming, 84:219–226, 1999.