Experiments in Robust Portfolio Optimization - Columbia University

0 downloads 0 Views 1003KB Size Report
Sep 27, 2007 - Daniel Bienstock ( Columbia University, New York). Experiments in .... a set (“tier”) Th of assets, and a parameter Γh > 0 for each h, ∑j∈Th ..... 5. W 7.406e-01 1.602e-01 6.084e-02 2.781e-02 5.313e-03. N. 37. 18. 12. 9. 2.
Experiments in Robust Portfolio Optimization Daniel Bienstock Columbia University, New York

27th September 2007

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

1 / 56

Mean-Variance Portfolio Optimization H. Markowitz – 1950s min λx T Qx − µ ¯T x Subject to: Ax ≥ b µ ¯ = vector of “expected returns”, Q = “covariance” matrix x = vector of “asset weights” Ax ≥ b: portfolio construction constraints λ ≥ 0 = “risk-aversion” multiplier → want to model errors in µ ¯

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

2 / 56

Mean-Variance Portfolio Optimization H. Markowitz – 1950s min λx T Qx − µ ¯T x Subject to: Ax ≥ b µ ¯ = vector of “expected returns”, Q = “covariance” matrix x = vector of “asset weights” Ax ≥ b: portfolio construction constraints λ ≥ 0 = “risk-aversion” multiplier → want to model errors in µ ¯

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

2 / 56

Robust Optimization Optimization under parameter (data) uncertainty Ben-Tal and Nemirovsky, El Ghaoui et al Bertsimas et al Uncertainty is modeled by assuming that data is not known precisely, and will instead lie in known sets. Example: a coefficient ai is uncertain. We allow ai ∈ [li , ui ]. Typically, a minimization problem becomes a min-max problem.

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

3 / 56

Robust Optimization Optimization under parameter (data) uncertainty Ben-Tal and Nemirovsky, El Ghaoui et al Bertsimas et al Uncertainty is modeled by assuming that data is not known precisely, and will instead lie in known sets. Example: a coefficient ai is uncertain. We allow ai ∈ [li , ui ]. Typically, a minimization problem becomes a min-max problem.

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

3 / 56

Robust Optimization Optimization under parameter (data) uncertainty Ben-Tal and Nemirovsky, El Ghaoui et al Bertsimas et al Uncertainty is modeled by assuming that data is not known precisely, and will instead lie in known sets. Example: a coefficient ai is uncertain. We allow ai ∈ [li , ui ]. Typically, a minimization problem becomes a min-max problem.

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

3 / 56

Robust Optimization Optimization under parameter (data) uncertainty Ben-Tal and Nemirovsky, El Ghaoui et al Bertsimas et al Uncertainty is modeled by assuming that data is not known precisely, and will instead lie in known sets. Example: a coefficient ai is uncertain. We allow ai ∈ [li , ui ]. Typically, a minimization problem becomes a min-max problem.

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

3 / 56

Robust Optimization Optimization under parameter (data) uncertainty Ben-Tal and Nemirovsky, El Ghaoui et al Bertsimas et al Uncertainty is modeled by assuming that data is not known precisely, and will instead lie in known sets. Example: a coefficient ai is uncertain. We allow ai ∈ [li , ui ]. Typically, a minimization problem becomes a min-max problem.

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

3 / 56

Robust Optimization Optimization under parameter (data) uncertainty Ben-Tal and Nemirovsky, El Ghaoui et al Bertsimas et al Uncertainty is modeled by assuming that data is not known precisely, and will instead lie in known sets. Example: a coefficient ai is uncertain. We allow ai ∈ [li , ui ]. Typically, a minimization problem becomes a min-max problem.

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

3 / 56

Robust Portfolio Optimization How to handle uncertain returns min λx T Qx − µ ¯T x Subject to: Ax ≥ b

→ M = set of allowable return vectors (“around” µ) ¯  minx maxµ∈M λx T Qx − µT x Subject to: Ax ≥ b

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

4 / 56

Robust Portfolio Optimization How to handle uncertain returns min λx T Qx − µ ¯T x Subject to: Ax ≥ b

→ M = set of allowable return vectors (“around” µ) ¯  minx maxµ∈M λx T Qx − µT x Subject to: Ax ≥ b

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

4 / 56

Robust Portfolio Optimization How to handle uncertain returns min λx T Qx − µ ¯T x Subject to: Ax ≥ b

→ M = set of allowable return vectors (“around” µ) ¯  minx maxµ∈M λx T Qx − µT x Subject to: Ax ≥ b

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

4 / 56

Robust Portfolio Optimization How to handle uncertain returns min λx T Qx − µ ¯T x Subject to: Ax ≥ b

→ M = set of allowable return vectors (“around” µ) ¯  minx λx T Qx − minµ∈M µT x Subject to: Ax ≥ b

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

5 / 56

Robust Portfolio Optimization How to handle uncertain returns min λx T Qx − µ ¯T x Subject to: Ax ≥ b

→ M = set of allowable return vectors (“around” µ) ¯ 1

The investor chooses a vector x of assets

2

The adversary chooses a returns vector µ ∈ M so as minimize return: obtain µmin (x)

Robust problem: minx λx T Qx − µmin (x) Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

6 / 56

Robust Portfolio Optimization How to handle uncertain returns min λx T Qx − µ ¯T x Subject to: Ax ≥ b

→ M = set of allowable return vectors (“around” µ) ¯ 1

The investor chooses a vector x of assets

2

The adversary chooses a returns vector µ ∈ M so as minimize return: obtain µmin (x)

Robust problem: minx λx T Qx − µmin (x) Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

6 / 56

Robust Portfolio Optimization How to handle uncertain returns min λx T Qx − µ ¯T x Subject to: Ax ≥ b

→ M = set of allowable return vectors (“around” µ) ¯ 1

The investor chooses a vector x of assets

2

The adversary chooses a returns vector µ ∈ M so as minimize return: obtain µmin (x)

Robust problem: minx λx T Qx − µmin (x) Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

6 / 56

The histogram model Parameters: 0 ≤ γ1 ≤ γ2 ≤ . . . ≤ γK ≤ 1, integers 0 ≤ ni ≤ Ni , 1 ≤ i ≤ K for each asset j: µ ¯ j = expected return between ni and Ni assets j satisfy: (1 − γi )µ ¯ j ≤ µj ≤ (1 − γi−1 )µ ¯j P

j

µj ≥ Γ

P

µ ¯ j ; Γ > 0 a parameter

j

(R. Tut ¨ unc ¨ u) ¨ For 1 ≤ h ≤ H, a set (“tier”) Th of assets, and a parameter Γh > 0

for each h,

P

j∈Th

µj ≥ Γh

P

j∈Sh

µ ¯j

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

7 / 56

The histogram model Parameters: 0 ≤ γ1 ≤ γ2 ≤ . . . ≤ γK ≤ 1, integers 0 ≤ ni ≤ Ni , 1 ≤ i ≤ K for each asset j: µ ¯ j = expected return between ni and Ni assets j satisfy: (1 − γi )µ ¯ j ≤ µj ≤ (1 − γi−1 )µ ¯j P

j

µj ≥ Γ

P

µ ¯ j ; Γ > 0 a parameter

j

(R. Tut ¨ unc ¨ u) ¨ For 1 ≤ h ≤ H, a set (“tier”) Th of assets, and a parameter Γh > 0

for each h,

P

j∈Th

µj ≥ Γh

P

j∈Sh

µ ¯j

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

7 / 56

The histogram model Parameters: 0 ≤ γ1 ≤ γ2 ≤ . . . ≤ γK ≤ 1, integers 0 ≤ ni ≤ Ni , 1 ≤ i ≤ K for each asset j: µ ¯ j = expected return between ni and Ni assets j satisfy: (1 − γi )µ ¯ j ≤ µj ≤ (1 − γi−1 )µ ¯j P

j

µj ≥ Γ

P

µ ¯ j ; Γ > 0 a parameter

j

(R. Tut ¨ unc ¨ u) ¨ For 1 ≤ h ≤ H, a set (“tier”) Th of assets, and a parameter Γh > 0

for each h,

P

j∈Th

µj ≥ Γh

P

j∈Sh

µ ¯j

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

7 / 56

The histogram model Parameters: 0 ≤ γ1 ≤ γ2 ≤ . . . ≤ γK ≤ 1, integers 0 ≤ ni ≤ Ni , 1 ≤ i ≤ K for each asset j: µ ¯ j = expected return between ni and Ni assets j satisfy: (1 − γi )µ ¯ j ≤ µj ≤ (1 − γi−1 )µ ¯j P

j

µj ≥ Γ

P

µ ¯ j ; Γ > 0 a parameter

j

(R. Tut ¨ unc ¨ u) ¨ For 1 ≤ h ≤ H, a set (“tier”) Th of assets, and a parameter Γh > 0

for each h,

P

j∈Th

µj ≥ Γh

P

j∈Sh

µ ¯j

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

7 / 56

Example: 1000 assets; 3 sectors and 2 tiers Between 600 and 900 assets have losses of up to 1% Between 0 and 100 assets have losses between 1% and 3%

Between 0 and 50 assets have losses between 3% and 6%

The total loss among the top return decile assets is at most 4%

The total loss among all assets is at most 3% Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

8 / 56

Ambiguous chance-constrained models 1

The investor chooses a vector x ∗ of assets

2

The adversary chooses a probability distribution P for the returns vector

3

A random returns vector µ is drawn from P

→ Investor wants to choose x ∗ so as to minimize value-at-risk (conditional value at risk, etc.) Erdogan and Iyengar (2004), Calafiore and Campi (2004) → We want to model correlated errors in the returns Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

9 / 56

Ambiguous chance-constrained models 1

The investor chooses a vector x ∗ of assets

2

The adversary chooses a probability distribution P for the returns vector

3

A random returns vector µ is drawn from P

→ Investor wants to choose x ∗ so as to minimize value-at-risk (conditional value at risk, etc.) Erdogan and Iyengar (2004), Calafiore and Campi (2004) → We want to model correlated errors in the returns Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

9 / 56

Ambiguous chance-constrained models 1

The investor chooses a vector x ∗ of assets

2

The adversary chooses a probability distribution P for the returns vector

3

A random returns vector µ is drawn from P

→ Investor wants to choose x ∗ so as to minimize value-at-risk (conditional value at risk, etc.) Erdogan and Iyengar (2004), Calafiore and Campi (2004) → We want to model correlated errors in the returns Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

9 / 56

Ambiguous chance-constrained models 1

The investor chooses a vector x ∗ of assets

2

The adversary chooses a probability distribution P for the returns vector

3

A random returns vector µ is drawn from P

→ Investor wants to choose x ∗ so as to minimize value-at-risk (conditional value at risk, etc.) Erdogan and Iyengar (2004), Calafiore and Campi (2004) → We want to model correlated errors in the returns Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

9 / 56

Ambiguous chance-constrained models 1

The investor chooses a vector x ∗ of assets

2

The adversary chooses a probability distribution P for the returns vector

3

A random returns vector µ is drawn from P

→ Investor wants to choose x ∗ so as to minimize value-at-risk (conditional value at risk, etc.) Erdogan and Iyengar (2004), Calafiore and Campi (2004) → We want to model correlated errors in the returns Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

9 / 56

Ambiguous chance-constrained models 1

The investor chooses a vector x ∗ of assets

2

The adversary chooses a probability distribution P for the returns vector

3

A random returns vector µ is drawn from P

→ Investor wants to choose x ∗ so as to minimize value-at-risk (conditional value at risk, etc.) Erdogan and Iyengar (2004), Calafiore and Campi (2004) → We want to model correlated errors in the returns Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

9 / 56

Uncertainty set Given a vector x ∗ of assets, the adversary 1

2

Chooses a vector w ∈ R n (n = no. of assets) with 0 ≤ wj ≤ 1 for all j. Chooses a random variable 0 ≤ δ ≤ 1

→ Random return: µj = µ ¯ j (1 − δwj ) (¯ µ = nominal returns). Definition (Rockafellar and Uryasev): Given reals ν and 0 ≤ θ ≤ 1 the value-at-risk of x ∗ is the real ρ ≥ 0 such that Prob(ν − µT x ∗ ≥ ρ) ≥ θ → The adversary wants to maximize VaR Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

10 / 56

Uncertainty set Given a vector x ∗ of assets, the adversary 1

2

Chooses a vector w ∈ R n (n = no. of assets) with 0 ≤ wj ≤ 1 for all j. Chooses a random variable 0 ≤ δ ≤ 1

→ Random return: µj = µ ¯ j (1 − δwj ) (¯ µ = nominal returns). Definition (Rockafellar and Uryasev): Given reals ν and 0 ≤ θ ≤ 1 the value-at-risk of x ∗ is the real ρ ≥ 0 such that Prob(ν − µT x ∗ ≥ ρ) ≥ θ → The adversary wants to maximize VaR Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

10 / 56

Uncertainty set Given a vector x ∗ of assets, the adversary 1

2

Chooses a vector w ∈ R n (n = no. of assets) with 0 ≤ wj ≤ 1 for all j. Chooses a random variable 0 ≤ δ ≤ 1

→ Random return: µj = µ ¯ j (1 − δwj ) (¯ µ = nominal returns). Definition (Rockafellar and Uryasev): Given reals ν and 0 ≤ θ ≤ 1 the value-at-risk of x ∗ is the real ρ ≥ 0 such that Prob(ν − µT x ∗ ≥ ρ) ≥ θ → The adversary wants to maximize VaR Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

10 / 56

Uncertainty set Given a vector x ∗ of assets, the adversary 1

2

Chooses a vector w ∈ R n (n = no. of assets) with 0 ≤ wj ≤ 1 for all j. Chooses a random variable 0 ≤ δ ≤ 1

→ Random return: µj = µ ¯ j (1 − δwj ) (¯ µ = nominal returns). Definition (Rockafellar and Uryasev): Given reals ν and 0 ≤ θ ≤ 1 the value-at-risk of x ∗ is the real ρ ≥ 0 such that Prob(ν − µT x ∗ ≥ ρ) ≥ θ → The adversary wants to maximize VaR Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

10 / 56

Uncertainty set Given a vector x ∗ of assets, the adversary 1

2

Chooses a vector w ∈ R n (n = no. of assets) with 0 ≤ wj ≤ 1 for all j. Chooses a random variable 0 ≤ δ ≤ 1

→ Random return: µj = µ ¯ j (1 − δwj ) (¯ µ = nominal returns). Definition (Rockafellar and Uryasev): Given reals ν and 0 ≤ θ ≤ 1 the value-at-risk of x ∗ is the real ρ ≥ 0 such that Prob(ν − µT x ∗ ≥ ρ) ≥ θ → The adversary wants to maximize VaR Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

10 / 56

Uncertainty set Given a vector x ∗ of assets, the adversary 1

2

Chooses a vector w ∈ R n (n = no. of assets) with 0 ≤ wj ≤ 1 for all j. Chooses a random variable 0 ≤ δ ≤ 1

→ Random return: µj = µ ¯ j (1 − δwj ) (¯ µ = nominal returns). Definition (Rockafellar and Uryasev): Given reals ν and 0 ≤ θ ≤ 1 the value-at-risk of x ∗ is the real ρ ≥ 0 such that Prob(ν − µT x ∗ ≥ ρ) ≥ θ → The adversary wants to maximize VaR Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

10 / 56

Uncertainty set Given a vector x ∗ of assets, the adversary 1

2

Chooses a vector w ∈ R n (n = no. of assets) with 0 ≤ wj ≤ 1 for all j. Chooses a random variable 0 ≤ δ ≤ 1

→ Random return: µj = µ ¯ j (1 − δwj ) (¯ µ = nominal returns). Definition: Given reals ν and 0 ≤ θ ≤ 1 the conditional value-at-risk of x ∗ equals E(ν − µT x ∗ | ν − µT x ∗ ≥ ρ)

where ρ = VaR

→ The adversary wants to maximize CVaR Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

11 / 56

Robust optimization problem (VaR case): Given 0 < , min V Subject to: λx T Qx − µT x ≤ v ∗ +  Ax ≥ b V ≥ VaRmax (x) Here,

. v ∗ = min λx T Qx − µT x

Subject to: Ax ≥ b Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

12 / 56

General methodology: Benders’ decomposition (= cutting-plane algorithm) Generic problem:

minx∈X maxd∈D f (x, d)

˜ of D (a “model”) → Maintain a finite subset D GAME 1

Investor: solve minx∈X maxd∈D˜ f (x, d), with solution x ∗

2

˜ Adversary: solve maxd∈D f (x ∗ , d), with solution d

3

˜ to D, ˜ and go to 1. Add d

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

13 / 56

General methodology: Benders’ decomposition (= cutting-plane algorithm) Generic problem:

minx∈X maxd∈D f (x, d)

˜ of D (a “model”) → Maintain a finite subset D GAME 1

Investor: solve minx∈X maxd∈D˜ f (x, d), with solution x ∗

2

˜ Adversary: solve maxd∈D f (x ∗ , d), with solution d

3

˜ to D, ˜ and go to 1. Add d

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

13 / 56

General methodology: Benders’ decomposition (= cutting-plane algorithm) Generic problem:

minx∈X maxd∈D f (x, d)

˜ of D (a “model”) → Maintain a finite subset D GAME 1

Investor: solve minx∈X maxd∈D˜ f (x, d), with solution x ∗

2

˜ Adversary: solve maxd∈D f (x ∗ , d), with solution d

3

˜ to D, ˜ and go to 1. Add d

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

13 / 56

General methodology: Benders’ decomposition (= cutting-plane algorithm) Generic problem:

minx∈X maxd∈D f (x, d)

˜ of D (a “model”) → Maintain a finite subset D GAME 1

Investor: solve minx∈X maxd∈D˜ f (x, d), with solution x ∗

2

˜ Adversary: solve maxd∈D f (x ∗ , d), with solution d

3

˜ to D, ˜ and go to 1. Add d

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

13 / 56

General methodology: Benders’ decomposition (= cutting-plane algorithm) Generic problem:

minx∈X maxd∈D f (x, d)

˜ of D (a “model”) → Maintain a finite subset D GAME 1

Investor: solve minx∈X maxd∈D˜ f (x, d), with solution x ∗

2

˜ Adversary: solve maxd∈D f (x ∗ , d), with solution d

3

˜ to D, ˜ and go to 1. Add d

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

13 / 56

Why this approach

Decoupling of investor and adversary yields considerably simpler, and smaller, problems Decoupling allows us to use more sophisticated uncertainty models If number of iterations is small, investor’s problem is a small “convex” problem

Most progress will be achieved in initial iterations – permits “soft” termination criteria Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

14 / 56

Why this approach

Decoupling of investor and adversary yields considerably simpler, and smaller, problems Decoupling allows us to use more sophisticated uncertainty models If number of iterations is small, investor’s problem is a small “convex” problem

Most progress will be achieved in initial iterations – permits “soft” termination criteria Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

14 / 56

Why this approach

Decoupling of investor and adversary yields considerably simpler, and smaller, problems Decoupling allows us to use more sophisticated uncertainty models If number of iterations is small, investor’s problem is a small “convex” problem

Most progress will be achieved in initial iterations – permits “soft” termination criteria Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

14 / 56

Why this approach

Decoupling of investor and adversary yields considerably simpler, and smaller, problems Decoupling allows us to use more sophisticated uncertainty models If number of iterations is small, investor’s problem is a small “convex” problem

Most progress will be achieved in initial iterations – permits “soft” termination criteria Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

14 / 56

Why this approach

Decoupling of investor and adversary yields considerably simpler, and smaller, problems Decoupling allows us to use more sophisticated uncertainty models If number of iterations is small, investor’s problem is a small “convex” problem

Most progress will be achieved in initial iterations – permits “soft” termination criteria Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

14 / 56

The game, applied to the robust optimization problem Histogram version Robust problem:  minx λx T Qx − minµ∈M µT x Subject to: Ax ≥ b

˜ Investor’s problem: M is replaced by a finite subset M

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

15 / 56

The game, applied to the robust optimization problem Histogram version Robust problem:  minx λx T Qx − minµ∈M µT x Subject to: Ax ≥ b

˜ Investor’s problem: M is replaced by a finite subset M

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

15 / 56

Investor’s problem A convex quadratic program At iteration m, solve

min λx T Qx − r Subject to: Ax ≥ b r ≤ µT(i) x, i = 1, . . . , m Here, µ(1) , . . . , µ(m) are given return vectors

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

16 / 56

The game, applied to the robust optimization problem Histogram version Robust problem:  minx λx T Qx − minµ∈M µT x Subject to: Ax ≥ b

Adversary’s problem: Given asset vector x, solve minµ∈M µT x

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

17 / 56

Back to the histogram model Parameters: 0 ≤ γ1 ≤ γ2 ≤ . . . ≤ γK ≤ 1, integers 0 ≤ ni ≤ Ni , 1 ≤ i ≤ K for each asset j: µ ¯ j = expected return between ni and Ni assets j satisfy: (1 − γi )µ ¯ j ≤ µj ≤ (1 − γi−1 )µ ¯j P

j

µj ≥ Γ

P

µ ¯ j ; Γ > 0 a parameter

j

For 1 ≤ h ≤ H, a set (“tier”) Th of assets, and a parameter Γh > 0

for each h,

P

j∈Th

µj ≥ Γh

P

j∈Sh

µ ¯j

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

18 / 56

Adversarial problem: A mixed-integer program x ∗ = given asset weights P

min

j

xj∗ µj

Subject to: µ ¯ j (1 − P

i

P

i

γi yij ) ≤ µj ≤ µ ¯ j (1 −

yij ≤ 1, ∀ j

ni ≤

P

P

µj ≥ Γh

j∈Th

j

P

i

γi−1 yij ) ∀i ≥ 1

(each asset in at most one segment)

yij ≤ Ni , 1 ≤ i ≤ K P

j∈Th

(segment cardinalities)

µ ¯j , 1 ≤ h ≤ H

(tier ineqs.)

µj free, yij = 0 or 1, all i, j Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

19 / 56

Why the adversarial problem is “easy”

( K = no. of segments, H = no. of tiers)

Theorem. For every fixed K and H, and for every  > 0, there is an algorithm that finds a solution to the adversarial problem with optimality relative error ≤ , in time polynomial in −1 and n (= no. of assets).

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

20 / 56

The simplest case max

P

j

xj∗ δj

Subject to: P

j

δj ≤ Γ

0 ≤ δj ≤ uj yj , yj = 0 or 1, all j P

j

yj ≤ N

· · · a cardinality constrained knapsack problem B. (1995), DeFarias and Nemhauser (2004)

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

21 / 56

Summary

At each iteration of the game, we solve:

1

A convex quadratic program – a new constraint added every iteration

2

A mixed-integer program, always of the same size.

Questions: How fast (slow?) is each iteration? How many iterations? Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

22 / 56

Summary

At each iteration of the game, we solve:

1

A convex quadratic program – a new constraint added every iteration

2

A mixed-integer program, always of the same size.

Questions: How fast (slow?) is each iteration? How many iterations? Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

22 / 56

Summary

At each iteration of the game, we solve:

1

A convex quadratic program – a new constraint added every iteration

2

A mixed-integer program, always of the same size.

Questions: How fast (slow?) is each iteration? How many iterations? Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

22 / 56

Summary

At each iteration of the game, we solve:

1

A convex quadratic program – a new constraint added every iteration

2

A mixed-integer program, always of the same size.

Questions: How fast (slow?) is each iteration? How many iterations? Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

22 / 56

Summary

At each iteration of the game, we solve:

1

A convex quadratic program – a new constraint added every iteration

2

A mixed-integer program, always of the same size.

Questions: How fast (slow?) is each iteration? How many iterations? Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

22 / 56

Example: 2464 assets, 152-factor model. CPU time: 300 seconds No Strengthening – straight Benders 10 segments (a: “heavy tail”) 6 tiers: the top five deciles lose at most 10% each, total loss ≤ 5%

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

23 / 56

Same run 2464 assets, 152 factors; 10 segments, 6 tiers

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

24 / 56

Comparison with mean-variance portfolio

10% Tier Weight Name Count Exp. return =

Mean-Variance Portfolio 1 2 3 4 8.301e-03 9.894e-01 8.616e-03 2.981e-03 76 6 3 3 1.00 Worst-case return = 0.234

10% Tier Weight Name Count Exp. return =

Robust Portfolio 1 2 3 4 9.997e-01 4.625e-05 1.908e-05 5.975e-04 208 12 6 3 0.974 Worst-case return = 0.847

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

25 / 56

Comparison with mean-variance portfolio

w r

w r

top 25 0.773 0.368

CUMULATIVE POSITIONS Mean-Variance Portfolio 50 75 100 125 150 0.948 1.000 1.000 1.000 1.000 0.649 0.806 0.853 0.901 0.929

175 1.000 0.950

200 1.000 0.974

top 25 0.130 0.122

Robust Portfolio 75 100 125 0.389 0.519 0.648 0.331 0.441 0.525

175 0.907 0.694

200 0.991 0.804

50 0.259 0.228

150 0.778 0.601

“w” – sorted by weight, “r” sorted by return Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

26 / 56

Comparison with mean-variance portfolio

W N

W N

1 7.406e-01 37

Mean-Variance Portfolio 2 3 4 1.602e-01 6.084e-02 2.781e-02 18 12 9

5 5.313e-03 2

1 2.744e-01 58

Robust Portfolio 2 3 4 2.456e-01 2.165e-01 2.601e-01 51 46 53

5 2.730e-05 5

Portfolio structure per 2.5% return quantile, “W” = weight, “N” = count Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

27 / 56

A popular heuristic

→ Impose the constraint xj ≤ w for all assets j

Here, w is a “small” value

Nomenclature: heuristic portfolio U-w

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

28 / 56

A popular heuristic

→ Impose the constraint xj ≤ w for all assets j

Here, w is a “small” value

Nomenclature: heuristic portfolio U-w

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

28 / 56

1 2 W 4.000e-01 2.896e-01 N 52 38 Exp. return = 0.983

U − 0.01 3 4 5 1.800e-01 9.077e-02 1.925e-02 25 17 5 Worst-case return = 0.712

U − 0.001 1 2 3 4 5 W 6.200e-02 5.900e-02 6.100e-02 5.967e-02 5.956e-02 N 62 59 61 60 60 Exp. return = 0.796 Worst-case return = 0.687 Heuristic non-robust portfolio characteristics, per 2.5% return quantile, “W” = weight, “N” = count Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

29 / 56

Updated heuristic

→ Impose the constraint xj ≤ w for all assets j in return decile 1

Nomenclature: heuristic portfolio T-w

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

30 / 56

Updated heuristic

→ Impose the constraint xj ≤ w for all assets j in return decile 1

Nomenclature: heuristic portfolio T-w

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

30 / 56

T 1 − 0.006 1 2 3 4 5 W 3.063e-01 2.171e-01 1.667e-01 1.597e-01 1.114e-01 N 57 43 34 34 11 Exp. return = 0.969 Worst-case return = 0.699 T 1 − 0.005 1 2 3 4 5 W 2.618e-01 1.999e-01 1.641e-01 1.530e-01 1.808e-01 N 57 44 39 37 15 Exp. return = 0.963 Worst-case return = 0.661 Robust Portfolio, exp. return = 0.974, worst case = 0.847 1 2 3 4 5 W 2.744e-01 2.456e-01 2.165e-01 2.601e-01 2.730e-05 N 58 51 46 53 5 Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

31 / 56

Summary of average problems with 3-4 segments, 2-3 tiers

1 2 3 4 5 6 7 8 9

columns

rows

iterations

500 500 703 499 499 1338 2019 2443 2464

20 20 108 140 20 81 140 153 153

47 3 1 3 19 7 8 2 111

time (sec.) 1.85 0.09 0.29 3.12 0.42 0.45 41.53 12.32 100.81

imp. time

adv. time

1.34 0.01 0.13 2.65 0.21 0.17 39.6 9.91 60.93

0.46 0.03 0.04 0.05 0.17 0.08 0.36 0.07 36.78

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

32 / 56

A C F G∗ I

time 327.04 29.32 74.06 681.12 124.82

bigQP 2.52 3.01 13.57 – 93.38

bigMIP 211.72 9.35 15.96 – 22.58

iters 135 27 27 19 1

impT 12.27 1.02 2.47 64.7 4.17

advT 100.24 15.76 41.42 615.54 2.46

01vars 5000 4990 13380 20190 24640

Table: Heavy-tailed instances, 10 segments, 6 tiers, tol. = 1.0e−03

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

33 / 56

500 × 20

500 × 20

499 × 20

499 b × 140

703 ∗ × 108

1338 × 81

2443 × 153

5.0e −2

214.53

14.81

144.86

122.53

11.77

274.64

140.29

1.0e −2

223.21

15.49

144.86

122.53

14.66

356.98

140.29

5.0e −3

254.73

16.03

162.41

126.63

34.16

363.84

140.29

1.0e −3

300.88

35.23

183.12

157.49

64.61

469.75

140.29

5.0e −4

361.20

37.92

216.52

167.40

73.87

598.94

140.29

error

Table: Convergence time on heavy-tailed instances, 10 segments, 6 tiers

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

34 / 56

What is the impact of the uncertainty model All runs on the same data set with 1338 columns and 81 rows 1 segment: (200, 0.5) robust random return = 4.57,

157 assets

2 segments: (200, 0.25), (100, 0.5) robust random return = 4.57, 186 assets 2 segments: (200, 0.2), (100, 0.6) robust random return = 3.25, 213 assets 2 segments: (200, 0.1), (100, 0.8) robust random return = 1.50, 256 assets 1 segment: (100, 1.0) robust random return = 1.24,

281 assets

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

35 / 56

Uncertainty set – ambiguous chance-constrained model Repetition Given a vector x ∗ of assets, the adversary 1

2

Chooses a vector w ∈ R n (n = no. of assets) with 0 ≤ wj ≤ 1 for all j. Chooses a random variable 0 ≤ δ ≤ 1

→ Random return: µj = µ ¯ j (1 − δwj ) (¯ µ = nominal returns). Definition (Rockafellar and Uryasev): Given reals ν and 0 ≤ θ ≤ 1 the value-at-risk of x ∗ is the real ρ ≥ 0 such that Prob(ν − µT x ∗ ≥ ρ) ≥ θ → The adversary wants to maximize VaR Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

36 / 56

Uncertainty set – ambiguous chance-constrained model Repetition Given a vector x ∗ of assets, the adversary 1

2

Chooses a vector w ∈ R n (n = no. of assets) with 0 ≤ wj ≤ 1 for all j. Chooses a random variable 0 ≤ δ ≤ 1

→ Random return: µj = µ ¯ j (1 − δwj ) (¯ µ = nominal returns). Definition (Rockafellar and Uryasev): Given reals ν and 0 ≤ θ ≤ 1 the value-at-risk of x ∗ is the real ρ ≥ 0 such that Prob(ν − µT x ∗ ≥ ρ) ≥ θ → The adversary wants to maximize VaR Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

36 / 56

Uncertainty set – ambiguous chance-constrained model Repetition Given a vector x ∗ of assets, the adversary 1

2

Chooses a vector w ∈ R n (n = no. of assets) with 0 ≤ wj ≤ 1 for all j. Chooses a random variable 0 ≤ δ ≤ 1

→ Random return: µj = µ ¯ j (1 − δwj ) (¯ µ = nominal returns). Definition (Rockafellar and Uryasev): Given reals ν and 0 ≤ θ ≤ 1 the value-at-risk of x ∗ is the real ρ ≥ 0 such that Prob(ν − µT x ∗ ≥ ρ) ≥ θ → The adversary wants to maximize VaR Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

36 / 56

Uncertainty set – ambiguous chance-constrained model Repetition Given a vector x ∗ of assets, the adversary 1

2

Chooses a vector w ∈ R n (n = no. of assets) with 0 ≤ wj ≤ 1 for all j. Chooses a random variable 0 ≤ δ ≤ 1

→ Random return: µj = µ ¯ j (1 − δwj ) (¯ µ = nominal returns). Definition (Rockafellar and Uryasev): Given reals ν and 0 ≤ θ ≤ 1 the value-at-risk of x ∗ is the real ρ ≥ 0 such that Prob(ν − µT x ∗ ≥ ρ) ≥ θ → The adversary wants to maximize VaR Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

36 / 56

Uncertainty set – ambiguous chance-constrained model Repetition Given a vector x ∗ of assets, the adversary 1

2

Chooses a vector w ∈ R n (n = no. of assets) with 0 ≤ wj ≤ 1 for all j. Chooses a random variable 0 ≤ δ ≤ 1

→ Random return: µj = µ ¯ j (1 − δwj ) (¯ µ = nominal returns). Definition (Rockafellar and Uryasev): Given reals ν and 0 ≤ θ ≤ 1 the value-at-risk of x ∗ is the real ρ ≥ 0 such that Prob(ν − µT x ∗ ≥ ρ) ≥ θ → The adversary wants to maximize VaR Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

36 / 56

Uncertainty set – ambiguous chance-constrained model Repetition Given a vector x ∗ of assets, the adversary 1

2

Chooses a vector w ∈ R n (n = no. of assets) with 0 ≤ wj ≤ 1 for all j. Chooses a random variable 0 ≤ δ ≤ 1

→ Random return: µj = µ ¯ j (1 − δwj ) (¯ µ = nominal returns). Definition (Rockafellar and Uryasev): Given reals ν and 0 ≤ θ ≤ 1 the value-at-risk of x ∗ is the real ρ ≥ 0 such that Prob(ν − µT x ∗ ≥ ρ) ≥ θ → The adversary wants to maximize VaR Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

36 / 56

Fleshing out the uncertainty model: → Random returnj = µ ¯ j (1 − δwj ) where 0 ≤ wj ≤ 1 ∀ j, and 0 ≤ δ ≤ 1 is a random variable. A discrete distribution: We are given fixed values 0 = δ0 ≤ δ2 ≤ ... ≤ δK = 1 example: δi = Ki Adversary chooses πi = Prob(δ = δi ), 0 ≤ i ≤ K The πi are constrained: we have fixed bounds, πil ≤ πi ≤ πiu (and possibly other constraints) Tier constraints: for sets (“tiers”) Th of assets, 1 ≤ h ≤ H, we require: P E(δ j∈Th wj ) ≤ Γh (given) P P or, ( i δi πi ) j∈Th wj ≤ Γh Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

37 / 56

Fleshing out the uncertainty model: → Random returnj = µ ¯ j (1 − δwj ) where 0 ≤ wj ≤ 1 ∀ j, and 0 ≤ δ ≤ 1 is a random variable. A discrete distribution: We are given fixed values 0 = δ0 ≤ δ2 ≤ ... ≤ δK = 1 example: δi = Ki Adversary chooses πi = Prob(δ = δi ), 0 ≤ i ≤ K The πi are constrained: we have fixed bounds, πil ≤ πi ≤ πiu (and possibly other constraints) Tier constraints: for sets (“tiers”) Th of assets, 1 ≤ h ≤ H, we require: P E(δ j∈Th wj ) ≤ Γh (given) P P or, ( i δi πi ) j∈Th wj ≤ Γh Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

37 / 56

Fleshing out the uncertainty model: → Random returnj = µ ¯ j (1 − δwj ) where 0 ≤ wj ≤ 1 ∀ j, and 0 ≤ δ ≤ 1 is a random variable. A discrete distribution: We are given fixed values 0 = δ0 ≤ δ2 ≤ ... ≤ δK = 1 example: δi = Ki Adversary chooses πi = Prob(δ = δi ), 0 ≤ i ≤ K The πi are constrained: we have fixed bounds, πil ≤ πi ≤ πiu (and possibly other constraints) Tier constraints: for sets (“tiers”) Th of assets, 1 ≤ h ≤ H, we require: P E(δ j∈Th wj ) ≤ Γh (given) P P or, ( i δi πi ) j∈Th wj ≤ Γh Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

37 / 56

Fleshing out the uncertainty model: → Random returnj = µ ¯ j (1 − δwj ) where 0 ≤ wj ≤ 1 ∀ j, and 0 ≤ δ ≤ 1 is a random variable. A discrete distribution: We are given fixed values 0 = δ0 ≤ δ2 ≤ ... ≤ δK = 1 example: δi = Ki Adversary chooses πi = Prob(δ = δi ), 0 ≤ i ≤ K The πi are constrained: we have fixed bounds, πil ≤ πi ≤ πiu (and possibly other constraints) Tier constraints: for sets (“tiers”) Th of assets, 1 ≤ h ≤ H, we require: P E(δ j∈Th wj ) ≤ Γh (given) P P or, ( i δi πi ) j∈Th wj ≤ Γh Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

37 / 56

Fleshing out the uncertainty model: → Random returnj = µ ¯ j (1 − δwj ) where 0 ≤ wj ≤ 1 ∀ j, and 0 ≤ δ ≤ 1 is a random variable. A discrete distribution: We are given fixed values 0 = δ0 ≤ δ2 ≤ ... ≤ δK = 1 example: δi = Ki Adversary chooses πi = Prob(δ = δi ), 0 ≤ i ≤ K The πi are constrained: we have fixed bounds, πil ≤ πi ≤ πiu (and possibly other constraints) Tier constraints: for sets (“tiers”) Th of assets, 1 ≤ h ≤ H, we require: P E(δ j∈Th wj ) ≤ Γh (given) P P or, ( i δi πi ) j∈Th wj ≤ Γh Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

37 / 56

Fleshing out the uncertainty model: → Random returnj = µ ¯ j (1 − δwj ) where 0 ≤ wj ≤ 1 ∀ j, and 0 ≤ δ ≤ 1 is a random variable. A discrete distribution: We are given fixed values 0 = δ0 ≤ δ2 ≤ ... ≤ δK = 1 example: δi = Ki Adversary chooses πi = Prob(δ = δi ), 0 ≤ i ≤ K The πi are constrained: we have fixed bounds, πil ≤ πi ≤ πiu (and possibly other constraints) Tier constraints: for sets (“tiers”) Th of assets, 1 ≤ h ≤ H, we require: P E(δ j∈Th wj ) ≤ Γh (given) P P or, ( i δi πi ) j∈Th wj ≤ Γh Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

37 / 56

Fleshing out the uncertainty model: → Random returnj = µ ¯ j (1 − δwj ) where 0 ≤ wj ≤ 1 ∀ j, and 0 ≤ δ ≤ 1 is a random variable. A discrete distribution: We are given fixed values 0 = δ0 ≤ δ2 ≤ ... ≤ δK = 1 example: δi = Ki Adversary chooses πi = Prob(δ = δi ), 0 ≤ i ≤ K The πi are constrained: we have fixed bounds, πil ≤ πi ≤ πiu (and possibly other constraints) Tier constraints: for sets (“tiers”) Th of assets, 1 ≤ h ≤ H, we require: P E(δ j∈Th wj ) ≤ Γh (given) P P or, ( i δi πi ) j∈Th wj ≤ Γh Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

37 / 56

Robust optimization problem (VaR case): Given 0 < , min V Subject to: λx T Qx − µT x ≤ v ∗ +  Ax ≥ b V ≥ VaRmax (x) Here,

. v ∗ = min λx T Qx − µT x

Subject to: Ax ≥ b Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

38 / 56

Typical convergence behavior – simple Benders

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

39 / 56

Heavy-tailed instances, θ = .05 Heavy tail, proportional error (100 points): 0.25 Upper Lower

0.2

0.15

0.1

0.05

0 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

0.9

1

27th September 2007

40 / 56

Heavy-tailed instances, θ = .05 Heavy tail, proportional error (100 points): 1.4 Upper Lower 1.2

1

0.8

0.6

0.4

0.2

0 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

0.9

1

27th September 2007

40 / 56

Heavy-tailed instances, θ = .05, K = 100 VaR time iters impt advt adj τ

CVaR time iters impt advt gap apperr

A 1.98 2 0.25 1.26 2.8e−04

A 7.10 2 0.16 6.72 9.8e−04 2.3e−04

D 5.02 2 2.25 1.14 2.4e−04

D 14.11 2 1.72 10.67 2.2e−05 2.2e−05

E 2.47 2 0.54 1.32 3.0e−04

E 6.23 2 1.18 4.74 7.3e−05 2.4e−04

F 2.03 2 1.07 0.24 2.5e−04

F 11.45 2 0.66 10.33 5.1e−05 1.6e−05

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

G 26.51 2 14.09 2.17 4.7e−05

G 33.13 2 9.56 12.2 3.2e−05 1.0e−04

I 38.32 2 19.90 1.47 2.1e−04

I 88.43 3 52.13 23.85 1.3e−04 2.2e−04

27th September 2007

41 / 56

Impact of tail probability “confidence level” = 1 − θ 4.5

4

3.5

3

2.5

2

1.5

1 0.65

0.7

0.75

0.8

0.85

0.9

0.95

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

1

27th September 2007

42 / 56

Impact of suboptimality target Fix θ = 0.95 but vary suboptimality criterion

3.65 3.6 3.55 3.5 3.45 3.4 3.35 3.3 3.25 3.2 3.15 0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

0.1

0.11

27th September 2007

43 / 56

Experiment: sensitivity of model to parameters

Adversary chooses πi = P(δ = δi ),

πil ≤ πi ≤ πiu

Experiment: choose ∆ ≥ 0, and solve robust problems for πil ← max{πil − ∆, 0}, πiu ← πiu + ∆

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

44 / 56

Experiment: sensitivity of model to parameters

Adversary chooses πi = P(δ = δi ),

πil ≤ πi ≤ πiu

Experiment: choose ∆ ≥ 0, and solve robust problems for πil ← max{πil − ∆, 0}, πiu ← πiu + ∆

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

44 / 56

VaR and CVaR as a function of data errors:

2.4 CVaR AVaR 2.2

2

1.8

1.6

1.4

1.2

1

0.8

0.6 0

0.001

0.002

0.003

0.004

0.005

0.006

0.007

0.008

( N = 104 for VaR case) Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

45 / 56

Part II: Methodology

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

46 / 56

Adversarial problem for histogram model x ∗ = given asset weights P

min

j

xj∗ µj

Subject to: µ ¯ j (1 − P

i

P

i

γi yij ) ≤ µj ≤ µ ¯ j (1 −

yij ≤ 1, ∀ j

ni ≤

P

P

µj ≥ Γh

j∈Th

j

P

i

γi−1 yij ) ∀i ≥ 1

(each asset in at most one segment)

yij ≤ Ni , 1 ≤ i ≤ K P

j∈Th

(segment cardinalities)

µ ¯j , 1 ≤ h ≤ H

(tier ineqs.)

µj free, yij = 0 or 1, all i, j Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

47 / 56

The LP relaxation x ∗ = given asset weights should (?) be tight P

min

j

xj∗ µj

Subject to: µ ¯ j (1 − P

i

P

i

γi−1 yij ) ≤ µj ≤ µ ¯ j (1 −

yij ≤ 1, ∀ j

ni ≤

P

P

µj ≥ Γh

j∈Th

j

P

i

γi yij ) ∀i ≥ 1

(each asset in at most one segment)

yij ≤ Ni , 1 ≤ i ≤ K P

j∈Th

(segment cardinalities)

µ ¯j , 1 ≤ h ≤ H

(tier ineqs.)

µj free, 0 ≤ yij ≤ 1, all i, j Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

48 / 56

Robust problem: . V = min λx T Qx − r Subject to: Ax ≥ b r ≤ µT x, ∀ µ achievable by adversary Robust problem for relaxed adversary: . V = min λx T Qx − r Subject to: Ax ≥ b r ≤ µT x, ∀ µ achievable by relaxed adversary V ∗ ≥ V , perhaps V ∗ ≈ V ,

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

49 / 56

Robust problem: . V = min λx T Qx − r Subject to: Ax ≥ b r ≤ µT x, ∀ µ achievable by adversary Robust problem for relaxed adversary: . V = min λx T Qx − r Subject to: Ax ≥ b r ≤ µT x, ∀ µ achievable by relaxed adversary V ∗ ≥ V , perhaps V ∗ ≈ V ,

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

49 / 56

Robust problem: . V = min λx T Qx − r Subject to: Ax ≥ b r ≤ µT x, ∀ µ achievable by adversary Robust problem for relaxed adversary: . V = min λx T Qx − r Subject to: Ax ≥ b r ≤ µT x, ∀ µ achievable by relaxed adversary V ∗ ≥ V , perhaps V ∗ ≈ V ,

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

49 / 56

Robust problem for relaxed adversary: . V = min λx T Qx − r Subject to: Ax ≥ b r ≤ µT x, ∀ µ achievable by relaxed adversary or,

. V = min λx T Qx − r Subject to: Ax ≥ b r ≤ minimum return(x)

but,

minimum return(x) = min

Subject to:

P

j

xj∗ µj

M1 µ + M2 y ≥ Ψ

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

50 / 56

Robust problem for relaxed adversary: . V = min λx T Qx − r Subject to: Ax ≥ b r ≤ µT x, ∀ µ achievable by relaxed adversary or,

. V = min λx T Qx − r Subject to: Ax ≥ b r ≤ minimum return(x)

but,

minimum return(x) = min

Subject to:

P

j

xj∗ µj

M1 µ + M2 y ≥ Ψ

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

50 / 56

Robust problem for relaxed adversary: . V = min λx T Qx − r Subject to: Ax ≥ b r ≤ µT x, ∀ µ achievable by relaxed adversary or,

. V = min λx T Qx − r Subject to: Ax ≥ b r ≤ minimum return(x)

but,

minimum return(x) = min

Subject to:

P

j

xj∗ µj

M1 µ + M2 y ≥ Ψ

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

50 / 56

Robust problem for relaxed adversary: . V = min λx T Qx − r Subject to: Ax ≥ b r ≤ µT x, ∀ µ achievable by relaxed adversary or,

. V = min λx T Qx − r Subject to: Ax ≥ b r ≤ minimum return(x)

duality: minimum return(x) = max ΨT α Subject to:

M1T α = x, M2T α = 0, α ≥ 0

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

51 / 56

Robust problem for relaxed adversary: . V = min λx T Qx − r Subject to: Ax ≥ b r ≤ minimum return(x)

Robust problem: min λx T Qx − r Subject to: Ax ≥ b r − ΨT α ≤ 0 M1T α − x = 0, M2T α = 0, α ≥ 0 Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

52 / 56

Robust problem for relaxed adversary: . V = min λx T Qx − r Subject to: Ax ≥ b r ≤ minimum return(x)

Robust problem: min λx T Qx − r Subject to: Ax ≥ b r − ΨT α ≤ 0 M1T α − x = 0, M2T α = 0, α ≥ 0 Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

52 / 56

Robust problem for relaxed adversary: . V = min λx T Qx − r Subject to: Ax ≥ b r ≤ minimum return(x)

Robust problem: min λx T Qx − r Subject to: Ax ≥ b r − ΨT α ≤ 0 M1T α − x = 0, M2T α = 0, α ≥ 0 Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

52 / 56

Robust problem for relaxed adversary: . V ∗ = min λx T Qx − r Subject to: Ax ≥ b r − ΨT α ≤ 0 (∗∗) M1T α − x = 0 M2T α = 0, α ≥ 0 Let µ ˆ = optimal duals for (**) V ∗ = min λx T Qx − r Subject to: Ax ≥ b r − µ ˆT x ≤ 0 ( r − µT x ≤ 0, ∀ µ available to strict adversary) Problem: Find µ available to strict adversary, and with µ ≈ µ ˆ

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

53 / 56

Robust problem for relaxed adversary: . V ∗ = min λx T Qx − r Subject to: Ax ≥ b r − ΨT α ≤ 0 (∗∗) M1T α − x = 0 M2T α = 0, α ≥ 0 Let µ ˆ = optimal duals for (**) V ∗ = min λx T Qx − r Subject to: Ax ≥ b r − µ ˆT x ≤ 0 ( r − µT x ≤ 0, ∀ µ available to strict adversary) Problem: Find µ available to strict adversary, and with µ ≈ µ ˆ

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

53 / 56

Robust problem for relaxed adversary: . V ∗ = min λx T Qx − r Subject to: Ax ≥ b r − ΨT α ≤ 0 (∗∗) M1T α − x = 0 M2T α = 0, α ≥ 0 Let µ ˆ = optimal duals for (**) V ∗ = min λx T Qx − r Subject to: Ax ≥ b r − µ ˆT x ≤ 0 ( r − µT x ≤ 0, ∀ µ available to strict adversary) Problem: Find µ available to strict adversary, and with µ ≈ µ ˆ

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

53 / 56

Robust problem for relaxed adversary: . V ∗ = min λx T Qx − r Subject to: Ax ≥ b r − ΨT α ≤ 0 (∗∗) M1T α − x = 0 M2T α = 0, α ≥ 0 Let µ ˆ = optimal duals for (**) V ∗ = min λx T Qx − r Subject to: Ax ≥ b r − µ ˆT x ≤ 0 ( r − µT x ≤ 0, ∀ µ available to strict adversary) Problem: Find µ available to strict adversary, and with µ ≈ µ ˆ

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

53 / 56

Robust problem for relaxed adversary: . V ∗ = min λx T Qx − r Subject to: Ax ≥ b r − ΨT α ≤ 0 (∗∗) M1T α − x = 0 M2T α = 0, α ≥ 0 Let µ ˆ = optimal duals for (**) V ∗ = min λx T Qx − r Subject to: Ax ≥ b r − µ ˆT x ≤ 0 ( r − µT x ≤ 0, ∀ µ available to strict adversary) Problem: Find µ available to strict adversary, and with µ ≈ µ ˆ

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

53 / 56

Benders’ algorithm with strengthening Step 1. Solve relaxed robust problem; answer = µ ˆ Step 2. Solve MIP to obtain vector µ ˘ which is legal for histogram model, and with µ ˘ ≈ µ ˆ Step 3. Run Benders beginning with the cut r − µ ˘T x ≤ 0

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

54 / 56

Alternate algorithm? Step 1. Solve relaxed robust problem, let µ ˆ be the min-max return vector

ˆ from the convex hull of Step 2. Find a cut αT µ ≤ α0 , that separates µ vectors available to the strict adversary

Step 3. Add αT µ ˆ ≤ α0 to the definition of the adversarial problem, and go to 1.

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

55 / 56

Adversarial problem – a nonlinear MIP Recall: random returnj µj = µ ¯ j (1 − δwj ) where δ = δi (given) with probability πi (chosen by adversary), 0 ≤ δ0 ≤ δ1 ≤ . . . ≤ δK = 1 and 0 ≤ w minπ,w ,V min1≤i≤k Vi Subject to l u 0 P≤ wj ≤ 1, all j, πi ≤ πi ≤ πi , all i, i πi = 1, P Vi = ¯ j (1 − δi wj )xj∗ , if πi + πi+1 + . . . + πK ≥ θ j µ Vi = M (large), otherwise P P ( i δi πi ) j∈Th wj ≤ Γh , for each tier h

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

56 / 56

Adversarial problem – a nonlinear MIP Recall: random returnj µj = µ ¯ j (1 − δwj ) where δ = δi (given) with probability πi (chosen by adversary), 0 ≤ δ0 ≤ δ1 ≤ . . . ≤ δK = 1 and 0 ≤ w minπ,w ,V min1≤i≤k Vi Subject to l u 0 P≤ wj ≤ 1, all j, πi ≤ πi ≤ πi , all i, i πi = 1, P Vi = ¯ j (1 − δi wj )xj∗ , if πi + πi+1 + . . . + πK ≥ θ j µ Vi = M (large), otherwise P P ( i δi πi ) j∈Th wj ≤ Γh , for each tier h

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

56 / 56

Adversarial problem – a nonlinear MIP Recall: random returnj µj = µ ¯ j (1 − δwj ) where δ = δi (given) with probability πi (chosen by adversary), 0 ≤ δ0 ≤ δ1 ≤ . . . ≤ δK = 1 and 0 ≤ w minπ,w ,V min1≤i≤k Vi Subject to l u 0 P≤ wj ≤ 1, all j, πi ≤ πi ≤ πi , all i, i πi = 1, P Vi = ¯ j (1 − δi wj )xj∗ , if πi + πi+1 + . . . + πK ≥ θ j µ Vi = M (large), otherwise P P ( i δi πi ) j∈Th wj ≤ Γh , for each tier h

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

56 / 56

Adversarial problem – a nonlinear MIP Recall: random returnj µj = µ ¯ j (1 − δwj ) where δ = δi (given) with probability πi (chosen by adversary), 0 ≤ δ0 ≤ δ1 ≤ . . . ≤ δK = 1 and 0 ≤ w minπ,w ,V min1≤i≤k Vi Subject to l u 0 P≤ wj ≤ 1, all j, πi ≤ πi ≤ πi , all i, i πi = 1, P Vi = ¯ j (1 − δi wj )xj∗ , if πi + πi+1 + . . . + πK ≥ θ j µ Vi = M (large), otherwise P P ( i δi πi ) j∈Th wj ≤ Γh , for each tier h

Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

56 / 56

Approximation P P ( i δi πi ) j∈Th wj ≤ Γh ,

for each tier h

(∗)

Let N > 0 be an integer. For 1 ≤ k ≤ N, write k P j∈Th wj ≤ Γh + M (1 − zhk ), where N P zhk = 1 if k −1 < i δi πi ≤ Nk N zhk = 0 otherwise P k zhk = 1 and M is large Lemma. Under reasonable conditions, replacing (∗) with this system creates an error of order O( N1 ) Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

57 / 56

Approximation P P ( i δi πi ) j∈Th wj ≤ Γh ,

for each tier h

(∗)

Let N > 0 be an integer. For 1 ≤ k ≤ N, write k P j∈Th wj ≤ Γh + M (1 − zhk ), where N P zhk = 1 if k −1 < i δi πi ≤ Nk N zhk = 0 otherwise P k zhk = 1 and M is large Lemma. Under reasonable conditions, replacing (∗) with this system creates an error of order O( N1 ) Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

57 / 56

Approximation P P ( i δi πi ) j∈Th wj ≤ Γh ,

for each tier h

(∗)

Let N > 0 be an integer. For 1 ≤ k ≤ N, write k P j∈Th wj ≤ Γh + M (1 − zhk ), where N P zhk = 1 if k −1 < i δi πi ≤ Nk N zhk = 0 otherwise P k zhk = 1 and M is large Lemma. Under reasonable conditions, replacing (∗) with this system creates an error of order O( N1 ) Daniel Bienstock ( Columbia University, New Experiments York) in Robust Portfolio Optimization

27th September 2007

57 / 56