Joint 48th IEEE Conference on Decision and Control and 28th Chinese Control Conference Shanghai, P.R. China, December 16-18, 2009
FrC15.3
Singular Control and Impulse Control with Application to Mutual Insurance Optimization Alain Bensoussan, John Liu and Jiguang Yuan Abstract— We consider a mutual insurance system whose reserve is determined by a Brownian motion. The controller tries to minimize the total cost by increase or reduce the reserve instantly. Both cases of zero and positive fixed cost are investigated. By applying the theory of stochastic control with QVI approach, we make the connection between singular control and impulse control. The procedures to solve these systems are also presented in this paper.
I. INTRODUCTION This article has several motivations. The first one is to extend the paper of S. Kumar and K. Muthuraman, published in Operations Research in 2004 [4]. This paper deals with singular stochastic control problems, and describes a new numerical method. The case of one-dimensional problem plays an important role, first because it is simpler and second because it provides a good background for the general case. We provide a full treatment of this case, completing the above paper. We solve analytically the Bellman equation, and we prove the existence of an optimal singular control. With this extended background we intend to consider ndimensional problems in future works. The second motivation is to connect singular control problems with impulse control problems. This has several interests. Singular control appears as a limit case of impulse control. It may not be a surprise, but it is not so much substantiated in the literature. The Bellman equation becomes what is known in the literature of Impulse Control a QuasiVariational Inequality, Q.V.I. We solve completely the Q.V.I analytically in onedimension and obtain an optimal impulse control. When the fixed cost of impulses goes to 0, we prove the convergence of the value function of the impulse control problem towards the value function of the singular control problem. We also prove that the sequence of impulse controls is a minimizing sequence for the singular control problem. In this way we provide a control which is implementable, whereas the optimal singular control (although it exists) is not implementable. The third objective concerns the Q.V.I itself and its application to problems of mutual insurance optimization. In fact this problem had been considered in the control of cash This research is support in part by GRF grant PolyU5230/06E A. Bensoussan is with International Center for Decision and Risk Analysis, School of Management, University of Texas - Dallas and Faculty of Business, The Hong Kong Polytechnic University
management, see Constantinides-Richard [3] or intervention of central bank in exchange rate control, see CadenillasZapatero [2]. They follow the general methodology of Impulse Control introduced in the context of Inventory Control, see Bensoussan-Lions [1]. In the case of Inventory Control, Impulse Control theory and Q.V.I. methodology justify completely the well known s, S policy. One replenishes the inventory when the level is below s and the replenishment brings the level to S. In the case of cash management, or in mutual insurance, two thresholds are necessary and the s, S policy becomes an a, A, B, b policy. It is obviously more complex, but we show here that the situation is much less complex than the one presented in the papers above, in which very sophisticated proofs are provided. This level of sophistication explains probably why this two threshold problem has been little considered after the initial works mentioned above. This is in contrast with the literature on s, S policy which remains abundant. The reason why our approach is less complex lies in the fact that we decompose the problem and deal only with one threshold problem at a time, whereas the initial approach has been to consider the system globally. Our method allows identifying many interesting properties, which simplify considerably the analytic as well as the numerical treatment. In our case we have to solve at most a system of 3 nonlinear equations, instead of 6 in the work of Cadenillas-Zapatero [2]. The fourth motivation and may be the main one concerns the application to mutual insurance optimization. This is an important problem, which deals with the general case of holding adequate reserves, an important question for financial institutions nowadays. The techniques presented in this paper are particularly adequate to provide solutions to the problem of mutual insurance. We propose in future work to extend the methodology to approach more realistic situation. II. I MPULSE C ONTROL A ND S INGULAR C ONTROL P ROBLEM A. Formulation of the Problem Consider a probability space (Ω, A, P) on which a standard Wiener process w (t) is defined. When there is no control we consider the process:
[email protected]
x (t) = x + µt + σw (t)
J. Liu is with Faculty of Business, The Hong Kong Polytechnic University
[email protected] J. Yuan is with Faculty of Business, The Hong Kong Polytechnic University
[email protected]
978-1-4244-3872-3/09/$25.00 ©2009 IEEE
t
(1)
We define F to be the filtration generated by w (t). An impulse control is an increasing sequence of F t stopping
8512
FrC15.3 times τi and random variables ξi 6= 0. We modify the trajectory (1) to get a controlled process by setting
and ν (t) = ν + (t) − ν − (t). Clearly X dν + (t) = ξi · 1Iξi >0 δτi (t)
x (τi ) = x (τi − 0) + ξi
i
x (t) = x (τi ) + µ (t − τi ) + σ (w (t) − w (τi ))
X dν − (t) = − ξi · 1Iξi >0 δτi (t)
(2)
for τi ≤ t < τi+1
i
ˆ
and
∞
e−rt dν + (t) =
x (0) = x ˆ
If we define ν (t) =
X
ξi · 1Iτi ≤t
(3)
0 ∞
i
X e−rt dν − (t) = − ξi · 1Iξi 0 + K − 1Iξ 0. We next set "ˆ
#
∞
f (x (t)) e
−rt
X dt + g (ξi ) e−rτi
0
V (x) = inf Jx (ν)
(4)
We next define a cost function to be optimized. We set
Jx (ν) = E
X ξi · 1Iξi >0 e−rτi
ν
B. Dynamic programming If the value function V (x) is C 1 then the classical optimality principle approach leads to the following analogue of Bellman equation ΓV (x) ≤ f (x) ∀x V (x) ≤ M V (x) (ΓV (x) − f (x)) (V (x) − M V (x)) = 0
(12)
where (7)
1 ΓV (x) = − σ 2 V 00 (x) − µV 0 (x) + rV (x) 2 M V (x) = inf [g (ξ) + V (x + ξ)]
i
and we consider the value function
(13)
ξ6=0
V (x) = inf Jx (ν) ν
(8)
This is the formulation of the impulse control problem. We turn now to the definition of the singular control problem. We consider now K + , K − = 0 in (6). A control is now a pair ν (t) = (ν + (t) , ν − (t)), where ν + (t) , ν − (t) are monotone non decreasing, positive, cadlag ( right continuous and left limit ) stochastic processes. adapted to the filtration F t . We assume that ν (t) satisfies ˆ ∞ E e−rt dν + (t) + dν − (t) < ∞ (9)
This problem is called a Q.V.I. ( Quasi Variational Inequality ). We solve (12) and find that it has a two band solution. Namely there exists a < 0, b > 0, such that a0 1Iτi b} (17) i t≥ˆ τ1 X x (ˆ τ2 ) = x (ˆ τ2 − 0) + ξˆ (x (ˆ τ2 − 0)) ν − (t) = − ξi · 1Iξi 0, we can obtain and optimal impulse control νx from the feedback rule defined by (16) ξˆ (x) corresponding to numbers a , b , A , B . We get a sequence τˆi,x of stopping times and impulses
(28)
M ϕ (x) = inf [g (ξ) + ϕ (x + ξ)]
(29)
Operator M :
ξ
QVI:
ξˆi,x = ξˆ (x (ˆ τi,x − 0))
ΓV (x) ≤ f (x) = hx+ + px− V (x) ≤ M V (x) (ΓV (x) − f (x)) (V (x) − M V (x)) = 0
we next consider, as seen in section II-A. P x, (ν + ) (t) = ξˆi,x · 1Iξˆ,x >0 1Iτˆi,x 0. Then we can find a ≤ 0, b ≥ 0 and V (x) solution of (33),...,(37) such that V (x) is a solution of the Q.V.I. (30) Proposition 1: If V (x), a, b is a solution of (33), (34), (35), (36), (37) which is C 1 and satisfies V 00 (a) ≤ 0
V 00 (b) ≤ 0
−c+ ≤ V (x) ≤ c−
x ∈ (a, b)
I MPULSE C ONTROL P ROBLEM
(39)
VI. N OTATION OF THE P ROBLEM The Impulse Control problem will appear as a generalization of the singular control problem. The function g (x), given this time by
V. F UNCTION v (x; a, b); C HOICE OF a, b We consider v = V 0 . It is useful to set
g (ξ) = K + 1Iξ>0 + K − 1Iξ 0
(41)
The function u (a) verifies u (−∞) = +∞ u (0) = a0 (b0 (0)) < 0, hence there must exists a ˆ such that (41) is satisfied. The corresponding ˆb = b0 (ˆ a).
(38)
then it is a solution of (30).
+
u (a) = a0 (b0 (a)) − a = 0
ξ6=0
h − rc− > 0
(40)
A. The function v + (x; a, b) For given b we can define a unique a0 (b) such that first condition (38) is satisfied if a < a0 (b), and not satisfied otherwise. Proposition 2: Under the first assumption (40), we have a ≥ a0 (b) ⇒ v + (x; a, b) > 0
∀ x ∈ (a, b]
(43)
When K + = K − = 0 we recover (29). The Q.V.I. is still defined by (30). The two band problem (33), (34), (35), (36), (37) becomes to find V (x) ∈ C 2 such that ΓV (x) = f (x), a.e. , a < x < b V (x) = −c+ x + K + + P (a, b)V, x ≤ a
(44)
V (x) = c− x + K − + Q(a, b)V, x ≥ b and the continuity condition
If a ≤ a0 (b), there exists a unique A (a, b) such that v + (x; a, b) < 0 ∀ x ∈ (a, A) and v + (x; a, b) > 0 ∀ x ∈ (A, b], with v + (A; a, b) = 0. We proceed by stating properties of the curve a0 (b). Proposition 3: Under the first assumption (40) the function a0 (b) is uniquely defined on [0, ∞). There exists a unique value b∗ such that a00 (b∗ ) = 0 a00 (b) > 0 0 ≤ b < b∗ and a00 (b) < 0 for b > b∗ . The function a0 (b) increases on (0, b∗ ) from a0 (0) to a0 (b∗ ) then decreases for b > b∗ from a0 (b∗ ) to a0 (∞).
V (a) = −c+ a + K + + P (a, b)V V (b) = c− b + K − + Q(a, b)V V 0 (a) = −c+
(45)
V 0 (b) = c− We shall obtain the equivalent of Theorem 2, Theorem 3: Assume h − rc− > 0, p − rc+ > 0. Then there exist a, b and V (x) solution of (44), (45) such that V (x) is a solution of the Q.V.I. (30).
8515
FrC15.3 VIII. P ROPERTIES OF a (b) AND b (a)
A. Solution of the QVI We here give the equivalent of Proposition 1, namely we give the conditions under which a solution of the two band control problem solves the Q.V.I. We state Proposition 6: If V (x) ∈ C 1 is a solution of (44), (45), such that V 00 (a) ≤ 0 V (x) ≤ −c+ x+K + + inf
x 0 finite
(56)
Whenever (61) or (63) are not satisfied, then the pair a, A or b, B have to be obtained simultaneously by solving a system
8516
FrC15.3 B. Properties of aα (b) and bβ (a) Proposition 8: Under the assumption (40), the functions aα (b) and bβ (a) are uniquely defined by (60) and (62) respectively on (−∞, 0) and (0, ∞). There exist unique points b∗α > 0 and a∗β < 0 such that a0α (b∗α ) = 0 , b0β a∗β = 0 The function aα (b) increases on (0, b∗α ), then decreases on (b∗α , ∞). The function bβ (a) decreases on −∞, a∗β , then increases on a∗β , 0 . Theorem 4: Assume (40). We have if if
aα (b∗α ) + α < 0 then a (b) = aα (b) bβ a∗β − β > 0 then b (a) = bβ (a)
(64) (65)
C. More properties and search procedure The only cases when a (b) 6= aα (b) or b (a) 6= bβ (a) is when one of the two conditions (64) and (65) is not satisfied, and if (64) is not satisfied, b1α < b < b2α if (65) is not satisfied, a1β < a < a2β If we consider a point of intersection a ˆ, ˆb and if Aˆ = ˆ = B a A a ˆ, ˆb , B ˆ, ˆb , then we may have Aˆ < 0 and ˆ > 0, in which case B Aˆ = a ˆ+α In such a case
,
ˆ = ˆb − β B
a ˆ = a ˆb = aα ˆb ˆb = b (ˆ a) = bβ (ˆ a)
and then a ˆ, ˆb is also a crossing point of the curves aα (b) and bβ (a). ˆ > 0 so we have If Aˆ > 0 then B
Moreover ˆb = b (ˆ a) 6= bβ (ˆ a), hence we must have bβ a∗β − β < 0 and the interval a1β , a2β is well defined ( possibly a1β = −∞ ). Necessarily a1β < a ˆ < a2β Since a ˆ = a ˆb = aα ˆb , then either aα (b∗α ) + α < 0, or if aα (b∗α ) + α > 0, then the values b1α , b2α are well defined (possibly b2α = +∞ ) and ˆb < b1 α
The second possibility disappears when b2α > β ( in particular when b2α = +∞). From this discussion we can define the following search procedure for the pair a ˆ, ˆb. We consider the curves aα (b) and bβ (a). The above search procedure defines a crossing point a ˆ, ˆb of the two curves a (b) and b (a). From this discussion we found that the solution is easily ∗ found if aα (bα ) + α < 0, bβ a∗β − β > 0, or the crossing point a ˆ, ˆb , of aα (b), bβ (a) satisfies that a ˆ + α < 0, ˆb − β > 0. Otherwise we need to solve system of 3 nonlinear algebraic equations with a minor limitations on the range of the solution, which facilitate the search. R EFERENCES [1] Bensoussan, A. and Lions, J.L., Impulse control and quasi-variational inequalities, Gauthier-Villars, 1984 [2] Cadenillas, Abel and Zapatero, Fernando, Optimal central bank intervention in the foreign exchange market, Journal of Economic Theory, 1999, pp. 218-242 [3] Constantinides, George M. and Richard, Scott F., Existence of optimal simple policies for discounted-cost inventory and cash management in continuous time, Operations Research, 1978, pp. 620-636 [4] Kumar, Sunil and Muthuraman, Kumar, A numerical method for solving singular stochastic control problems, Operations Research, 2004, pp. q563-582
ˆ = ˆb − β , b (ˆ B a) = bβ (ˆ a) Moreover a ˆ = a ˆb 6= aα ˆb , hence we must have aα (b∗α ) + α > 0 and the interval b1α , b2α is defined ( possibly b2α = +∞ ). Necessarily b1α < ˆb < b2α Since also ˆb = b (ˆ a) = bβ (ˆ a), either bβ a∗β − β > 0, or if bβ a∗β − β < 0, the interval a1β , a2β is well defined (possibly a1β = −∞ ) and a ˆ > a2β
or ˆb > b2α
or a ˆ < a1β
The second possibility disappears if a1β +α < 0 ( in particular if a1β = −∞). ˆ < 0 then Aˆ < 0 so we have Similarly if B Aˆ = a ˆ + α , a ˆb = aβ ˆb
8517
