Nonlinear Filtering Equation of a Jump Process with ... - Springer Link

Acta Applicandae Mathematicae 66: 139–154, 2001. © 2001 Kluwer Academic Publishers. Printed in the Netherlands.

139

Nonlinear Filtering Equation of a Jump Process with Counting Observations CLAUDIA CECI1 and ANNA GERARDI2

1 Dip. di Scienze, Università di Chieti, Pescara, Italy 2 Dip. Ingegneria Elettrica, Università de L’ Aquila, Italy

(Received: 20 May 1999) Abstract. We study the filtering problem of an Rd -valued pure jump process when the observations is a counting process. We assume that the dynamic of the state and the observations may be strongly dependent and that the two processes may jump together. Weak and pathwise uniqueness of solution of the Kushner–Stratonovich equation are discussed. Mathematics Subject Classifications (2000): Primary 60J75, 93E11; Secondary 60G55, 60G35. Key words: Markov jump processes, filtering.

Introduction A partially observable model is a stochastic process (Xt , Yt ) in which Xt represents a signal which cannot be observed directly and we get information about Xt by observing a process Yt , related to Xt . Thus, at time t, the σ -algebra F Yt := σ {Ys ; s t} provides all the available information about Xt , and our knowledge of Xt reduces to πt , which is the conditional distribution of Xt given Ft Y . The primary aim of filtering theory is to characterize this conditional distribution. In particular, this issue plays a central role in the study of a control problem under partial observations. Because of the difficulty of a direct approach to a partially observable control problem a ‘separated’, completely observed control problem can be introduced (N. El Karoui, D. Huu Nguyen and M. Jeanblanc-Picqué, 1988; Elliott, 1992; Fleming, 1980; Fleming and Pardoux, 1982; Hijab, 1989; Mazliak, 1992; Ceci and Gerardi, 1998, 1999). The original and the separated problems are related via the filter equation. More precisely, the separated problem is the problem of control (under complete information) of the measure-valued solution of the filter equation. Uniqueness for this equation is then necessary to prove that this new problem is equivalent to the original one, in the sense that the infimum expected cost of the separated problem is equal to the infimum expected cost of the original problem. In this paper, we study how to characterize the conditional distribution for filtering models in which (Xt , Yt ) is a pure jump Markov process, Xt takes val-

140

CLAUDIA CECI AND ANNA GERARDI

ues in Rd and Yt is a counting process. To this end we will write down the so called Kushner–Stratonovich equation for the filter πt and we will show that, under suitable assumptions, it has a unique solution both in weak and strong sense. The problem of uniquely characterize the conditional distribution of a partially observed system, has been largely studied (see, for instance, Brémaud, 1980; Kurtz and Ocone, 1988; Kliemann, Koch and Marchetti, 1990; Bhatt, Kallianpur and Karandikar, 1995 and references therein). Kurtz and Ocone (1988) approached the problem of uniquely characterize the conditional distribution of a partially observed system by defining a ‘filtered martingale problem’ (FMP), a martingale-type problem involving a probability-measure-valued process and the generator for the martingale problem for (Xt , Yt ). They gave conditions on the original martingale problem solved by (Xt , Yt ) that insure that the conditional distribution is, in fact, the unique solution to the filtered martingale problem. On the strength of these results, they obtained uniqueness theorems for the Kushner–Stratonovich and Zakai equations for the conditional distribution of a Markov process Xt , observed via a process, Yt , function of {Xs ; s t} corrupted by Gaussian white noise. Kliemann, Koch and Marchetti (1990) applied the results to filtering problems for a partially observed system, in which the state process is a quite general Ito process and the observations is a counting process. Under some hypotheses, in particular boundedness of the jump sizes, they proved weak uniqueness for the Kushner–Stratonovich equation. In the most part of existing literature, a diffusion component is present in the dynamics of the state-observation process. Nevertheless in a great variety of significant problems, the filtering of jump processes naturally arises (Fan, 1996). In this paper, we consider a pure jump Markov process (Xt , Yt ), whose dynamics may be strongly dependent, namely the jump intensities both of state and observation processes depend mutually on each other and the two processes have common jumps. Hence, neither the state nor the observation process is, in general, a Markov process. The special case in which the state process is a Markov process and state and observation processes have only common jumps has been dealt with in Ceci and Gerardi (2000). Our main contribution is to give a fairly large set of possible hypotheses on the generator for the martingale problem for (Xt , Yt ) which insure uniqueness for the solutions of the associated filtered martingale problem and, as a consequence, to guarantee weak uniqueness for the Kushner–Stratonovich equation. Moreover, under some suitable additional conditions, we prove pathwise uniqueness of solution of this equation. The paper is organized as follows. In Section 1 we introduce the dynamic of the pair state-observations as the unique solution of a martingale problem. The Kushner–Stratonovich equation and the filtered martingale problem are introduced in Section 2. We prove, by using uniqueness results for the filtered

141

NONLINEAR FILTERING EQUATION

martingale problem, weak uniqueness for the Kushner–Stratonovich equation. The main results of this section are then summarized in Theorem 2.8. In Section 3, under fairly reasonable assumptions, a Girsanov-type change of probability measure is established and pathwise uniqueness of solution of the filtering equation is obtained even though the intensity λt of Yt , is not supposed to be strictly positive (as it was supposed in Ceci and Gerardi, 2000). Finally, in Section 4 a discussion about the assumptions used in Section 3 to prove strong uniqueness for the Kushner–Stratonovich equation is performed. The central result of the last two sections is then summarized in Theorem 4.3. NOTATIONS

If (E, B(E)) is a metric space endowed with its Borelian σ -algebra, B(E) denotes the space of bounded measurable functions on E. Cb (E) is the space of bounded continuous functions on E, Lip(E) the set of Lipschitz functions on E and DE [0, ∞) the space of right continuous E-valued functions on [0, ∞) having left limits. Let (E) be the space of probability measures on E. Finally L(X) denotes the law of a process X. 1. The Model In this section we introduce the dynamic of the pair state-observations ξt = (Xt , Yt ) as the unique solution of a martingale problem. Let E = Rd × N and L be the operator on B(E) (1.1) Lf (ξ ) = (ξ ) (f (η) − f (ξ ))µ(ξ, dη), E

where ξ = (x, y) ∈ E, µ(ξ, dη) is a transition function on E × B(E), and (ξ ) is a nonnegative measurable function on E. More precisely, L may be written as Lf (ξ ) = L0 f (ξ ) + L1 f (ξ ), where

L0 f (x, y) = λ0 (x, y)

Rd

L1 f (x, y) = λ(x, y) and (ξ ) = λ0 (ξ ) + λ(ξ ).

Rd

(f (x , y) − f (x, y))µ0 (x, y, dx ),

(f (x , y + 1) − f (x, y))µ1 (x, y, dx ),

(1.2)

(1.3)

142


The operator L0 describes the evolution of the state between the jump times of Yt , namely in the time interval when Yt = y, whereas L1 describes the evolution of the state at a jump time of Yt . When L0 = 0 the state can not jump alone, as in the case in which Yt counts the total jumps of Xt (see Ceci and Gerardi, 2000). In the sequel we will assume one of the following classes of hypotheses which ensure existence of a solution of the martingale problem (MGP) for (L, ν0 × δ0 ), for each ν0 ∈ (Rd ) with finite first moment. (H0) ∃K > 0: 0 (ξ ) K ∀ξ ∈ E, (H1) ∃K >0: 0 (ξ ) K(|ξ | + 1) ∀ξ ∈ E, (ξ ) E |η − ξ |µ(ξ, dη) K(|ξ | + 1) ∀ξ ∈ E. Under (H0) there exists a unique solution of the MGP(L, ν0 × δ0 ) with sample paths in DE [0, ∞). In the next propositions we will discuss the case of unbounded intensity. PROPOSITION 1.1. Under assumption (H 1), and if E|ξ0 | < +∞, there exists a solution ξt of the MGP(L, ν0 × δ0 ) with sample paths in DE [0, ∞). Moreover, E|ξt | eKt (E|ξ0 | + 1). Proof. On the space of the sample paths of marked point processes, let (τn , ξn )n0 be the canonical process (we set τ0 = 0). Define ξn 1I{τn t C } µ(ξ, {|η| C}) = 0,

∀ξ ∈ E,

then there exists a unique solution of the MGP(L, ν0 × δ0 ) with sample paths in DE [0, ∞). Proof. By (H1) and either (H2) or (H3) it follows that ∀f ∈ C0 (E), Lf ∈ B(E). Hence, taking D = C0 (E), we get that L ⊂ Cb (E) × B(E) and we can apply Theorem 6.3 of (Ethier and Kurtz, 1986). Then the proof is a consequence of the following facts. (i) Setting Un = {ξ ∈ E : |ξ | n}, the Stopped MGP(L, ν0 × δ0 , Un ) has a unique solution ξn (t) with sample paths in DE [0, ∞). A solution is provided by ξt ∧ρn , where ρn = inf{t 0 : |ξt | n}. (ii) Define ρ n = inf{t 0 : |ξn (t)| n}. It is easy to verify that P ( ρn t) P (ρn t) and that, since ξt is a nonexplosive ✷ process, limn→∞ P (ρn t) = 0. Remark. Observe that (H1) and the condition (H4) ∃R > 0: ∀ξ, µ(ξ, B(ξ, R)) = 1, where B(ξ, R) = {η ∈ E: |η − ξ | R}, implies (H2).

2. Weak Uniqueness for the Kushner–Stratonovich Equation It is known that there exists a cadlag probability measure-valued, Ft Y -adapted process πt such that πt (f ) = E(f (Xt )|Ft Y )

∀f ∈ B(Rd ).

144


Moreover the Ft Y -intensity of Yt is given by πt (λ(· , Yt )) = E(λ(Xt , Yt ) | Ft Y ) and πt solves the following Kushner–Stratonovich (KS) equation (Elliott, 1982) t πs (Lf (·, Ys )) ds + (KS) πt (f ) = ν0 (f ) + 0 t + {πs− (L1 f (·, Ys− )) − πs− (f )πs− (λ(·, Ys− )) + 0

+ πs− (λ(·, Ys− )f )}(πs− (λ(·, Ys− )))+ (dYs − πs− (λ(·, Ys− )) ds), where ∀f ∈ B(Rd ) the third term of the right-hand side is an Ft Y -martingale. We want to prove that the conditional law πt is the unique solution of (KS). As in Kurtz and Ocone (1988), Kliemann, Koch and Marchetti (1990) and Ceci and Gerardi (2000), in order to uniquely characterize πt , we use the filtered martingale problem (FMP) approach. We start by giving the following definition: ) with sample paths in Dπ(Rd )×N [0, ∞) is a DEFINITION 2.1. A process ( π, Y solution of the FMP for L, if π is Ft Y -adapted and ∀F ∈ B(E) t t ) − s ) ds πt F (·, Y πs LF (·, Y (2.1) 0 Y

is an Ft -martingale. The filter-observation process, (πt , Yt ), solves the FMP for L, thus if uniqueness holds for the solutions of the FMP satisfying the initial condition 0 )) = E(F (X0 , Y0 )), E( π0 F (· , Y ) has the same distribution as (π, Y ). then ( π, Y Moreover, there exists a Borel measurable function ,t : DN [0, ∞) → (Rd ) such that ) = ,t (Y (· ∧ t)) πt = ,t (Y

a.s.

(2.2)

and, hence, πt = ,t (Y ). In the next proposition we prove that uniqueness for the FMP (L, ν0 × δ0 ) implies weak uniqueness for the Kushner–Stratonovich equation. First, we give the following definition: ) defined DEFINITION 2.2. As a weak solution of (KS) we mean a process ( π, Y on a probability space (-, Ft , P ) with sample paths in Dπ(Rd )×N [0, ∞), such that πt |x| = Rd |x| πt (dx) < ∞, P-a.s. – π is Ft Y -adapted and t )), t is a nonexplosive point process with minimal intensity given by πt (λ(·, Y –Y – the pair ( πt , Yt ) solves the equation (KS).

145


For each f ∈ B(Rd ), under the above assumptions, all terms in (KS) make sense and the third term of the right-hand side of (KS) is an Ft Y -local martingale. Fur thermore, the third term of the right-hand side of (KS) is an Ft Y -martingale if in t )) < ∞. For example, this is fulfilled if E( πt |x|) < ∞ and πt λ(·, Y addition E( E(Yt ) < ∞. ) is a weak solution of (KS) then ( ) solves, for PROPOSITION 2.3. If ( π, Y π, Y each F ∈ B(E), the local FMP(L, ν0×δ0 ). If uniqueness holds for the FMP(L, ν0× ) = L(π, Y ) and in particular L( π, Y π ) = L(π ). δ0 ), then L( Proof. It is sufficient to prove (2.1) for each function F of the form F (x, y) = f (x)g(y) with f ∈ B(Rd ) and g ∈ B(N). t )), for each g ∈ B(N) we t is a point process with intensity πt (λ(·, Y Since Y have the following representation t s + 1) − g(Y s )) s )) ds + t ) = g(0) + (g(Y πs (λ(·, Y g(Y 0 t s − + 1) − g(Y s − ))(dY s − s ))ds), (g(Y πs (λ(·, Y (2.3) + 0 where the third term of the rhs is a (P, Ft Y )-local martingale. Taking into account (KS), (2.3) and the product rule, t s ) + πs − (f ) dg(Y πt (f )g(Yt ) = ν0 (f )g(0) + 0 t s − ) d s ), g(Y πs (f ) + / πs (f )/g(Y + 0

one obtains that t ) − πt (f )g(Y

t

st

s + 1)( s )f (·)) + {g(Y πs (λ(·, Y πs (Lf )) −

0

s )f (·))} ds s ) πs (λ(·, Y − g(Y is a local (P, Ft Y )-martingale. Consequently, for each F ∈ B(E) t s ) ds πt F (·, Yt ) − πs LF (·, Y Mt = 0 is a local (P, Ft Y )-martingale. Hence, there exists a sequence of Ft Y -stopping t ∧τn is an uniformly integrable τn = ∞ and M times, { τn }n0 such that limn→∞ martingale. By Corollary 3.4 of Kurtz and Ocone (1988), it follows that

)1I{t 0: λ0 (ξ ) H λ(ξ ), ∀ξ ∈ E, (H7) ∃H > 0: 0 (ξ ) λ(ξ )H (|ξ | + 1), ∀ξ ∈ E, (ξ ) E |η − ξ |µ(ξ, dη) H λ(ξ )(|ξ | + 1), ∀ξ ∈ E. They mean that when the observation process stops and so the available information about the state process does not increase anymore the state process stops too. We are then able to establish a Girsanov-type change of probability measure. As in Proposition 1.1, let Q be the probability measure on the space of the sample paths of marked point processes such that the random measure m(dt, dξ ) has predictable projection given by (Jacod, 1975) ν (dt, dη) = λ+ (ξt − )(ξt − )µ(ξt − , dη) dt

(3.1)

and, under Q, L(ξ0 ) = ν0 × δ0 . One can prove that, under Q, ξt is a nonexplosive process such that E Q |ξt | < ∞ ∀t 0, and that it is the unique solution for the MGP(A, ν0 × δ0 ), i.e. ∀f ∈ B(E) ∪ Lip(E) t Af (ξs ) ds = (Q, Ft )-martingale, f (ξt ) − f (ξ0 ) − 0

where +

Af (ξ ) = λ(ξ ) (ξ )

E

(f (η) − f (ξ ))µ(ξ, dη).

More precisely, let γ (ξ ) = 1I{λ>0} (ξ ) and γ0 (ξ ) = λ(ξ )+ λ0 (ξ ). Then Af (ξ ) = A0 f (ξ ) + A1 f (ξ ) and

(f (x , y) − f (x, y))µ0 (x, y, dx ), A0 f (x, y) = γ0 (x, y) d R (f (x , y + 1) − f (x, y))µ1 (x, y, dx ). A1 f (x, y) = γ (x, y) Rd


149

In particular the point process Yt has (Q, Ft )-intensity given by γ (ξt ) = 1I{λ(ξt )>0} . Observe that either (H6) or (H7) imply that = 0 if and only if λ = 0. Consequently, the probability measures P and Q are equivalent on FT , ∀T > 0 (Jacod, 1979). Denote by qt the cadlag version of the Q-conditional distribution of Xt given Ft Y , namely qt (f ) = E Q f (Xt )|Ft Y , ∀f ∈ B(E), then

qt (γ (·, Yt )) = E Q 1I{λ(Xt ,Yt )>0} |Ft Y

is the minimal intensity of Yt under the probability measure Q. We now prove that uniqueness for the FMP(L, ν0 × δ0 ) implies strong uniqueness for the solutions of (KS), under the hypothesis that inft ∈[0,T ] qt (γ (· , Yt )) > 0, Q-a.s. (for example, this assumption is fulfilled if λ > 0, Q-a.s.; we will provide a less trivial condition in the next section) THEOREM 3.1. Assume that uniqueness holds for the FMP(L, ν0 × δ0 ) and that inft ∈[0,T ] qt (γ (·, Yt )) > 0, Q-a.s. Assume, moreover either, (H6) or (H7). πt be an Ft Y -adapted, Let (-, F , P ), and ξt as introduced in Section 1. Let d πt (λ(·, Yt )) > 0, πt (|x|) < ∞ P -a.s. and cadlag, (R )-valued process such that satisfying, ∀f ∈ B(Rd ) and ∀t T t πs (Lf (·, Ys )) ds + πt (f ) = ν0 (f ) + 0 t { πs− (L1 f (·, Ys− )) − πs− (f ) πs− (λ(·, Ys− )) + (3.2) + 0

πs− (λ(·, Ys− )))+ (dYs − πs− (λ(·, Ys− )) ds) + πs− (λ(·, Ys− )f )}( then πt = πt for all t < T P -a.s. Proof. Let lt be the exponential Q-supermartingale related to the bounded variation zero-mean local martingale t πs− (λ(·, Ys− )) − 1 dYs − qs− (γ (·, Ys− )) ds . Bt = qs− (γ (·, Ys− )) 0 It is known that lt is the unique solution of t ls− dBs lt = 1 + 0

and lt =

πs− (λ(·, Ys− )) − 1 /Ys × 1+ qs− (γ (·, Ys− )) st t πs (λ(·, Ys− )) qs (γ (·, Ys− )) ds . 1− × exp qs (γ (·, Ys−)) 0

(3.3)

150


Note that lt is not necessarily a martingale, therefore we define the sequence of Ft Y -stopping times πt (λ(·, Yt )) n ∧ T. σn = inf t 0 : qt (γ (·, Yt )) By (3.3) one gets that lt ∧σn nYt∧σn e(1+n)t and it is easy to verify that E Q (nYt∧σn ) < ∞, ∀t 0. Consequently the following holds

t ∧σn

πs (λ(·, Ys )) Q

− 1 qs (γ (·, Ys )) ds < ∞ E ls qs (γ (·, Ys )) 0 and this in turn implies that lt ∧σn is a (Q, Ft Y )-martingale. Introduce the sequence of probability measures Pn on FTY through their restrictions to Ft Y defined as

dPn

= lt ∧σn , ∀t ∈ [0, T ]. dQ FtY One gets that, under Pn , Yt has intensity given by πt (λ(·, Yt ))1I{t 0, ∀t, and that inft ∈[0,T ] qt (γ (·, Yt )) > 0, P -a.s., under condition (H6) and a further assumption on the transition function µ1 (see Propositions 4.1 and 4.2 below). First observe that, since under Q, Yt has minimal intensity given by qt (γ (· , Yt )) and ξt has generator A, the Q-conditional law of Xt given Ft Y , qt , is a solution of the equation t qs (Af (·, Ys )) ds + qt (f ) = ν0 (f ) + 0 t {qs− (A1 f (·, Ys− )) − qs− (f )qs− (γ (·, Ys− )) + (4.1) + 0

+ qs− (γ (· , Ys− )f )}(qs− (γ (·, Ys− )))+ (dYs − qs− (γ (·, Ys− )) ds). By (H6), A is a bounded operator, therefore by Proposition 2.3, we can identify qt with the unique (in law) solution of (4.1). Define I (n) = {x ∈ Rd : λ(x, n) = 0}. PROPOSITION 4.1. Under the assumption (H8) ∀n ∈ N,

∀x ∈ / I (n − 1),

µ1 (x, n − 1, I (n)) < 1,

we get that qtn (γ (· , n)) > 0 whenever qtn − (γ (·, n − 1)) > 0. Here {tn } denotes the sequence of the jump times of Yt .

(4.2)

152


Proof. By (4.1) we get qtn (γ (·, n)) 1I{qtn − (γ (·,n−1))>0} γ (x, n)µ1 (·, n − 1, dx) qt − γ (·, n − 1) = qtn − (γ (·, n − 1)) n Rd 1I{qtn − (γ (·,n−1))>0} = 1 − µ1 (x, n − 1, I (n)) qtn − (dx) qtn − (γ (·, n − 1)) Rd −I (n−1) then qtn (γ (· , n)) = 0 and qtn − (γ (·, n − 1)) > 0 imply µ1 (x, n − 1, I (n)) = 1,

∀x ∈ I (n − 1) qtn − -a.s. ✷

which is absurd.

In order to study the behaviour of qt (γ (·, Yt )) for t ∈ [tn , tn+1 ), by (4.1) we obtain that, for any bounded f , t qs (A0 f (·, Ys )) + qs (f )qs (γ (·, Ys )) − qt (f ) = qtn (f ) + tn

− qs (γ (·, Ys )f ) ds.

(4.3)

We want to construct a particular representation of the solution of (4.3). Consider, for n 0, the operator n (f (x ) − f (x))µ0 (x, n, dx ) A0 f (x) = γ0 (x, n) Rd

be the unique solution of the MGP(An0 , qtn ) after tn . and let n Let P = L(X.n ), and E n (·) = E P (·). Then, as in Kliemann, Koch and Marchetti (1990), we can set, for t ∈ [tn , tn+1 ) t E n f (Xtn ) exp − s γ (Xrn , n) dr (4.4) qt (f ) =

. t E n exp − s γ (Xrn , n) dr s=tn Xtn n

PROPOSITION 4.2. ∀t ∈ [tn , tn+1 ), qt (γ (·, n)) qtn (γ (·, n))e−(H +1)(t −tn) . Proof. By (4.4) we can write t E n γ (Xtn, n) exp − s γ (Xrn , n) dr

qt (γ (·, n)) = t

E n exp − s γ (Xrn, n) dr s=tn −(t −tn ) n n e E γ (Xt , n) . Setting

γ0 (x, n) γ0 (x, n) δx (dy) + µ0 (x, n, dy), pn (x, dy) = 1 − H H

153


one has An0 f (x) = H

Rd

f (y) − f (x) pn (x, dy)

and An0 γ (x, n) = Hpn (x, I c (n)) − H γ (x, n). By the definition of Xtn E n γ (Xtn , n) = qtn (γ (·, n))e−H (t −tn ) + t −H (t −tn ) eH u E n pn (Xun , I c (n)) du +He tn

and the thesis follows.

✷

As a conclusion observe that #{n ∈ N : tn T } < ∞, ∀T > 0 and therefore, if ν0 (I (0)) < 1 (i.e. ν0 (γ (·, 0)) > 0) then ∀n ∈ N inf qt (γ (·, n)) > 0.

t ∈[0,T ]

Moreover πt (λ(·, Yt )) > 0 a.s. ⇐⇒ qt (γ (·, Yt )) > 0 a.s. (They are the minimal intensities of Yt with respect to equivalent measures.) Our main result about pathwise uniqueness for the solutions of the (KS) equation is given, as a consequence of the previous discussions, in the following THEOREM 4.3. The same assumptions of Theorem 2.8 and the further conditions (H6) and (H8) provide strong uniqueness for the solutions of the (KS) equation.

References Bhatt, A. G., Kallianpur, G. and Karandikar, R. L. (1995), Uniqueness and robustness of solution of a measure-valued equations of nonlinear filtering, Ann. Probab. 23(4), 1895–1938. Brémaud, P. (1980), Point Processes and Queues, Springer, New York. Ceci, C. and Gerardi, A. (1997), Filtering of a branching process given its split times, J. Appl. Probab. 34(3), 565–574. Ceci, C. and Gerardi, A. (1998), Partially observed control of a Markov jump process with counting observations: Equivalence with the separated problem, Stochastic Process. Appl. 78(2), 245–260. Ceci, C. and Gerardi, A. (1999), Optimal control and filtering of the reproduction law of a branching process, Acta Appl. Math. 55(1), 27–50. Ceci, C. and Gerardi, A. (2000), Filtering of a Markov jump process with counting observations, Appl. Math. Optim. 42, 1–18. El Karoui, N., Huu Nguyen, D. and Jeanblanc-Picqué, M. (1988), Existence of an optimal Markovian filter for control under partial observations, SIAM J. Control Optim. 26(5), 1025–1061.

154


Elliott, R. J. (1992), A partially observed control problem for Markov chains, Appl. Math. Optim. 25, 151–169. Ethier, S. and Kurtz, T. (1986), Markov processes: Characterization and Convergence, Wiley, New York. Fan, K. (1996), On a new approach to the solution of the nonlinear filtering equation of jump processes, Probab. Engrg. Inform. Sci. 10, 153–163. Fleming, W. (1980), Measure-valued processes in the control of partially-observable stochastic system, Appl. Math. Optim. 6, 271–285. Fleming, W. and Pardoux, E. (1982), Optimal control for partially observed diffusions, SIAM J. Control Optim. 20(2), 261–285. Hijab, O. (1989), On partially observed control of Markov processes, Stochastics Stochastics Rep. 28, 123–144. Jacod, J. (1975), Multivariate point processes: Predictable projection, Radon–Nikodym derivatives, representation of martingales, Z. Wahrsch. Verw. Gebiete 31, 235–253. Jacod, J. (1979), Calcul stochastique et problèmes de martingales, Springer, New York. Kurtz, T. G. (1998), Martingale problems for conditional distribution of Markov processes, Electronic J. Probab. 3(9). Kurtz, T. G. and Ocone, D. (1988), Unique characterization of conditional distributions in nonlinear filtering, Ann. Probab. 16, 80–107. Kliemann, W., Koch, G. and Marchetti, F. (1990), On the unnormalized solution of the filtering problem with counting process observations, IEEE Trans. Inf. Theory 36(6), 1415–1425. Mazliak, L. (1992), Mixed control problem under partial observation, Appl. Math. Optim. 27, 57–84.