Abstruct-In practical applications of importance sampling (IS) simulation, we encounter two basic problems, that of determining the estimation variance and ...
1413
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 39, NO. 10, OCTOBER 1991
Transactions Letters Estimation Variance Bounds of Importance Sampling Simulations in Digital Communication Systems D. Lu and K. Yao
Abstruct-In practical applications of importance sampling (IS) simulation, we encounter two basic problems, that of determining the estimation variance and evaluating the proper IS parameters needed in the simulations. We derive new upper and lower bounds on the estimation variance which are applicable to IS techniques. Furthermore, the upper bound is simple to evaluate and may be minimized by the proper selection of the IS parameter. Thus, lower and upper bounds on the improvement ratio of various IS technique relative to the direct Monte-Carlo simulation are also available. These bounds are shown to be useful and computationally simple to obtain. Based on the above proposed technique, we can readily find practical suboptimum IS parameters. Specific numerical results indicate that these bounding techniques are useful for IS simulations of linear and nonlinear communication systems with IS1 in which BER and IS estimation variances cannot be obtained readily by prior techniques.
Fig. 1. Digital communication system model with rectangular gating function P(.), memoryless device g 1 ( . ) , and linear filter h( .).
y(t), which is sampled at symbol-spaced instants to produce a sequence of decision variables {y~,,k = 1 , 2 , . . .}. The input and the output are related by y(t) = g ( s ( t ) )where g ( - ) may be linear or nonlinear and usually with memory. In simulation, we deal with discrete-time samples of the realizations of the processes. Let the output of the simulation be denoted by YI, = g ( X ) where X = [XI,,Xk-1,. ,XI,-M+~] = A N, is the input vector with M components, A = [ A ~ , A I , - ~ , , AI,-^+^] is an i.i.d. input data vector, and N = [NI,,NI,-1,. . . ,NI,-M+~] is an i.i.d. distributed noise vector. Their values are denoted by the corresponding lower case vectors of 2, U , and n. In a binary detection problem, with AL = f A and a decision threshold of T, the conditional BER under HO (where AI, = -A), is given by PO= J-", D(y - T)fO(y) dy where fo(y) is the pdf of the system output r.v. y~,under HO and D(.)is the indicator function. Then POcan be reexpressed as
+
I. INTRODUCTION
A
LTHOUGH IS techniques have been proposed [1]-[5] for the estimation of low BER, various practical problems are encountered in their applications. Two basic problems in the use of any IS technique are the determination of the estimation variance (and thus the improvement ratio of the IS approach relative to the direct MC approach), and also the proper selection of IS parameters needed in the simulation. Direct analysis can be used to solve these problems only in a few simple cases. However, for most practical digital communication systems, the analysis and evaluation of these variances and improvement ratios are quite difficult. Indeed, the complexity of the evaluation of these quantities may be comparable to the evaluation of the BER of the system. Thus, if we wish to rely only on direct analysis and numerical evaluation for solving these problems, we may be trading one difficult problem on the evaluation of the BER for another difficult problem on the evaluation of the estimation variance. In this letter, we propose a bounding technique to solve these problems. Let the input to the system be a ( t ) consisting of a signal plus a noise as shown in Fig. 1. the output is a waveform Paper approved by the Editor for Signal Design, Modulation, and Detection of the IEEE Communications Society. Manuscript received October 5, 1988; revised September 12, 1989 and December 1,1990. This work was supported in part by NASNAmes under Grant NCC-2-374 and under an NSF Grant NCR-8814407. This paper was presented at the Conference on Information Science, Princeton, NJ, March 1987. D. Lu is with EEsof Inc., Westlake Village, CA 91362. K. Yao is with the Department of Electrical Engineering, University of California, Los Angeles, CA 90024. IEEE Log Number 9102267.
s_, 00
Po =
+
D[g(u n ) - TIfo(a)f1(n)dad?
(1)
where fo(u) and f i ( n ) are the pdf's of the input data A under HO and noise vectors N , respectively. Consider the modification of the original pdf fl(n) to a new pdf fT(n). For all n such that f : ( n ) > 0, then the weighting function is defined by w ( n ) = fi(n)/fT(n). There are two possible approaches to construct IS estimators. From (1): we propose the first scheme, in which the IS estimator PG is divided into J = 2 M - 1 subestimators P,*(j),j = 1 , 2 , .. . ,J, such that J
P; = ( l / J ) E P ; ( j )
(2)
j=1
+
Pt(j) = (J/N*)CEYJD[g(u(j) n(j,i)) T]w(n(j,i)), N* is the total runs in the IS simulation, and where
n ( j , i ) represents the ith realization of the noise conditioned
0090-6778/91$01.00 0 1991 IEEE
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 39, NO. 10, OCTOBER 1991
1414
on each a ( j ) realization of A. We assume N * and J are chosen such that N * / J is an integer. References [4] and [5] used the estimator in the form of (2). Based on evaluations of the estimation variances and improvement ratios of conventional importance sampling (CIS) and improved importance sampling (11s) techniques, the 11s approach was shown to be uniformly more efficient than the direct MC and the CIS approaches in terms of number of runs under a fixed simulation estimation variance constraint for a linear system with multidimensional Gaussian noises [4]. In [5], by using the estimator of (2), based on large deviation theory, the IIS scheme has been extended to a non-linear communication system, and a large improvement ratio has been obtained. However, for the estimator of the form in (2), we need to use J = 2M-1 subestimators. For medium to large value of M , the computational cost of (2) is high. Thus we are motivated to consider other practical IS estimation approaches. From (l),a second scheme for constructing an IS estimator is given by N'
Po = (1/N*)
+
D[g(a, ni)- T]w(ni).
(3)
i=l
In this scheme, by averaging N* samples of signals and noises in the estimation region, the BER can be obtained directly as shown in Fig. 2. While this scheme is more practical, nevertheless several difficult problems such as the evaluation of estimation variances of IS simulations, and the proper selection of IS parameters, are still present. In this correspondence, we shall consider the estimator of the form in (3). A new bounding technique introduced in Section I1 can be used to evaluate IS estimation variances and parameters. Since the IS estimator of (3) is unbiased, the estimation variance for the estimator of (3), assuming independent errors, is given by
1s Estimator Based on (3)
Original
Modified
Fig. 2. IS simulation model for estimating BER performance.
11. BOUNDSON ESTIMATION VARIANCE AND IMPROVEMENT RATIO In using any IS technique, an important basic problem is the evaluation of the estimation variance, which is a simple measure of the estimation accuracy. It is well known that an estimation technique whose variance cannot be determined has little practical meaning. From (4), it is clear that the evaluation of the variance is difficult. We shall propose a new bounding technique to solve this problem. Recall that in using any IS technique, the variance c2 of the original noise pdf may be increased to 0: [1]-[5], or the original noise pdf f l ( n ) may be translated to fl(n - c ) [4]. As shown in [1]-[5], the usefulness and effectiveness of these IS techniques, depend critically on the proper selection of (T* and c parameters, which we shall denote as IS parameters represented generically by a. As an intermediate step, we consider Theorem 1, which provides upper and lower bounds on the IS simulation variance and improvement ratio. Theorem 1: Consider a binary detection problem, with pdf's fo(a)and f l ( n )of the random data vector A (with AI, = -A) and noise vector N , and IS scheme with an IS parameter a, any weighting function w ( n , a ) , and a decision space 0 = [(a, n ) : g(a + n ) 2 TI. Then the estimation variance is upper and lower bounded by
.,",(a) 5 a6B(a) = [pO/N*][sup w ( n ,a ) nECl
m
0:s =
1 N*
J D[g(a+
*
1 f1(n)fo(a)dadn = N*
*
S,[w(n)- f3oIf1(n)fo(a) d a h
where PI(&)is the error probability using the modified pdf f:(n, a) of the IS simulation. The improvement ratio is then upper and lower bounded by (4)
+
where R = [ ( a , n ): g(a n ) 2 TI. By letting w ( n ) = 1,the MC estimation variance is given by
1
00
= (l/N)
-cm
(6)
n ) - T][w(n) - Po]
-cm
&c
lp,
~ M C / I S ( ~I ) TUB(^)
= P i ( a ) [ l - pol / p o [ l - pl(a)ll (8)
D[g(a+ n ) - TIP - Pol
. f1(n)fo(a) d a d n = Po(1 - Po)/N.
(5)
We use the notation of N* for the number of IS simulations and N for the number of MC runs. The improvement ratio (IR) of IS to MC simulations is defined by T M C / I S = Na&,/N*a&. It describes the reduction of simulation runs of the IS approach with respect to the direct MC scheme for the same estimation variance, or equivalently as the reduction of the IS estimation variance relative to the MC estimation variance for the same number of runs.
Furthermore, the equalities in the bounds of (6)-(9) are attained if w ( n , a ) = 200 is any constant in the decision region R. Then the bounds of (6)-(7) and (8)-(9) provide direct approaches for estimating the simulation variance and the improvement ratio, respectively. Another basic problem encountered in the application of IS techniques is the proper selection of IS parameters. The optimum IS parameters are chosen to minimize the exact
I'
1
1
1415
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 39, NO. 10, OCTOBER 1991
0 AIIS Acrs estimation variance expression for a given IS technique. However, in practice, it is not realistic to find the optimum IS parameters exactly due to the complexity of minimizing the estimation variance with respect to the parameters. We shall proposed a practical approach for finding suboptimum IS parameters. By using the suboptimum IS parameter, the estimation variance of the simulation can be controlled in an acceptable manner. From Theorem 1, for a given IS scheme and associated PO and N*, we want to minimize oiB(a).The suboptimum IS parameter sop) is defined by aiB(a(s0p)) = inf, aiB(a) = (Po/N*)[inf, SUP, ~ ( n a ), - Po] = yal del (Po/N*)[sup, w(n, a(sop)) - PO]. Correspondingly, ~ L B ( C X Fig. 3. Normalized deviation of IS parameters versus SNR for A4 = 1. (sop)) is defined by +
TLB(~SOP))= ~
-
u P ~ L B=( infa ~)
1- Po w(n,
logrus(CIS)
1 - Po
SUP, w(n, a(sop))- Po 1
N
SUP,
4% 4sop)).
+ logrE,,(CIS)
logrLs(CIS)
(10)
Since the bounds in (6)-(9) are valid for any a, then these bounds are also valid for a(sop), and we I 4 j B ( a ( s o p ) ) and have & ( a ( s o p ) ) I T L B ( ~ S O P ) ) I ~ M C / I S ( ~ S O P )I ) ~ u B ( ~ ~ o P ) ) . In practice, we optimize the bound in (6) or (9) to find suboptimum IS parameters. In practical simulations, the bound in (9) is more useful. As can be seen from (9), for PO
