SURFACE WAVE PROPAGATION

4 downloads 0 Views 617KB Size Report
It is assumed here that the True pdf is of the Wakeby form. The five ..... Klemeš, V., 2000a: “Tall tales about tails of hydrological distributions, I”, J. Hydrol. Eng. 5.
A

C

T

A

G

E

O

P

H

Y

S

I

C

A

P

O

L

O

N

I

C

Vol. 53, no. 4, pp. 437-457

A 2005

ARE PARSIMONIOUS FF MODELS MORE RELIABLE THAN TRUE ONES? II. COMPARATIVE ASSESSMENT OF THE PERFORMANCE OF SIMPLE MODELS VERSUS THE PARENT DISTRIBUTIONS 1

1

Krzysztof KOCHANEK , Witold G. STRUPCZEWSKI , 2 3 Vijay P. SINGH and Stanisław WĘGLARCZYK 1

Institute of Geophysics, Polish Academy of Sciences ul. Księcia Janusza 64, 01-452 Warszawa, Poland e-mails: [email protected]; [email protected]

2

Department of Civil and Environmental Engineering, Louisiana State University Baton Rouge, LA 70803-6405, USA; e-mail: [email protected]

3

Institute of Water Engineering and Water Management, Cracow University of Technology ul. Warszawska 24, 31-155 Kraków, Poland; e-mail: [email protected]

Abstract Applying the methodology described in Strupczewski et al. (2005a; this issue), the performance of various parsimonious models combined with three estimation methods versus Flood Parent Distributions is comparatively assessed by simulation experiments. Moments (MOM), L-moments (LMM) and maximum likelihood (MLM) are used as alternative methods. Five four-parameter Specific Wakeby Distributions (SWaD) are employed to serve as Flood Parent Distributions and forty Distribution/Estimation (D/E) procedures are included in respect to the estimation of upper quantiles. The relative bias (RB), relative root mean square error (RRMSE) and reliability of procedures are used for the assessment of the relative performance of alternative procedures. Parsimonious two-parameter models generally perform better for hydrological sample sizes than their three-parameter counterparts with respect to RRMSE. However, the best performing procedures differ for various SWaDs. As far as estimation methods are concerned, MOM usually produces the smallest values of both RB and RRMSE of upper quantiles for all competing methods. The second place in rank is occupied by LMM, whereas, MLM produces usu-

K. KOCHANEK et al.

438

ally the highest values. Considerable influence of sampling bias on the value of the total bias has been ascertained. The improper choice of a model fitted to SWaD samples causes that the reliability of some three-parameter parsimonious D/E procedures does not always rise with the sample size. Also odd is that True model does not always give one hundred percent reliability for very large samples, as it should. This means that estimating algorithms still require improvements. Key words: flood frequency analysis, Monte Carlo, probability distribution, estimation methods, quantiles, bias.

1. INTRODUCTION

In flood frequency analysis (FFA), a probability density function (pdf) is selected more or less subjectively from among positively skew pdfs of continuous type. The vast variety of pdfs (D) and estimation methods (E) creates problems for practitioners: which D/E procedure should be used for a particular purpose. The choice of a D/E procedure is influenced by several factors of which the most important are: purpose of the modelling, sample size and the quality of data, country’s regulations and recommendations and finally the procedure’s robustness. On the other hand, the proper choice of robust D/E procedures for modelling of flood frequency events is of great importance, because it can have a key effect on the reliability of costly flood-related structures. Therefore, exploiting regional information and comparing several alternatives hydrologists attempt to look for the best procedure for a given condition and records. The analysis of flood phenomena in UK (Flood Estimation Handbook, 1999) reveals that the most suitable pdfs for modelling of extreme hydrological events are the heavy-tail distributions. In fact, the heavy-tail distributions such as GEV and GLL are becoming popular in FFA. Since Wakeby distribution also accounts for heavy-tail distributions, one can expect particularly good fit to the Wakeby generated samples. Applying the methodology described by Strupczewski et al. (2005; hereafter Paper I), the performance of various parsimonious models combined with three estimation methods versus Flood Parent Distribution (FPD) is comparatively assessed. Moments (MOM), L-moments (LMM) and maximum likelihood (MLM) are used as alternative methods. Simulation experiments are used for this purpose. As a tool the Accuracy of Quantiles and Moments Estimation (AQME) software package described in Part I is used. The paper is organised as follows. Parent and parsimonious distributions with information on algorithms applied for parameter estimation as well as design of simulation experiment are described in Section 2. The next section provides a short analysis of the reliability of D/E procedures. Then Sections 4 and 5 give results on relative bias and relative root mean square error, respectively. The last part, Section 6, summarizes and concludes the paper.

ARE PARSIMONIOUS FF MODELS … PART II

439

2. SIMULATION EXPERIMENT

It is assumed here that the True pdf is of the Wakeby form. The five Specific Wakeby Distributions (SWaDs) applied by Landwehr et al. (1980) serve as parent distributions and the base for computer pseudo-random generators. The parameters of SWaDs, their basic statistical characteristics and 1% theoretical quantiles are shown in Table 1 (see also: Paper I, eq. 3). The last column in Table 1 was added, because all the values of bias and root mean square error presented in this article refer to the theoretical value of the quantile. Figure 1 shows the density function and probability distribution function of all SWaDs. T ab le 1 Specific Wakeby Distributions (SWaD) Parameters

SWaD m a Wa I

0

b

1 16.0

c

Moment characteristics d

µ

σ

CV

CS

4 0.20 1.94 1.34 0.69 4.14

CK 63.74

L-moment characteristics

λ1

λ2

1.94 0.61

Quantile

L-CV

L-CS

L-CK

Xp = 1%

0.31

0.33

0.11

7.05

Wa II

0

1

7.5

5 0.12 1.56 0.90 0.58 2.01

14.08

1.56 0.46

0.29

0.17

0.09

4.69

Wa III

0

1

1.0

5 0.12 1.18 1.03 0.87 1.91

10.73

1.18 0.53

0.45

0.26

0.09

4.68

Wa IV

0

1 16.0 10 0.04 1.36 0.51 0.38 1.10

7.69

1.36 0.26

0.19

0.12

0.04

3.02

Wa V

0

1

4.73

0.92 0.38

0.41

0.18

0.05

3.01

1.0 10 0.04 0.92 0.70 0.76 1.11

Fig. 1. Density functions (left) and probability distribution functions (right) of SwaDs.

Calculations were carried out for all alternative parsimonious distributions; one can learn from the moment ratio diagrams (Fig. 2) as to which parsimonious models are suitable to model the specific Wakeby distribution with given moment characteristics. The black dots on the graph mark SWaDs, i.e., Wa I, Wa II, etc. One can notice, that Wa IV “lays closest” to the two-parameter parsimonious distribution of all considered SWaDs in terms of the CS vs. CV relationship.

440

K. KOCHANEK et al.

Fig. 2. Ratio diagram for the 2-parameter parsimonious distributions used in this study. The black dots denote the positions of SWaD.

The whole list of the D/E procedures used in the study is presented in Table 2. In addition to the parent procedure (Wa4/LMM) and parsimonious D/E procedures, the five-parameter Wakeby, i.e., Wa5/LMM and Wa5/PWM procedures were included for completeness. Altogether it makes forty D/E procedures. One can easily note that almost all two-parameter distributions are lower bounded at zero, except for the Gumbel distribution (Gu). Since the interest in FFA is in extreme flood estimation, our study is focused on upper quantile xp=1% and xp=0.1% (the results for the latter are not presented due to limitation of space for the paper), while moment characteristics were briefly discussed in Paper I. The computations were carried out for various sample sizes N = 10 (5) 50 (10) 100 (50) 500 (100) 1000; however, due to limited space the results are presented only for N = 20, 50, 100 and 1000 elements. 3. RELIABILITY OF ESTIMATION PROCEDURES

Tables in the article present values of bias and mean square error averaged out by the number of successful Monte Carlo cycles calculated by AQME. Table 3 shows the percent of successful Monte Carlo cycles, i.e., the reliability of the particular D/E procedures. All two-parameter D/E procedures give 100% reliability regardless of both the SWaD and sample size. On the other hand, if H ≠ True three-parameter D/E procedures in many cases do not give the 100% reliability even for a large sample size. When H = True, i.e., for the Wa4/LMM procedure, the performance is improved with increasing sample size, which is rather a commonsensical behaviour.

ARE PARSIMONIOUS FF MODELS … PART II

441

Tab le 2 List of D/E procedures used in the study Distribution Four-parameter Wakeby (Wa4) Two-parameter log normal (LN2)

Three-parameter log normal (LN3)

Two-parameter gamma (Pe2)

Pearson type III (Pe3)

Log-Gumbel (GEV2) Generalised Extreme Value (GEV3) Two-parameter Weibull (We2)

Three-parameter Weibull (We3)

Gumbel (Gu)

Two-parameter log logistic (GLL2)

Generalised log logistic (GLL3) Two-parameter convective diffusion (CD2) Three-parameter convective diffusion (CD3) Five-parameter Wakeby (Wa5)

Estimation Algorithm references method LMM Hosking (1991) MOM MLM LMM MOM MLM LMM MOM MLM LMM MOM MLM LMM MOM MLM LMM MOM MLM LMM MOM MLM LMM MOM MLM LMM MOM MLM LMM MOM MLM LMM MOM MLM LMM MOM MLM MOM MLM LMM PWM

Rao and Hamed (2000) pp. 96-98 Rao and Hamed (2000) p. 98 Rao and Hamed (2000) p. 99 Rao and Hamed (2000) pp. 109-110 Rao and Hamed (2000) pp. 110-111 Rao and Hamed (2000) p. 112 Rao and Hamed (2000) pp. 141-142 Rao and Hamed (2000) pp. 142-144 Rao and Hamed (2000) p. 144 Rao and Hamed (2000) pp. 156-157 Rao and Hamed (2000) pp. 157-158; Kite (1988) pp. 117-119 Rao and Hamed (2000) p. 159 Węglarczyk and Strupczewski (2002) Kochanek (2002) Węglarczyk and Strupczewski (2002) Rao and Hamed (2000) pp. 208-212 Kochanek (2002) Rao and Hamed (2000) pp. 216-218 Kochanek (2002) Kochanek (2002) Kochanek (2002) Rao and Hamed (2000) pp. 243-245 Rao and Hamed (2000) pp. 245-246 Rao and Hamed (2000) pp. 246-247 Rao and Hamed (2000) p. 232 Rao and Hamed (2000) pp. 232-234 Rao and Hamed (2000) p. 234 Węglarczyk and Strupczewski (2002) Kochanek (2002) Węglarczyk and Strupczewski (2002) Rao and Hamed (2000) pp. 304-307 Kochanek (2002) Rao and Hamed (2000) p. 311 Węglarczyk and Strupczewski (2002) Węglarczyk and Strupczewski (2002) Kochanek (2002) Kochanek (2002) Hosking (1991) Greenwood et al. (1979)

K. KOCHANEK et al.

442

However, one can note in Table 3 that the Wa4/LMM procedure for two SWaDs, namely Wa III and Wa V, does not reach 100% even for N = 1000, as it should be. This shows that the algorithm (Hosking, 1991) still needs some improvement. When H = False, the percentage of successful estimation does not always increase with sample size. Moreover, the improper choice of a model fitted to a sample is more and more noticeable by the algorithms with increasing sample. This fact can be easily noticed in Table 3 where reliability of a few three-parameter D/E procedures drops with N. The reliability of procedures depends also on SWaD, i.e., on sampling statistical characteristics of the generated series (Table 1). For instance, the reliability of GEV3/LMM procedure exceeds 80% for every sample size when generated by Wa I, whereas it does not reach even 40% for Wa IV. The fact that not every D/E procedure calculation is successful in every Monte Carlo loop undermines the comparativeness of the performance of various models. Therefore, additional calculations were carried out to evaluate as to how much average values presented in this paper differ from the ones obtained only for cases when every method (E) is successful for a given pdf (D). The results showed that the difference is rather small and cannot affect the final conclusions drawn from computations. 4. RELATIVE BIAS OF UPPER QUANTILES

The bias of an estimator of quantile xp got by a D/E procedure is defined by

(

)

(

)

B xˆ p (D E) = E xˆ p ( D E) − xp ( FP ) , S

(1)

where xp(FP) denotes the theoretical value of the quantile of probability p which is given for p = 1% in tables. S denotes the number of successful estimations. It is convenient to express bias as a ratio with respect to the quantile itself. Consequently, the relative bias (RB) is expressed as

(

)

RB xˆ p (D E) =

(

B xˆ p (D E) x p ( FP )

) × 100% .

(2)

The values of relative bias of xp=1% quantile got for five SWaDs, all D/E procedures and various sample sizes, N, are presented in Table 4. Almost all numbers displayed here are of similar magnitude pointing to a strong competition among all D/E procedures. The cases of extraordinarily large bias, indicated by shading in Table 4, originate from fitting two-parameter LN, GEV and GLL by MLM to data generated by some SWaDs. As far as GEV and GLL models are considered, their lines on CV vs. CS ratio diagram (see Fig. 2) lay far from all SWaDs dots. On the other hand, even though the position of Wa II and Wa IV SWaDs on the ratio diagrams is relatively close to LN2 line, the respective biases of this procedure are still large. This means that contrary to Kuczera’s (1982) conclusions, the application of LN2/MLM procedure is rather limited when True pdf is of Wakeby form with parameters as given in

ARE PARSIMONIOUS FF MODELS … PART II

443

444

K. KOCHANEK et al.

ARE PARSIMONIOUS FF MODELS … PART II

445

446

K. KOCHANEK et al.

Table 1. Such large MLM bias may be explained by the fact that MLM estimates are based on the main probability mass, while the Wakeby distribution function exhibits relative independence of the left-hand and right-hand tails. The results presented in Tables 4 and 5 confirm our expectations that in general the absolute values of bias got from parsimonious models should exceed the bias got from True pdf. The number of parsimonious D/E procedures that give higher bias varies with respect to SWaD and sample size. All conclusions drawn from the xˆ p=1% results remain valid for xˆ p=0.1% (results not shown). However, the tendency of increasing difference between the relative bias values for parsimonious D/E procedures and True model is observed, especially for smaller sample sizes. Therefore, for N ∈ [20, 100] the number of parsimonious D/E procedures that produce lower absolute relative bias than the Wa4/LMM diminished. The biases of xˆ p got by the True model (H = True) should tend to zero with increasing sample size (N → ∞) regardless of the estimation method. In fact, this relation is observed in Table 4 for Wa I, Wa II and Wa IV for both four- and five-parameter Wakeby distributions, while for Wa III and Wa V bias, although small, it exists even for N = 1000. In case of False model one has additionally the model bias, which is caused by the wrong distributional assumption. The model bias of upper quantiles estimates depends on the estimation method and it does not vanish in the asymptotic case (N → ∞). What is derived by the simulation experiment (Table 4) is the total of sampling and model bias. These two components can differ in sign. Such cases can be identified in Table 4 by the change of sign of the total bias with N or by finding that the total bias remains of the same sign but its absolute value grows with N. Their numbers for various SWaDs and estimation methods are listed in Table 6. Analysis of the average absolute bias with respect to both the sample size and number of parameters of models gives the possibility to draw conclusions considering the performance of particular methods of estimation. Table 7 presents the averaged over models absolute bias xˆ p=1% for Wa IV as parent. Computations do not include GEV2/MLM, because it significantly distorts the results (compare also Table 4). For two-parameter models and when two- and three-parameter parsimonious models are analyzed together, MOM gives the lowest bias of all three methods regardless of the sample size. On the other hand, analyzing three-parameter pdfs separately one finds that the bias obtained by MOM exceeds one for LMM for smaller sample sizes (N < 100) and is lower when N is large. It is observed that the average value of bias got by MOM for two- and especially for three-parameter pdfs decreases with sample size, whereas in other estimation methods it does not. As far as xˆ p=0.1% results are concerned (not shown), the values of average absolute RB are bigger than the ones for xˆ p=1% (in many cases they are about twice as much). However, the patterns observed for 0.1% quantile are even more regular than the ones in Table 7: MOM always occupies the first place regardless of the sample size and the number of model’s parameters, whereas MLM is nearly always the worst of three competing methods.

ARE PARSIMONIOUS FF MODELS … PART II

Tab le 5 Number of D/E procedures whose absolute value of bias exceeds bias for parent distribution of xˆ p =1% N= SWaD Wa I Wa II Wa III Wa IV Wa V

20

50

100

1000

34/37 37/37 35/37 29/37 28/37

34/37 36/37 37/37 3437 35/37

36/37 37/37 34/37 36/37 34/37

37/37 37/37 36/37 37/37 31/37

Tab le 6 Opposite sign of model bias and estimation bias of xˆ p =1% (number of cases) E SWaD Wa I Wa II Wa III Wa IV Wa V

MOM

MLM

LMM

Total

3/13 5/13 6/13 2/13 11/13

2/13 2/13 6/13 5/13 6/13

7/11 8/11 5/11 9/11 6/11

11/37 16/37 16/37 15/37 24/37

Tab le 7 Average absolute relative bias of xˆ p =1% for parsimonious models for Wa IV as parent (in %) Number of parameters Two

Three

Two and three

N= E MOM MLM LMM MOM MLM LMM MOM MLM LMM

20

50

100

1000

7.65 13.27 9.93 8.23 11.25 6.42 7.92 12.26 8.34

6.56 14.74 10.02 7.05 11.60 5.92 6.79 13.17 8.16

6.37 15.89 10.06 6.32 11.39 6.52 6.35 13.64 8.45

6.17 19.29 10.09 3.98 11.21 6.94 5.16 15.25 8.66

447

K. KOCHANEK et al.

448

For both xˆ p=1% and xˆ p=0.1% the maximum likelihood method gives in general the largest average RB of upper quantiles of all three methods considered. For p = 1% it is less evident but still noticeable (Table 4). The values for L-moments method lay, of course, between MOM and MLM but generally are closer to MLM than to MOM results. The numbers shown in Table 7 present averaged results for all models and sets of two- and three-parameter models for Wa IV only for selected sample sizes (N = 20, 50, 100 and 1000). However, a similar order of estimation methods can be observed also for particular two- and three-parameter models in Table 8, where the bias values for large sample only (N = 1000) and for all SWaDs are shown. The results presented in this table are especially interesting because, as it was stated earlier, they serve as a relatively good approximation of asymptotic bias. Thus, Table 8 shows, in fact, results close to model bias with respect to competing estimation methods whose position in rank is denoted by the bracketed numbers in superscript. According to this table, MOM gives generally the smallest bias (marked by superscript(1)), regardless of the number of the pdf parameters and parent distribution. There are exceptions, however, where for a particular SWaD a model places MOM in the second, or even third, position. The second place in rank is mostly occupied by LMM but one can easily spot many cases where MLM is better. A similar order of the estimation methods is for xˆ p=0.1% (table not shown). 5. ROOT MEAN SQUARE ERROR

The root mean square error (RMSE) is an overall measure of estimation accuracy. RMSE combines the bias and the variance of an estimator. It is computed from simulation data by

(

)

(

)

(

)

RMSE xˆ p (D E) = ⎡ var xˆ p (D E) + B2 xˆ p (D E) ⎤ ⎣ ⎦

{

= E ⎡⎣ xˆ p (D E) − x p ( FP) ⎤⎦ S

}

2 0.5

0.5

(3)

.

It is convenient to express it as a proportion of the quantile, i.e., as the relative RMSE:

(

)

RRMSE xˆ p (D E) =

(

RMSE xˆ p (D E) x p ( FP )

) × 100% .

(4)

The values of RRMSE for p = 1%, all SWaDs, D/E procedures and various sample sizes are displayed in Table 9. As one can see, for hydrological sample sizes, the most parsimonious models have a smaller RRMSE than these of respective specific Wakeby distribution with the L-moment method. This is even more visible for the 0.1% quantile results which are not shown. Shading in Table 9 indicates the D/E procedures which are particularly good for a given SWaD. Although the shading pattern

ARE PARSIMONIOUS FF MODELS … PART II

449

450

K. KOCHANEK et al.

ARE PARSIMONIOUS FF MODELS … PART II

451

K. KOCHANEK et al.

452

varies from one SWaD to another, certain regularity can be observed. For instance, the Limited Existence of Moments (LEM) distributions combined with some estimation methods, i.e., GEV2/MOM, GEV3/MOM, GEV3/MLM and GLL2/MOM, produce smaller RRMSE than the one based on the True model, i.e., Wa4/LMM for all SWaDs. As far as estimation methods are concerned (Table 10), MOM was found to be the best method in respect to RRMSE of xˆ p=1% and xˆ p=0.1%, regardless of the sample size (N) for a majority of two- and three-parameter models. MLM is the second in turn for N ∈ [20, 1000] and for N ≤ 50 for two- and three-parameter distributions, respectively. T ab l e 1 0 Number of the cases where a particular estimation method, E, performs the best of the three competing methods in terms of RMSE ( xˆ p =1% ) Number of parameters Two

Three

Two and three

N= E MOM MLM LMM MOM MLM LMM MOM MLM LMM

20

50

100

1000

18/35 9/35 8/35 13/30 13/30 4/30 31/65 22/65 12/65

18/35 10/35 7/35 12/30 9/30 9/30 30/65 19/65 16/65

19/35 9/35 7/35 9/30 6/30 15/30 28/65 15/65 22/65

26/35 6/35 3/35 18/30 3/30 9/30 44/65 9/65 12/65

However, the performance of MLM in respect to upper quantiles estimators was found to be a bit erratic. This method gives for some two-parameter distributions extraordinarily large values of RRMSE resulting from large bias (Table 4) as observed for LN2, GEV2 and GLL2, and, to a certain extent, for three-parameter LN. This is due to the undesirable properties of Wakeby as a parent distribution (see Paper I), coming out, in particular, when MLM is applied. Comparing the RB values of Table 4 with the RRMSE values of Table 9, one can notice that when a sample size is large (N = 1000), the RRMSE values are close to the RB values. The other alternative estimation methods, i.e., MOM and LMM, mostly give RRMSE of similar magnitude, which depends on model, sample size (N) and SWaD. It is also interesting to compare RRMSE for two-parameter parsimonious pdfs and their three-parameter counterparts. Table 11 presents the number of two-parameter D/E procedures that give lower mean square error of xˆ p=1% than the corresponding three-parameter pdf related to all procedures for selected sample sizes: N = 20, N = 30 and N = 50 which are typical for Polish hydrological logs of extreme floods. Analysis of Table 11 as well as Table 9 reveals that contrary to the bias (Table 4), two

ARE PARSIMONIOUS FF MODELS … PART II

453

-parameter models with MOM produce mostly lower RRMSE than three-parameter ones for almost all SWaDs. The only exception can be spotted for Wa V, N = 50 where only half of all two-parameter D/MOM procedure is better. Ascendancy of two-parameter models over their three-parameter counterparts slightly decreases with the increase of a sample size. The results of two other estimation methods are not so conclusive. Assuming the selected Specific Wakeby distributions to be representative of a flood regime, one may conclude from Table 11 that there is no use to employ three-parameter D/E procedures when estimating xp=1% by MOM from a sample of “hydrological” size. Obviously, a procedure built on the True model should give for large samples more accurate results than the one using a parsimonious model. Hence, it is interesting to learn the upper limit of the sample size for priority of parsimonious D/E procedures over Wa4/LMM in respect to the RMSE values. To this effect Table 12 is given. The “< 10” means that RRMSE of D/E parsimonious procedure is lower than SWaDs for the whole range of sample sizes considered, whilst “> 1000” points to the upper limit over 1000 elements. One can note that for MOM there are 52 out of 65 values greater than 50 elements, whereas for MLM there are 30 only. As far as the results of xp=0.1% are concerned (not shown), only a few Nequal values are less than 100 (including “< 10”). This means that for higher quantiles the majority of parsimonious models D/E perform better in terms of MSE, even for sample sizes greater than 100 elements. The method of L-moments usually places itself between MOM and MLM except Wa I, where Nequal for LMM is mostly the highest of all three methods, regardless of the number of the parameters of models. 6. CONCLUSIONS AND RECOMMENDATIONS

Applying the methodology described in Part I (Strupczewski et al., 2005a), the performance of various two-parameter and three-parameter parsimonious models combined with three estimation methods versus the flood parent distributions is comparatively assessed by simulation experiments. The method of moments (MOM), linear moments (LMM) and maximum likelihood (MLM) are used as alternative methods. Five bounded at zero Specific Wakeby Distributions (SWaD) whose parameters are presented in Table 1 are employed to serve as Flood Parent Distributions and forty distribution/estimation (D/E) procedures (Table 2) are included. Their performances are evaluated in respect to the estimation of upper quantiles. The relative bias (RB), relative root mean square error (RRMSE) and reliability of procedures are used for the assessment of the relative performance of alternative procedures. Three-parameter parsimonious distributions are supposed to produce lower bias of quantile estimates than their two-parameter counterparts (Cunnane, 1989). In practice, however, one deals with limited samples, hardly exceeding 50 elements, while our interest is in large quantiles being out of observation range. Working with false

454

K. KOCHANEK et al.

ARE PARSIMONIOUS FF MODELS … PART II

455

distributional assumption, a bias of such quantiles is out of control of estimation method. Cunnane (1989, p. 50) also claims, that “robustness studies indicate that quantile estimates using two-parameter distributions suffer more from bias than those based on multi-parameter ones”. Hence biases of large quantile estimates may not follow the general rule of multi-parameter pdfs’ superiority (Section 4). Moreover, one should note that the total estimation bias consists of model bias and sampling bias, which can be of opposite sign. With the sample size tending to infinity the share of sampling bias declines and the total bias approaches the model bias. It is odd that even for N = 1000 one can find a few cases where bias for three-parameter pdfs remains slightly bigger than it is for two-parameters counterparts. Apparently, the share of sampling bias may still affect the total bias value even for samples of 1000 elements. The results shown in Section 4 also present a few peculiar cases when parsimonious models (two- and three-parameter) estimated by LMM give a smaller RB than theoretically more flexible, parent procedure (Wa4/LMM). One should note that the third parameter of parsimonious models is just a location parameter, which equals zero for all SWaDs. Hence, the possibility to improve the fit of False distribution to True distribution by adding the location as a parameter is limited. Therefore, it would be better to have the second shape parameter instead of the location parameter, e.g., instead of CD with lower bound parameter to use type A Halphen distribution, which has two shape parameters (Perrault et al., 1999a, b). As far as estimating methods are concerned, the method of moments usually produces the smallest RB of upper quantiles of all competing methods. One has to bear in mind, however, that MOM should be used carefully when LEM models are applied, because for certain cases higher population moments may not exist (see also Part I). This problem does not concern LMM for which existence of the mean is sufficient. Being reliable and handy to compute, LMM becomes more and more popular in FFA. The third competing method, i.e., MLM, produces usually the highest values of relative bias. This estimation method proved to be the least sensitive to the upper part of samples, especially for two-parameter parsimonious pdfs. The peculiarity of the Wakeby distribution is the relative mutual independence of the left-hand and righthand tails (see Part I). Such a property of the parent distribution affects especially MLM, placing it in the last position of three competing methods. On the other hand, the Wakeby distribution provides a convenient analytical and a reasonable hydrologic basis for assessing the alternative D/E procedures and its structure is consistent with opinion on different distributions of small and large floods (KlemeÓ, 2000a, b). Being a five-parameter distribution, it is capable of fitting adequately to flood records. The Wakeby pdf also provides a means for representing the seemingly long, stretched upper tail structure of distributions of floods and other hydrologic phenomena, which justifies our choice of this model as the parent distribution. Contrary to bias, the standard error of three-parameter models usually exceeds those for two-parameter ones, often disqualifying three-parameter estimators. For hy-

456

K. KOCHANEK et al.

drological sample sizes one can notice that a majority of parsimonious models produce lower RRMSE than the parent distribution (Section 5). In particular, the two-parameter pdfs are usually better in terms of RRMSE than their three-parameter counterparts when estimated by means of MOM. The other estimating methods do not give such outstanding results but contrary to the RB results, MLM produces slightly smaller RRMSE than does LMM, especially for small samples. When N → ∞ the values of RRMSE of all methods tend to the relative bias (compare Table 4 and Table 9). As far as the reliability of D/E procedures is concerned, the improper choice of a model fitted to a sample may result in calculation’s failure even when sample size is large (see Section 3). There is growing belief that probability functions of extreme floods are “heavytail” distributions (e.g., Katz et al. 2002; Rossi et al. 1984). Hydrologists and statisticians are still looking for theoretical and empirical arguments substantiating the use of heavy-tail distributions. Therefore, GEV, GLL, LN (which is on the edge between pdfs having moments and heavy-tail pdfs heaving no moments) deserve here for special comments. Since the Wakeby parent distribution is also a heavy-tail pdf one could have expected a particularly good fit of other heavy-tail distribution to samples generated by SWaDs. Notwithstanding, generally none of GEV, GLL or LN models proved particularly better than other distributions with respect to both RB and RRMSE. Moreover, the especially high values of RB and RRMSE for LN2/MLM, GEV2/MLM and GLL2/MLM disqualify these D/E procedures from further competition. To sum up, one can draw a general conclusion that parsimonious flood frequency models are mostly more reliable than complex multi-parameter pdfs for hydrological samples with respect to upper quantiles. This means that two-parameter pdfs chosen by a discrimination procedure with MOM estimates are likely to be more robust in terms of RRMSE than their three-parameter counterparts. It has been proved, that MOM is the best of the three competing methods in respect to upper quantile estimates. However, it would be naive to expect to find one universal D/E procedure recommended for upper quantile estimation in FFA. Even though one had distinguished it for a given set of SWaDs, it could be doubtfully the same for other sets. A c k n o w l e d g e m e n t. This work was supported by the Polish Ministry of Science and Informatics under the Grant 2 PO4D 05729 entitled “Enhancement of statistical methods and techniques of flood events modelling”.

References Cunnane, C., 1989: “Statistical distribution for flood frequency analysis”, World Meteorological Organisation, Operational Hydrology, Report No. 33, pp. 73 + Appendices. Flood Estimation Handbook, 1999: Institute of Hydrology, Wallingford, UK.

ARE PARSIMONIOUS FF MODELS … PART II

457

Greenwood, J.A., J.M. Landwehr, N.C. Matalas and J.R. Wallis, 1979: “Probability weighted moments: definition and relation to parameters of several distributions expressible in inverse form”, Water Resour. Res. 15 (5), pp. 1049-54. Hosking, J.R.M., 1991: Fortran routines for use with the method of L-moments, Version 2, Res. Rep. RC 17079, IBM Research Division, York-town Heights, NY 10598. Katz, R.W., M.B. Parlange and P. Naveau, 2002: “Statistics of extremes in hydrology”, Adv. in Water Resour. 25, pp. 1287-1304. Kite, G.W., 1988: “Frequency and risk analysis in hydrology”, Water Research Publications, Littleton, Colorado, pp. 257. Klemeš, V., 2000a: “Tall tales about tails of hydrological distributions, I”, J. Hydrol. Eng. 5 (3), pp. 227-231. Klemeš, V., 2000b: “Tall tales about tails of hydrological distributions, II”, J. Hydrol. Eng. 5 (3), pp. 232-239. Kochanek, K., 2002: Manual of accuracy of quantiles and moments estimation (AQME), Soft Package, Internal Report of Inst. Geophys. Pol. Acad. Sc. (in Polish, unpublished). Kuczera, G., 1982: “Robust flood frequency models”, Water Resour. Res. 18 (2), 315-324. Landwehr, J.M., N.C. Matalas and J.R. Wallis, 1980: “Quantile estimation with More or Less Floodlike Distributions”, Water Resour. Res. 16 (3), pp. 547-555. Perrault, L., B. Bobée and P.F. Rasmussen, 1999a: Halphen distribution system, I: Mathematical and statistical properties, J. Hydrol. Eng.-ASCE, 4 (3), pp. 189-199. Perrault, L., B. Bobée and P.F. Rasmussen, 1999b: Halphen distribution system, II: Parameter and quantile estimation, J. Hydrol. Eng.-ASCE, 4 (3), pp. 200-208. Rao, A.R., and K.H. Hamed, 2000, Flood frequency analysis, CRC Press LLC. pp. 350. Rossi, F., M. Fiorentino and P. Versace, 1984: “Two-component extreme value distribution for flood frequency analysis”, Water Resour. Res. 20, pp. 847-56. Strupczewski, W.G., K. Kochanek, V.P. Singh and S. Węglarczyk, 2005: Are parsimonious FF models more reliable than true ones? I. Accuracy of Quantiles and Moments Estimation (AQME) – method of assessment (submitted). Węglarczyk, S., and W.G. Strupczewski, 2002: Asymptotic relative bias of mean, δ(EFX), variance, δ(varFX) and quantile, δ(xpF) resulting from improper pdf assumption F with respect to estimation method when True pdf is known, Internal Report of Institute of Geophysics, Polish Academy of Sciences (in Polish, unpublished). Received 25 May 2005 Accepted 25 July 2005