The impact of considering uncertainty in measured ...

11 downloads 36142 Views 1MB Size Report
Feb 20, 2015 - Parameter Estimation and uncertainty Analysis Tool. (IPEAT) propagates .... Local agencies including ... Colorado Watershed (Seo et al. 2014).
Stoch Environ Res Risk Assess (2015) 29:1891–1901 DOI 10.1007/s00477-015-1047-z

ORIGINAL PAPER

The impact of considering uncertainty in measured calibration/ validation data during auto-calibration of hydrologic and water quality models Haw Yen • Yamen Hoque • Robert Daren Harmel Jaehak Jeong



Published online: 20 February 2015  Springer-Verlag Berlin Heidelberg 2015

Abstract The importance of uncertainty inherent in measured calibration/validation data is frequently stated in literature, but it is not often considered in calibrating and evaluating hydrologic and water quality models. This is due to the limited amount of data available to support relevant research and the limited scientific guidance on the impact of measurement uncertainty. In this study, the impact of considering measurement uncertainty during model auto-calibration was investigated in a case study example using previously published uncertainty estimates for streamflow, sediment, and NH4-N. The results indicated that inclusion of measurement uncertainty during the autocalibration process does impact model calibration results and predictive uncertainty. The level of impact on model predictions followed the same pattern as measurement uncertainty: streamflow \ sediment \ NH4-N; however, the direction of that impact (increasing or decreasing) was not consistent. In addition, inclusion rate and spread results did not indicate a clear relationship between predictive uncertainty and the magnitude of measurement uncertainty. The purpose of this study was not to show that inclusion of measurement uncertainty produces better calibration results or parameter estimation. Rather, this study demonstrated that uncertainty in measured calibration/validation data can play a crucial role in parameter estimation during

USDA is an equal opportunity employer and provider. H. Yen (&)  Y. Hoque  J. Jeong Blackland Research and Extension Center, Texas A&M Agrilife Research, 720 East Blackland Road, Temple, TX 76502, USA e-mail: [email protected]; [email protected] H. Yen  R. D. Harmel Grassland, Soil & Water Research Laboratory, USDA-ARS, 808 East Blackland Road Temple, Temple, TX 76502, USA

auto-calibration and that this important source of predictive uncertainty should be not be ignored as it is in typical model applications. Future modeling applications related to watershed management or scenario analysis should consider the potential impact of uncertainty in measured calibration/validation data, as model predictions influence decision-making, policy formulation, and regulatory action. Keywords Measurement uncertainty  Model calibration  SWAT  Uncertainty analysis

1 Introduction Simulation models such as the Soil & Water Assessment Tool (SWAT, Arnold et al. 1993), Agricultural Policy/ Environmental Extender Tool (APEX, Williams et al. 2012), Hydrological Simulation Program-Fortran (HSPF, Bicknell et al. 1997) and others are useful tools that further our understanding of watershed processes and assist in devising watershed management plans. Improvements in such models have expanded their effectiveness in properly simulating what is happening in the actual watershed being studied. However, such improvements have led to substantial increases in the number of model parameters used to govern the functions representing hydrological processes (Yang et al. 2008; Bai et al. 2009). This means that there are more parameters to calibrate and validate before a model output can be deemed satisfactory. Fortunately, issues related to high-dimensional and/or computationally intensive model calibration/validation procedures have been mostly resolved thanks to recent advances in the development of various auto-calibration algorithms (Duan et al. 1992;

123

1892

Tolson and Shoemaker 2007; Vrugt et al. 2009; Yen 2012; Yen et al. 2014b). A shortcoming of traditional calibration processes is the incorrect assumption that model uncertainty is attributed only from parameter uncertainty (Ajami et al. 2007), along with the common disregard for uncertainty analysis in hydrologic research (Balin et al. 2010). Additional sources of uncertainty in hydrologic model output are related to model structure and parameters as well as measured input data (e.g., climatic) and measured data used for calibration/ validation (e.g., flow, water quality) (Ajami et al. 2007; Salamon and Feyen 2009; Balin et al. 2010; Yen et al. 2014a; Harmel et al. 2014). Ignoring one or more sources of uncertainty may cause final results of model calibration to be biased, consequently leading to incorrect conclusions (McMillan et al. 2011). Because of the importance of each source of model uncertainty, Yen et al. (2014a) developed a comprehensive framework that incorporates all major sources of uncertainty. This framework called Integrated Parameter Estimation and uncertainty Analysis Tool (IPEAT) propagates uncertainty sources from parameter, model structure and measurement (both input and calibration/validation data) uncertainty. The influence of uncertainty in measured input data particularly rainfall data has been emphasized as an important source of uncertainty in hydrological and water quality models in a number of studies (Balin et al. 2010; McMillan et al. 2011), but other types of input data such air temperature, hydraulic conductivity, etc. have received less attention. In addition, measurement uncertainty attributed from measured input data has also been explicitly incorporated into models by implementing latent variables on precipitation data while conducting calibration (Kavetski et al. 2002; Ajami et al. 2007; Yen et al. 2014a, c). However, the impact of uncertainty in measured input data is not the focus of this paper. Details can be found in Yen et al. (2014c). The effect of uncertainty in measured calibration/validation data (measured calibration/validation data represents data are used to conduct calculation of error statistics to ensure the performance of model predictions are as close to real world processes as possible) on model calibration has not been previously researched in general, even though it is an important error source (Harmel and Smith 2007; Harmel et al. 2010). It is often difficult to obtain sufficient data to calculate uncertainty for measured calibration/ validation data; however, Harmel et al. (2006, 2009) provide estimates for use when project-specific uncertainty data are not available. Similarly, techniques were recently developed to adjust model evaluation goodness-of-fit indicators to reflect measurement and/or model uncertainty (Harmel and Smith 2007; Harmel et al. 2010). In a previous study (Yen et al. 2014a), IPEAT was developed as a

123

Stoch Environ Res Risk Assess (2015) 29:1891–1901

sophisticated framework and tool to explore interactions among various sources of uncertainty and the corresponding impact on model predictions. Scientists are able to take advantage of IPEAT for detailed investigation of the specific sources of uncertainty (e.g., input uncertainty of precipitation was further examined in Yen et al. 2014c). However, the associated influence of the uncertainty in measured calibration/validation data on simulation results has not been previously explored. Since the impact of this important source of uncertainty has not been previously evaluated, the goal of this study was to evaluate the impact of uncertainty in measured calibration and validation data on watershed model auto-calibration. The specific objective was to quantify the impact on calibration results and predictive uncertainty of considering various levels of measured data uncertainty in SWAT auto-calibration for streamflow and water quality.

2 Materials and methods 2.1 SWAT model The SWAT model is a physically-based, spatially distributed watershed scale simulation model. It was developed by the United States Department of Agriculture— Agricultural Research Service (USDA-ARS) to evaluate the impact of land management and climate change on water quantity and quality (Gassman et al. 2007; Arnold et al. 2012). Major components of the model include hydrology, weather, erosion, soil temperature, crop growth, nutrients, pesticides and agricultural management. SWAT has the ability to predict changes in hydrology, sediment, nutrient, pesticides, and bacteria loading from different management conditions in large ungauged basins (Lin et al. 2013; Zhenyao et al. 2013). It is a continuous time model that operates on a daily time-step and can be used for longterm simulations. Model output is available at daily, monthly, and annual time-scales. SWAT has been successfully applied to numerous studies related to investigating water quality issues (sediments, nutrients, and/or pesticides) in watersheds, including but not limited Ng et al. (2010), and Hoque et al. (2014). 2.2 Study area The Arroyo Colorado watershed (ACW) is located in the Texas, along the border of USA and Mexico (1,692 km2). ACW is a sub-watershed of the Nueces-Rio Grande Coastal Basin located in the Lower Rio Grande Valley of South Texas. It extends eastward from near Mission, Texas, to the Laguna Madre (Fig. 1). It is created by the Arroyo

Stoch Environ Res Risk Assess (2015) 29:1891–1901

1893

Fig. 1 Location of the Arroyo Colorado Watershed (Seo et al. 2014)

Colorado River, a tributary of the Rio Grande River that flows through the Hidalgo, Cameron, and Willacy counties into the Laguna Madre. Dominated by agricultural lands, the ACW is irrigated via a network of canals, ditches, and pipes with water from the Rio Grande River. The watershed is extensively urbanized along the main stem of the Arroyo Colorado River, particularly in the western and central parts of the basin including the cities of Mission, McAllen, Pharr, Donna, Weslaco, Mercedes, Harlingen, and San Benito. The land-use/land-cover distribution in ACW is agriculture (54 %), range land (18.5 %), and urban areas (12.5 %). The major cultivated crops include grain sorghum, cotton, sugar cane, and citrus as well as some vegetables and fruit. Major soil series within the watershed include Harlingen, Hidalgo, Mercedes, Raymondville, Rio Grande, and Willacy (USDA-SCS 1972). Soils in the watershed are mostly clays, clay loams, and sandy loams with soil depths ranging between 1600 and 2000 mm. The watershed is characterized by a semi-arid climate with annual rainfall ranging from about 530 to 680 mm, generally from west to east and average annual temperature of 22.7 C with mean monthly temperatures ranging from 14.5 C in January to 28.9 C in July. 2.3 Input data Input data for the SWAT model of ACW include: •

30 m resolution Digital Elevation Model (DEM) downloaded from the National Elevation Dataset (NED) maintained by the U.S. Geological Survey (USGS,





accessible online at http://ned.usgs.gov/; last accessed on September 18, 2013). Land-use map created from remote sensing data and field surveys to represent land-cover conditions for 2004–2007. County-scale soil properties data acquired from the Soil Survey Geographic (SSURGO) database of USDA National Resources Conservation Service (NRCS) (accessible online at http://websoilsurvey.sc.egov.usda.gov/ App/WebSoilSurvey.aspx; last accessed on September 18, 2013).

In the SWAT model for this application, ACW was divided into 17 subbasins and further divided into 475 Hydrological Response Units (HRUs) based on land-use, soil, and slope combinations. Local agencies including Texas AgriLife Research, Texas Commission of Environmental Quality (TCEQ), Texas State Soil and Water Conservation Board (TSSWCB), and field offices of NRCS and Soil and Water Conservation Districts (SWCD) provided information on crops and management practices (e.g., tillage practices, irrigation management, and nutrient application rate and timing). In addition, TSSWCB provided current and historical Best Management Practice data (e.g., land leveling, irrigation management, nutrient management methods) for the period of 1999–2006. Daily weather data in the form of precipitation and min/max air temperature collected at three stations (COOPID 419588 near Weslaco, COOPID 415836 near Mercedes, COOPID 413943 near Harlingen, see Fig. 1) over four years (2000–2003) were used. These data were obtained from the Texas State Climatologist Office located

123

1894

Stoch Environ Res Risk Assess (2015) 29:1891–1901

at Texas A&M University at College Station. Daily streamflow data for two stations, one near Llano Grande at FM 1015 south of Weslaco and the other near US 77 in South West Harlingen, were provided by the International Boundary and Water Commission. Additionally, there are 21 permitted point sources discharges located within the ACW (16 municipal, three industrial, and two shrimp farms). The discharge permit limits of the municipal plants range from 0.4 to 10 million gallons per day. The shrimp farms discharge infrequently (Rains and Miranda 2002). Water quality data from grab samples were obtained for suspended sediment and ammonium nitrogen (NH4-N). Because only periodic grab samples were available, instead of daily or continuous data, we used the LOAD ESTimator (LOADEST) software developed by USGS (Runkel et al. 2004) to estimate daily values from which to characterize water quality trends at the monitoring station. From the daily values, monthly sediment yield and ammonia data were estimated for model calibration. 2.4 Incorporation of measurement uncertainty In this study, the impact of measurement uncertainty on auto-calibration was investigated with arbitrary data quality scenarios ranging from worst to best case from previous research work. Harmel et al. (2006) estimated uncertainty in measured streamflow, sediment, and NH4-N for each of these scenarios (Table 1). The probability distribution method of Harmel and Smith (2007) was adopted to perform calculations involving measurement uncertainty. This method assigns a correction factor to the error value for each observed-predicted data pair as shown in Eq. (1). Ek ¼

CFk obs ðQ  Qsim k Þ 0:5 k

ð1Þ

where, Ek is adjusted error between the observed (Qobs k ) and simulated data (Qsim k ) at time k; and CFk is the correction factor at time k. CFk is divided by 0.5 because that is the maximum probability of one half of the PDF (Probability Density Function), thus ensuring that the maximum value for CFk is 0.5 as well. It is assumed that the measured data are normally distributed. A normal cumulative distribution function can be Table 1 Scenarios and associated probable error ranges (PER) from Harmel et al. (2006)

Disregarding measuring implies that the uncertainty is 0

123

where, r2 is the variance of Qobs k ; l is the mean which contains a certain amount of the normal probability distribution (e.g. l = ±3.9 represents standard deviation which includes [99.99 % of a normal probability distribution); UBlower and UBupper are the upper and lower k k boundaries of every measured data point at time k which can be calculated as follows (Eqs. 3, 4): UBupper ¼ Qobs k þ k

PERk  Qobs k 100

ð3Þ

UBlower ¼ Qobs k k 

PERk  Qobs k 100

ð4Þ

where, PERk is the probable error range (PER) as reported by Harmel et al. (2006) at time k. Please refer to Harmel and Smith (2007) for more details on error term modification and to Yen et al. (2014a) for implementation details. 2.5 Description of Auto-Calibration Scenarios A total of six scenarios were evaluated in this study (summarized in Table 1). Scenario 01 is the baseline calibration case with no measurement data uncertainty incorporated, which represents typical model applications. Scenarios 02–06 were adapted from the data quality scenarios in Harmel et al. (2006). Scenario 02 represents the best case data quality scenario with concentrated quality assurance/quality control (QA/QC) unconstrained by financial and personnel resource limitations and in ideal hydrologic conditions. Scenarios 03–05 represent data collected with typical QA/QC and under typical hydrologic conditions producing average values and maximum and minimum uncertainty boundaries to represent a more reasonable ‘‘typical’’ range of uncertainty. Finally, scenario 06 represents the worst case for

Sediment (± %)

NH4-N (± %)

Scenario

Streamflow (± %)

No measurement uncertainty assumed (01)

0a

0

Best case (02)

3

3

3

Typical scenario minimum (03)

6

7

11

10

18

31

Typical scenario average (04) a

calculated with known mean and variance. The variance can be computed as follows (Eq. 2): 8  lower 2 > Qobs > k  UBk > < l ð2Þ r2 ¼  upper 2 > UBk  Qobs > k > : l

0

Typical scenario maximum (05)

19

53

100

Worst case scenario (06)

42

117

246

Stoch Environ Res Risk Assess (2015) 29:1891–1901

projects conducted with minimal attention to QA/QC, with limited financial and personnel resources, and in difficult hydrologic conditions. 2.6 Model calibration The SWAT model for the Arroyo Colorado watershed was calibrated from 2002 to 2003. Streamflow was calibrated at a daily time-step while water quality (sediment and NH4N) were calibrated at a monthly time-step. The calibration scheme utilized was the auto-calibration algorithm Dynamically Dimensioned Search (DDS, Tolson and Shoemaker 2007). The available observed data record was too short to be split into two separate periods for calibration and validation. However, this study focused on evaluating the impact on calibration results by incorporating measurement uncertainty not on trying to match the model output rigorously to observed flow and water quality. Hence, the model was not validated for a different time period postcalibration. The outlet of the ACW is close to the Gulf of Mexico; therefore, flow measured near the outlet was impacted by the diurnal fluctuations of tidal waves. To avoid this tidal effect, the SWAT model for ACW was calibrated using observed data from the gauge station located near Llano Grande at FM 1015 south of Weslaco. Daily streamflow data were available from 2002 to 2003, and water quality data for sediment and NH4-N were limited to several grab samples. LOADEST (applying the Maximum Likelihood Estimation, MLE, method) was used to generate monthly data from the grab samples which were then used for calibration. A total of 31 SWAT parameters for related processes were selected in all case scenarios for calibrating flow and water quality. The parameters and their recommended ranges are listed in Appendix 1. Calibration performance using DDS was evaluated with the Nash–Sutcliffe coefficient of efficiency (NSE) (Nash and Sutcliffe 1970) and the modified objective function as shown in Eqs. 5 and 6. NSE normalizes the residual of error between observed and simulated against the mean observation and is one of the most commonly used statistical measures to evaluate model performance (Servat and Dezetter 1991; ASCE 1993). 2 PN  Obs y  ySim i NSE ¼ 1  PNi¼1 i ð5Þ Obs  yMean Þ2 i i¼1 ðyi XV ð1  NSEv Þ ð6Þ OF ¼ v¼1 Where yObs is the observed response at time step i; ySim is i i Mean the simulated response at time step i; yi is the mean of observed response at time step i; and N is the total number of time steps. NSE equal to one indicates a perfect match

1895

between observation and simulation, a negative or small value of NSE indicates poor performance. Therefore, during the auto-calibration process, the ideal global optimal solution for the OF is defined in such a way that it tends to be minimized to zero to get a perfect match (i.e., NSE = 1). The objective function is calculated as the sum of 1-NSE for the output variables, here OF is the final objective function value; NSEv is the NSE value for output variable v; and V is the total number of output variables. In this case study, output variables were the constituents that were calibrated (streamflow, sediment, and NH4-N). Additional detail related to this modified objective function using NSE can be found in Seo et al. (2014). To evaluate the impact of measurement uncertainty on predictive uncertainty, inclusion rate and spread were calculated as described in Yen et al. (2014a). Inclusion rate is the percentage of observed data points located within the 95 % confidence interval of the predicted outputs. Spread is the average width of corresponding uncertainty band around the predicted time-series.

3 Results and discussion 3.1 Overall impacts The overall performance of objective function values versus model iterations for all scenarios is shown in Fig. 2. All six scenarios achieved convergence before the maximum model run limit (5,000 iterations). The exact iteration at which this occurred varied; however, the patterns of convergence are similar among different scenarios (no major improvement after 2,500 runs through visual inspection). As shown in Fig. 2, the converged optimized OF values for the six scenarios ranged from 0.986 to 1.386. For the scenarios that included measurement uncertainty, the OF values decreased as the measurement uncertainty increased, which makes intuitive sense; however, the OF value for Scenario 01 in which measurement uncertainty is implied to be ±0.0 % did not follow this pattern. To explore this result, the model prediction results were studied in further detail. 3.2 Impacts on model goodness-of-fit The level of impact on model predictions followed the same pattern as measurement uncertainty: streamflow \ sediment \ NH4-N. As shown in Table 2 and Fig. 3, NSE for stream flow predictions changed little for the six scenarios only decreasing from 0.61 to 0.57 (modified NSE from Harmel and Smith 2007). However, more noticeable variability was observed for sediment and

123

1896

Stoch Environ Res Risk Assess (2015) 29:1891–1901

Fig. 2 Optimized objective function values and model convergence iterations for the six scenarios (e.g., the final converged objective function value for Scenario 01 is 1.195)

Scenario 01

Scenario 02

Scenario 03

Scenario 04

Scenario 05

Scenario 06

2.5

2.3

Objective Function Value

2.1

1.9

1.7

1.5

Scenario 02 : 1.386 Scenario 03 : 1.385 Scenario 04 : 1.317

1.3 Scenario 01 : 1.195 Scenario 05 : 1.125

1.1 Scenario 06 : 0.986

0.9

0

1000

2000

3000

4000

5000

Model Iteration

Table 2 NSE values for each output variable corresponding to the optimized ‘‘auto-calibrated’’ results of six scenarios calculated modified (Harmel and Smith 2007) and as traditionally calculated (Nash and Sutcliffe 1970) Scenario

Streamflow

NH4-N

Sediment

NSE Modified

NSE Traditional

NSE Modified

NSE Traditional

NSE Modified

NSE Traditional

01

0.61

0.61

0.74

0.74

0.64

0.64

02

0.58

0.58

0.64

0.64

0.60

0.60

03

0.58

0.58

0.64

0.64

0.60

0.60

04

0.58

0.58

0.67

0.67

0.64

0.64

05

0.57

0.57

0.78

0.76

0.74

0.61

06

0.57

0.56

0.77

0.67

0.89

0.56

NH4-N predictions. The modified NSE for sediment was 0.74 in Scenario 01 then dropped to 0.64 in Scenarios 02 and 03 but increased to 0.67 to 0.76 in Scenarios 04, 05, and 06. For NH4-N, the modified NSE was 0.64 for Scenario 1, 0.60 for Scenarios 02 and 03, and increased to 0.64 (Scenario 04), 0.74 (Scenario 05), and 0.89 (Scenario 06). At first glance, the lack of a consistent increase in NSE corresponding to the increases in measurement uncertainty for each output variable might be troubling; however, this

123

result was expected because measurement uncertainty was explicitly incorporated during the auto-calibration process. In addition, the Harmel and Smith (2007) method was used to modify the NSE value based on measurement uncertainty in post-calibration model performance evaluation. During auto-calibration, automated techniques generate new proposed solutions (parameter set values) based on the current best (or relatively better) objective function results. Thus, whether measurement uncertainty is included or

Stoch Environ Res Risk Assess (2015) 29:1891–1901 Streamflow

Sediment

1897

Ammonia

Streamflow

1.0

Correction Factor

0.9

NSE

0.8 0.7 0.6 0.5 0.4

1

2

3

4

5

6

Fig. 3 NSE values for each output variable corresponding to the optimized ‘‘auto-calibrated’’ results of six scenarios

Streamflow *

2

3

Sediment

4

Ammonia

5

6

Case Scenario

Case Scenario

(A)

1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0

Fig. 5 Averaged correction factor values corresponding to the optimized ‘‘auto-calibrated’’ results of Scenario 02–06 (measurement uncertainty in measured calibration/validation data was not applied on Scenario 01)

Streamflow **

1.0

NSE

0.9 0.8 0.7 0.6 0.5

1

2

3

4

5

6

5

6

5

6

Case Scenario

(B)

Sediment *

Sediment **

1.0

NSE

0.9 0.8 0.7 0.6 0.5

1

2

3

4

Case Scenario

(C)

Ammonia *

Ammonia **

1.0

NSE

0.9 0.8 0.7 0.6 0.5

1

2

3

4

Case Scenario

Fig. 4 NSE values for each output variable corresponding to the optimized ‘‘auto-calibrated’’ results of six scenarios calculated based on the Harmel and Smith (2007) modification (*) and the traditional Nash and Sutcliffe (1970) calculation (**)

disregarded, auto-calibration can produce optimized parameter sets with better performance in terms of objective function values. To further examine the impact of incorporating measurement uncertainty in auto-calibration, model calibration

for the six scenarios was also carried out using the traditional NSE calculation (Table 2; Figs. 4a–c). Figure 4a shows that for streamflow, the incorporation of measurement uncertainty had negligible impact on NSE values likely because uncertainty estimates were relatively low (±3 to ±42 %). In contrast, as measurement uncertainty increased in Scenarios 05 and 06 to ±53 to ±177 % for sediment and to ±100 to ±246 % (Table 1), the divergence in the modified and traditionally calculated NSE values was noticeable. The effect of measurement uncertainty on goodness-of-fit indictor values may warrant additional study in the context of recent model performance evaluation guidelines (e.g., Moriasi et al. 2007; Harmel et al. 2014). The average correction factor values for optimized predicted streamflow, sediment, and NH4-N values are shown in Fig. 5. Average values are presented instead of all correction factors to aid in visualization. From the figure, it can be seen that values of correction factors range from 0.78 to 0.98 for streamflow, 0.67 to 0.99 for sediment, and 0.37 to 0.99 for NH4-N (correction factors close to 1.0 indicate little adjustment). It is clear that the error term (difference observed and predicted data) adjustment is less when scenarios with relatively low level of measurement uncertainty are involved (Scenarios 02, 03, and 04). Similar trends can also be seen in Figs. 3, 4a–b where streamflow data was evidently less affected by measurement uncertainty levels while NH4-N was most affected. The impact of incorporating measurement uncertainty in auto-calibration, especially as uncertainty increases, is also evident in Figs. 6a–c. These figures present the predicted time series (daily streamflow and monthly sediment, NH4-N) for all six scenarios for the optimized parameter set. When the uncertainty is larger, it has a more discernible impact on model performance. This highlights the importance of incorporating measurement uncertainty into model auto-calibration, especially in cases of high measurement uncertainty.

123

1898 Fig. 6 Best results in streamflow, sediment, and NH4N predictions of all scenarios: a Streamflow; b Sediment; c NH4-N

Stoch Environ Res Risk Assess (2015) 29:1891–1901

(A)

Observation

Scenario 01

Scenario 02

Scenario 04

Scenario 05

Scenario 06

Scenario 03

70 60

Flow Rate (m3/s)

50 40 30 20 10 0

0

100

200

300

400

500

600

700

Day

(B)

Observation

Scenario 01

Scenario 02

Scenario 04

Scenario 05

Scenario 06

Scenario 03

Sediment Load (Ton/Ha)

0.18 0.16 0.14 0.12 0.10 0.08 0.06 0.04 0.02 0.00

0

2

4

6

8

10

12

14

16

18

20

22

24

22

24

Month

NH4-N Load (Kg/Ha)

(C) 0.20 0.18 0.16 0.14 0.12 0.10 0.08 0.06 0.04 0.02 0.00

0

2

Observation

Scenario 01

Scenario 02

Scenario 04

Scenario 05

Scenario 06

4

6

8

10

12

14

16

Scenario 03

18

20

Month

It is important to emphasize that model predictions are not necessarily better when including measurement uncertainty in the auto-calibration process; thus, the justification to include measurement uncertainty in autocalibration is not to increase NSE values for instance but to ensure that this important source of predictive uncertainty is considered in the process. All goodness-of-fit indicators

123

are calculated with measured data with that have some level of inherent uncertainty (Harmel et al. 2006; Harmel and Smith 2007; Yen et al. 2014a); however, the common assumption that measured data are without uncertainty— which is in fact implied when measurement uncertainty is disregarded—results from practical concerns and the additional effort required to make reasonable uncertainty

Stoch Environ Res Risk Assess (2015) 29:1891–1901

1899

Table 3 Inclusion rate of observed streamflow, sediment, and NH4-N within the 95 % confidence interval and the corresponding spread for the simulation period (2002–2003) Scenario

Inclusion ratea

Spreadb

Streamflow (%)

Sediment (%)

NH4-N (%)

Streamflow (m3/s)

Sediment (t/ha)

NH4-N (kg/ha)

01

49.59

62.50

91.67

1.815

0.044

0.147

02

31.92

66.67

75.00

1.662

0.051

0.144

03 04

31.92 34.11

66.67 32.50

75.00 75.00

1.662 1.743

0.051 0.049

0.144 0.136

05

49.86

70.83

75.00

1.846

0.052

0.139

06

33.70

58.33

83.33

1.387

0.048

0.145

a

Inclusion rate (%) percentage of observed data points located within the 95 % confidence interval

b

Spread average width of the corresponding uncertainty band along the predicted time series

estimates. In this study, we demonstrated the potential impact on model predictions when uncertainty in measured calibration/validation is explicitly considered in autocalibration, which further emphasizes the importance of measured data sets with higher quality. 3.3 Impact on predictive uncertainty Inclusion rate and spread results for each scenario is presented in Table 3. For streamflow, no clear pattern was evident in inclusion rate or spread values. For sediment and NH4-N, spread values varied little between scenarios, and similar to streamflow that no clear pattern was evident in the inclusion rate. The use of inclusion rate and spread for uncertainty analysis was not to distinguish the superiority of any specific scenario but to identify the character and behavior of the calibrated models. The variation in predictive uncertainty as expressed by inclusion rate and spread further demonstrated that incorporation of measurement uncertainty does not guarantee a fixed trend of calibration behavior (i.e., higher inclusion rate or wider spread with increasing uncertainty in measured calibration/validation data). As mentioned previously, the incorporation of measurement uncertainty during auto-calibration may alter derived solutions. Results of uncertainty analysis are not predictable and may also be case specific. Therefore, in cases in which measurement uncertainty is not disregarded, the impact on flow and water quality predictions should be carefully investigated relative to the quality of the measured data set.

4 Conclusion In this study, measurement uncertainty of measured calibration/validation data was incorporated explicitly into the

model auto-calibration process. Specifically, the potential impact of various data quality scenarios on model predictions was evaluated. The inclusion of uncertainty in measured calibration/validation data, especially high uncertainty, increased the probability that prediction results for sediment and NH4-N were considered ‘‘better’’ based on modified NSE values. Higher levels of measurement uncertainty did have larger impacts on model predictions, but there was no clear trend related to this impact. In addition, inclusion rate and spread results did not indicate a clear tendency whether the overall predictive uncertainty will increase or decrease when measurement uncertainty is included. The purpose of this study was not to show that inclusion of measurement uncertainty guarantees better calibration or parameter estimation. Rather, this study demonstrated that the uncertainty in measured calibration/validation data plays a crucial role in parameter estimation especially in the auto-calibration process. It has been shown that intrawatershed responses are be altered by including multiple uncertainty sources (Yen et al. 2014a). In addition, the measured input uncertainty (e.g., precipitation) (Yen et al. 2014c) and structural uncertainty (Yen et al. 2014d) can produce minor to moderate impact on model performance. For uncertainty in measured calibration/validation data, the only current measurement uncertainty guidelines (in terms of probable error ranges by Harmel et al. (2006)) indicated substantial ranges among output variables (see Table 1). This study demonstrates that model predictions may be affected substantially depending on the level of uncertainty associated with measured calibration/validation data. It is hoped that future modeling applications (both within and outside the field of hydrologic and water quality modeling) will take this important source of uncertainty into account. Addressing all sources of model uncertainty is increasingly important as models are increasingly used in management, regulatory, and policy decision-making.

123

1900

Stoch Environ Res Risk Assess (2015) 29:1891–1901

Acknowledgments This study was supported by the funding from the United States Department of Agriculture—Natural Resources Conservation Service (USDA-NRCS) Conservation Effects Assessment Project (CEAP)—Wildlife and Cropland components. Awesome comments provided by reviewers and editorial board greatly improved the quality of the manuscript. Also USDA is an equal opportunity employer and provider.

Appendix 1 See Table 4.

Table 4 Calibration parameters for all case scenarios Parameters

Input file

Units

Range

Description

ADJ_PKR

.bsn



0.5–2

Peak rate adjustment factor for sediment routing in the subbasin (tributary channels)

CMN

.bsn



0.001–0.003

Rate factor for humus mineralization of active organic nitrogen

EPCO

.bsn



0–1

Plant uptake compensation factor

NPERCO

.bsn



0–1

Nitrogen percolation coefficient

PRF

.bsn



0–2

Peak rate adjustment factor for sediment routing in the main channel

SPCON

.bsn



0.0001–0.01

Linear parameter for calculating the maximum amount of sediment that can be re-entrained during channel sediment routing

SPEXP

.bsn



1–1.5

Exponent parameter for calculating sediment re-entrained in channel sediment routing

SURLAG

.bsn

Day

1–24

Surface runoff lag time

SOL_NO3

.chm

mg/kg

0–100

Initial NO3 concentration in the soil layer

ALPHA_BF

.gw

1/Day

0–1

Baseflow alpha factor

GW_DELAY

.gw

Day

0–500

Groundwater delay

GW_REVAP

.gw



0.02–0.2

Groundwater ‘‘revap’’ coefficient

GWQMN

.gw

mm H2O

0–5000

Threshold depth of water in the shallow aquifer required for return flow to occur

ESCO SLSUBBSN

.hru .hru

– M

0–1 10–150

Soil evaporation compensation factor Average slope length

CN_F

.mgt

%

±10

Initial SCS CN II value

USLE_P

.mgt



0–1

USLE equation support practice factor

CH_COV2

.rte



-0.001–1

Channel cover factor

CH_K2

.rte

mm/hr

-0.01–500

Effective hydraulic conductivity in main channel alluvium

CH_N2

.rte



-0.01–0.3

Manning’s ‘‘n’’ value for the main channel

SOL_AWC

.sol

%

±10

Available water capacity of the soil layer

SOL_K

.sol

%

±10

Saturated hydraulic conductivity

USLE_K

.sol

%

±10

USLE equation soil erodibility (K) factor

CH_K1

.sub

mm/hr

0–300

Effective hydraulic conductivity in tributary channel alluvium

CH_N1

.sub



0.01–30

Manning’s ‘‘n’’ value for the tributary channels

BC1

.swq

1/day

0.1–1

Rate constant for biological oxidation of NH4 to NO2 in the reach at 20 C

BC2

.swq

1/day

0.2–2

Rate constant for biological oxidation of NO2 to NO3 in the reach at 20 C

BC3

.swq

1/day

0.2–0.4

Rate constant for hydrolysis of organic N to NH4 in the reach at 20 C

RS3

.swq

mg/m2day

0–1

Benthic source rate for NH4-N in the reach at 20 C

RS4

.swq

1/day

0.001–0.1

Rate coefficient for organic N settling in the reach at 20 C

USLE_C

crop.dat

%

±10

Min value of USLE C factor applicable to the land cover/plant

Parameter values for CN_F, SOL_AWC, SOL_K, USLE_K, and USLE_C are the changes of fraction from default values

123

Stoch Environ Res Risk Assess (2015) 29:1891–1901

References Ajami NK, Duan Q, Sorooshian S (2007) An integrated hydrologic Bayesian multimodel combination framework: confronting input, parameter, and model structural uncertainty in hydrologic prediction. Water Resour Res 43(1):1–19 Arnold JG, Allen PM, Bernhardt G (1993) A comprehensive surfacegroundwater flow model. J Hydrol 142:47–69 Arnold JG, Kiniry JR, Srinivasan R, Williams JR, Haney EB, Neitsch SL (2012) Soil and Water Assessment Tool Input/Output Documentation 2012. Texas Water Resources Institute Technical Report No. 439, Texas A&M University System Bai Y, Wagener T, Reed P (2009) A top-down framework for watershed model evaluation and selection under uncertainty. Environ Model Softw 24(8):901–916 Balin D, Lee H, Rode M (2010) Is point uncertain rainfall likely to have a great impact on distributed complex hydrological modeling. Water Resour Res, 46: doi: 10.1029/2009WR007848. issn: 0043–1397 Bicknell BR, Imhoff JC, Kittle Jr. JL, Donigian AS, Johanson RC (1997) Hydrological Simulation Program–Fortran: User’s manual for version 11. US. Environmental Protection Agency, National Exposure Research Laboratory, Athens, GA, EPA/600/ R-97/080. 755 p Duan Q, Sorooshian S, Gupta V (1992) Effective and efficient global optimization for conceptual rainfall-runoff models. Water Resour Res 28(4):1015–1031 Gassman PW, Reyes MR, Green CH, Arnold JG (2007) The soil and water assessment tool: historical development, applications and future research directions. Trans Am Soc Agric Biol Eng 50(4):1211–1250 Harmel RD, Smith P (2007) Consideration of measurement uncertainty in the evaluation of goodness-of-fit in hydrologic and water quality modeling. J Hydrol 337(3–4):326–336 Harmel RD, Cooper RJ, Slade RM, Haney RL, Arnold JG (2006) Cumulative uncertainty in measured streamflow and water quality data for small watersheds. Trans ASABE 49(3):689–701 Harmel RD, Smith DR, King KW, Slade RM (2009) Estimating storm discharge and water quality data uncertainty: a software tool for monitoring and modeling applications. Environ Model Softw 24(7):832–842 Harmel RD, Smith PK, Migliaccio KW (2010) Modifying goodnessof-fit indicators to incorporate both measurement and model uncertainty in model calibration and validation. Trans ASABE 53(1):55–63 Harmel RD, Smith PK, Migliaccio KL, Chaubey I, Douglas-Mankin K, Benham B, Shukla S, Mun˜oz-Carpena R, Robson BJ (2014) Evaluating, interpreting, and communicating performance of hydrologic/water quality models considering intended use: a review and recommendations. Environ Model Softw 57:40–51. doi:10.1016/j.envsoft.2014.02.013 Hoque YM, Cibin R, Hantush MM, Chaubey I, Govindaraju RS (2014) How do land use and climate change affect watershed health? A sceanrio-based analysis. Water Qual Expo Health 6(1–2):19–33 Kavetski D, Franks SW, Kuczera G (2002) Confronting input uncertainty in environmental modeling. Water Sci Appl 6:49–68 Lin S, Jing C, Coles NA, Chaplot V, Moore NJ, Wu J (2013) Evaluating DEM source and resolution uncertainties in the soil and water assessment tool. Stoch Environ Res Risk Assess 27(1):209–221. doi:10.1007/s00477-012-0577-x McMillan H, Jackson B, Clark M, Kavetski D, Woods R (2011) Input uncertainty in hydrological models: an evaluation of error models for rainfall. J Hydrol 400(1–2):83–94 Moriasi DN, Arnold JG, Liew MWV, Bingner RL, Harmel RD, Veith TL (2007) Model evaluation guidelines for systematic quantification of accuracy in watershed simulations. Trans ASABE 50(3):885–900

1901 Nash JE, Sutcliffe JV (1970) River flow forecasting through conceptual models: Part I A discussion of principles. J Hydrol 10(3):282–290 Ng TL, Eheart JW, Cai X, Miguez F (2010) Modeling miscanthus in the soil and water assessment tool (SWAT) to simulate its water quality effects as a bioenergy crop. Environ Sci Technol 44(18):7138–7144 Rains TH, Miranda RM (2002) Simulation of flow and water quality of the Arroyo Colorado, Texas, 1989–1999, United States Geological Survey-Water Resources Investigations Report, No: 02-4110 Runkel R, Crawford C, Cohn T (2004) Load Estimator (LOADEST): A Fortran program for estimating constituent loads in streams and rivers 2004. US Geological Survey Techniques and Methods Book, 4 Salamon P, Feyen L (2009) Assessing parameter, precipitation, and predictive uncertainty in a distributed hydrological model using sequential data assimilation with the particle filter. J Hydrol 376:428–442. doi:10.1016/j.jhydrol.2009.07.051 Seo M-J, Yen H, Jeong J (2014) Transferability of input parameters between SWAT 2009 and SWAT 2012. J Environ Qual 43:869–880. doi:10.2134/jeq2013.11.0450 Servat E, Dezetter A (1991) Selection of calibration objective functions in the context of rainfall-runoff modeling in a Sudanese savannah area. Hydrol Sci J 36:307–330 Tolson BA, Shoemaker CA (2007) Dynamically dimensioned search algorithm for computationally efficient watershed model calibration. Water Resour Res 43(1):1–16 USDA-SCS (1972) National Engineering Handbook, Hydrology, Section 4, Chapter 4–10. Soil Conservation Service, Washington Vrugt JA, Braak CJF, Diks CGH, Robinson BA, Hyman JM, Higdon D (2009) Accelerating Markov Chain Monte Carlo simulation by differential evolution with self-adaptive randomized subspace sampling. Int J Nonlinear Sci Numer Simul 10:271–288 Williams JW, Izaurralde RC, Steglich EM (2012) Agricultural Policy/ Environmental Extender Model Theoretical Documentation Version 0806. 131 Yang J, Reichert P, Abbaspour KC, Xia J, Yang H (2008) Comparing uncertainty analysis techniques for a SWAT application to the Chaohe Basin in China. J Hydrol 358(1–2):1–23 Yen H (2012) Confronting input, parameter, structural, and measurement uncertainty in multi-site multiple responses watershed modeling using Bayesian inferences. Ph.D. dissertation, Colorado State University Yen H, Wang X, Fontane DG, Harmel RD, Arabi M (2014a) A framework for propagation of uncertainty contributed by parameterization, input data, model structure, and calibration/validation data in watershed modeling. Environ Model Softw 54:211–221. doi:10.1016/j.envsoft.2014.01.004 Yen H, Bailey RT, Arabi M, Ahmadi M, White MJ, Arnold JG (2014b) Evaluation of watershed model performance using general watershed information: beyond typical accuracy. J Environ Qual. doi:10.2134/jeq2013.03.0110 (published online) Yen H, Jeong J, Feng Q, Deb D (2014c) Assessment of input uncertainty in SWAT using latent variables. Water Resour Manag. doi:10.1007/s11269-014-0865-y (published online) Yen H, White MJ, Arabi M, Arnold JG (2014d) Evaluation of alternative surface runoff accounting procedures using the SWAT model. Int J Agric Biol Eng. doi:10.3965/j.ijabe. 20150801.005 (published online) Zhenyao S, Lei C, Tao C (2013) The influence of parameter distribution uncertainty on hydrological and sediment modeling: a case study of SWAT model applied to the Daning watershed of the Three Gorges Reservoir Region, China. Stoch Env Res Risk Assess 27(1):235–251. doi:10.1007/s00477-012-0579-8

123