MCDM 2004, Whistler, B.C. Canada August 6 –11, 2004
INCORPORATING A DISTANCE-BASED UNCERTAINTY ANALYSIS APPROACH INTO PROMETHEE 1 Kylie M. Hyde and Holger R. Maier Centre for Applied Modelling in Water Engineering, School of Civil & Environmental Engineering, The University of Adelaide, Adelaide, SA, 5005, Australia. Email:
[email protected], Email:
[email protected]
Keywords: Uncertainty, Multi-criteria decision analysis, PROMETHEE, Euclidean distance Summary: Analyses of decision problems, involving tradeoffs among multiple criteria, can be undertaken using multi-criteria decision analysis (MCDA). However, various sources of uncertainty exist in the application of MCDA techniques such as PROMETHEE, including the definition of criteria weights, the assignment of criteria performance values and the specification of the generalised criterion functions. Sensitivity analysis methods can be used to analyse the effects of these uncertainties on the ranking of alternatives, however, many existing methods have been found to have numerous limitations when applied to MCDA. In this paper, a distance-based approach is proposed which enables the decision maker to examine the robustness of a solution, allowing a decision to be made with confidence that the alternative chosen is the best performing alternative under the range of probable circumstances. In addition, the most critical input parameters to a reversal in ranking of two alternatives can be identified using the proposed approach. The benefits of the approach are illustrated by applying it to a water resources case study.
1.
Introduction
The Preference Ranking Organisation Method of Enrichment Evaluation (PROMETHEE) multi-criteria decision analysis (MCDA) technique developed by Brans et al. (1986) belongs to the class of outranking approaches and is one of the best known and most widely applied outranking methods because it follows a transparent computational procedure and can be easily understood by actors (i.e. people involved in the decision analysis process) and decision makers (DMs) (i.e. people responsible for making the final decision) (Georgopoulou et al., 1998). This is evident by its widespread use in decision making situations such as assessing water resource management problems (Martin et al., 1999), energy planning (Georgopoulou et al., 1998), and waste management (Hokkanen and Salminen, 1997). The basic PROMETHEE methods build a valued outranking relation, which is used to obtain either a partial (PROMETHEE I) or a complete (PROMETHEE II) ranking of the alternatives. The preference function associated with each criterion gives the degree of preference, expressed by the DM, for alternative a with respect to alternative b on criterion x j . Further details of the PROMETHEE method are contained in Brans et al. (1986). The input data required by the majority of MCDA techniques, including PROMETHEE, is the assignment of criteria performance values (PVs) by experts and the elicitation of criteria weights (CWs) from actors. Providing precise figures for the criteria PVs is often difficult, as the alternatives being assessed are generally predicted future events. There may therefore be some imprecision, contradiction, arbitrariness and / or lack of consensus concerning the criteria PVs used in the analysis (Mousseau et al., 2003). PROMETHEE has attempted to take this form of uncertainty into account by incorporating generalised criterion functions into the analysis. Six types of generalised criterion functions have been suggested by Brans et al. (1986) with the aim of realistically modelling the DM’ s preference, which gradually increases from indifference to strict preference and to facilitate the inclusion of the inherent 1
The authors wish to thank the Australian Research Council, Department for Water, Land, Biodiversity and Conservation and the Office of Economic Development, State Government of South Australia for the funding to undertake this research. In addition, Jean-Philippe Waaub for providing the case study data.
uncertainty in the criteria PVs in the decision analysis process. However, the selection of an appropriate function and the associated thresholds for each criterion is a complex and ambiguous task for DMs and actors and therefore adds another element of uncertainty into the decision analysis process (Salminen et al., 1998) . CWs indicate a criterion’s relative importance and allow actors’ views and their impact on the ranking of alternatives to be expressed explicitly. The inclusion of the generalised criterion functions does not address the inherent imprecision and subjectiveness of the CWs, which are elicited by the decision analyst from the actors for each criterion using one of a variety of available techniques. The limitations of using the generalised criterion functions and deterministic CWs imply that considerable uncertainty remains in the ranking of the alternatives following the completion of the MCDA process using the PROMETHEE method. The CWs and PVs have been found to influence the resultant ranking of alternatives and therefore the uncertainty associated with the values assigned to these parameters should be taken into consideration as part of the decisionmaking process (Mareschal, 1988). However, despite their importance, these factors have been largely ignored in numerous studies to which PROMETHEE has been applied (Al-Kloub et al., 1997; Haralambopoulos and Polatidis, 2003). To assess the extent the solution is dependent on and sensitive to the input parameter estimates, sensitivity analysis is commonly used. Sensitivity analysis allows an assessment to be undertaken of the impact that any assumptions made have on the ranking of the alternatives, which generally involves altering the CWs or changing the generalised criterion functions or threshold values. However, this analysis is frequently incomplete and unsatisfactory, with values often altered arbitrarily depending on the desired outcome. Decision Lab 2000 (formerly PROMCALC) is a software package which implements the PROMETHEE methods. The consequences of modifications of the initially specified weights are able to be determined in Decision Lab 2000 but they do not give insight into the way the ranking changes if the stability boundaries are exceeded (Wolters and Mareschal, 1995). Numerous distance-based sensitivity analysis methods have been proposed in the literature. Wolters and Mareschal (1995) and Triantaphyllou and Sanchez (1997) have proposed approaches to determine the minimum modification to the CWs required to make a specific alternative ranked first while taking into account specific requirements on the weight variations. Isaacs (1963) and Schneller and Sphicas (1985) have utilised the Euclidean distance to determine the sensitivity of decisions to probability estimation errors. Barron and Schmidt (1988) and Soofi (1990) use Euclidean distances in problems where there is some imprecision in the CWs of an additive value function. Rios Insua and French (1991) introduce a framework for sensitivity analysis in multiobjective decision making within a Bayesian context and also utilise the Euclidean distance and Chebyshev distance. Each of these methods is limited, however, in that they are either applicable to only one type of MCDA method, consider only one of the input parameters (i.e. CWs or PVs), or vary only one input parameter at a time while the remaining parameters are kept constant. The effective management of uncertainty is one of the most fundamental problems in decision making (Felli and Hazen, 1998). To overcome the shortcomings of existing sensitivity analysis methods for MCDA, and in particular PROMETHEE, an approach is presented in this paper that may be summarised as the determination of the set of all input parameter values that is the minimum distance from the original set of input parameter values, which result in the reversal in the ranking of two alternatives. The proposed approach, therefore, determines how sensitive the rankings of alternatives are to the simultaneous variation in all of the input parameters over their expected range, which significantly increases the level of confidence in a selected alternative if the ranking of the solution is robust to the choice of values of the CWs and PVs assigned to the criteria. Conversely, if the rankings of alternatives are sensitive to the input parameters, it is difficult to conclude that one alternative is superior to another, and other factors might need to be considered to make a final decision. This paper presents details of the proposed approach, which is then demonstrated using a water resources allocation case study undertaken by Martin et al. (1999).
2.
Proposed Approach
The purpose of the proposed deterministic uncertainty analysis approach is to determine the minimum modification of the MCDA input parameters (i.e. CWs and PVs) that is required to modify the total flow of a selected alternative (ay) such that it is equal to an initially higher ranked alternative (a x), by exploring the feasible input parameter range. The minimum modification of the input parameters is obtained by translating the problem into an optimisation problem and exploring the feasible input parameter ranges. The objective function minimises a distance metric,
which provides a numerical value of the amount of dissimilarity between the original input parameters of the two alternatives under consideration and the optimised PVs of the initially lower ranked alternative and the CWs, which are common to all alternatives. Optimised refers to the set of parameters that is the smallest distance from the original parameter set, such that the total flow of the two alternatives being assessed is equal. An alternative methodology involves optimising the PVs of the initially higher ranked alternative, which are held constant in the approach presented in this paper, in addition to the CWs and the PVs of the initially lower ranked alternative. The Euclidean Distance, d e, has been selected as the distance metric to use in this paper, as it is the most commonly used metric. When the Euclidean Distance is used, the objective function may be defined as:
Minimise
de =
M
∑ (w m =1
− wmo ) + (x mni − xmno ) 2
mi
2
(1)
Subject to the following constraints: M
∑w m =1
mi
=
M
∑w
(2)
mo
m =1
φ (a y )opt = φ ( ax )opt
(3)
LL x ≤ x mno ≤ UL x for m = 1 to M
(4)
LLw ≤ wmo ≤ ULw
(5)
for m = 1 to M, and for actor j
where wmi is the initial CW of criterion m, wmo is the optimised CW of criterion m, x mni is the initial PV of criterion m of initially lower ranked alternative n, x mo is the optimised PV of criterion m of initially lower ranked alternative n, M is the total number of criteria, φ(a y )opt is the modified total flow of the initially lower ranked alternative (calculated using optimised PVs and CWs), φ(ax)opt is the modified total flow of the initially higher ranked alternative (calculated using original PVs and optimised CWs), LLx and ULx are the lower and upper limits, respectively, of the PVs of each criterion for the lower ranked alternative (a y), LLw and ULw are the lower and upper limits, respectively, of each of the CWs elicited from actor j. In some situations, one alternative will always be superior to another, regardless of the values of the input parameters. In this case, the ranking of the alternatives is robust, as it is insensitive to the input parameters. However, in many instances, this is not the case, and a number of different combinations of the input parameters will result in rank equivalence. By determining the smallest overall change that needs to be made to the input parameters (e.g. CWs and PVs) of the lower ranked alternative in order to achieve rank equivalence, the robustness of the ranking of two alternatives (a x and a y) is obtained. The above concept is illustrated in Figure 1 for a simple two-dimensional example. In Figure 1, the input parameter values of the lower ranked alternative (IP1,y , IP2,y ), which result in a total flow of φ(a y), are given by point Y. All combinations of IP1,y and IP2,y on the curved line labelled φ(a x )opt=φ(a y)opt will modify the total flow of alternative y so that it is equivalent to that of alternative x once the input parameters of both alternatives have been modified, resulting in rank equivalence of the two alternatives.
IP2
φ(a x)opt=φ(ay)opt
X d
Y = (IP1,y, IP2,y ) with φ(a y) IP1 Figure 1. 2D Concept of Proposed Approach
Consequently, the robustness of the ranking of alternatives x and y is given by the shortest distance between point Y and the φ(a x)opt=φ(ay)opt line (i.e. d). If this distance is large, then more substantial changes need to be made to the
input parameters in order to achieve rank equivalence, and the ranking of the two alternatives is relatively insensitive to input parameter values (i.e. robust). Conversely, if this distance is small, minor changes in the input parameters will result in rank equivalence, and the ranking of the alternatives is sensitive to input parameter values (i.e. not robust). As the proposed approach identifies the combination of input parameters that is the shortest distance from the original parameter set, the input parameters to which the rankings are most sensitive are also identified. The proposed approach is repeated for each pair of alternatives (i.e. a x and a y) and using each actor’s set of CWs following the definition of a number of constraints, including the total sum of the optimised CWs must be equal to the total sum of the original CWs (Equation 2). In addition, the net flow of the initially lower ranked alternative (φ( a y )opt) is constrained to equal the net flow of the initially higher ranked alternative (φ(ax)opt), with the net flows determined using PROMETHEE (Brans et al., 1986) and using the Level I or Usual generalised criterion functions for all of the criteria (Equation 3). The actor or DM does not need to assign the generalised criterion functions because uncertainties associated with the criteria PVs are considered by utilising the expected range of values (Equation 4). The net flow of the alternatives is determined using the optimised CWs, the optimised PVs of the initially lower ranked alternative and the original PVs of the other alternatives. The expected ranges that the input parameters of the lower ranked alternative can be varied between to obtain a reversal in ranking of the selected alternatives (i.e. φ( a y)opt > φ(a x)opt) are further constraints. Specification of the minimum and maximum ranges of the input parameters represent the potential uncertainty and variability in the assignment of these values in the initial stage of the decision analysis process. The range of values (i.e. upper and lower bounds) that are specified for each PV of every criterion of the initially lower ranked alternative represent the set of possible values for that variable, which can either be based upon knowledge of the experts or the data that are available (Equation 4). The feasible range of the CWs for each actor represents the expected variability in the CWs due to the subjective and ambiguous nature of the preference values. The CW ranges can be defined by either the DM or actors or, alternatively, actual ranges of the available CWs can be utilised (e.g. the minimum and maximum values of the CWs elicited from the actors involved in the decision process) (Equation 5). In a situation where the experts or actors are confident in the original input parameter values, the lower and upper bounds of the particular parameter would be equal to the original input parameter. For example, this may be particularly relevant for the situation where qualitative data ranges (e.g. High to Low, where 1 equals High and 5 equals Low) are used for a particular criterion. In order to obtain the robustness of the ranking of a pair of alternatives, the optimisation problem given by Equations 1 –5 needs to be solved. This can be achieved by using a number of optimisation techniques. In this paper, the Generalised Reduced Gradient (GRG2) nonlinear optimisation method is used to solve the objective function by changing the input parameters within their specified ranges subject to the defined constraints, although other optimisation methods such as genetic algorithms (GAs) could also be used. GRG2 works by first evaluating the function and their derivatives at a starting value of the decision vector and then iteratively searching for a better solution using a search direction suggested by the derivatives (Stokes and Plummer, 2004). The search continues until one of several termination criteria is met. Among these are: (i) the optimality criteria have been met to within a specified tolerance, (ii) the difference between the objective function values at successive points is less than some tolerance for a specified number of consecutive iterations, (iii) a default or user-specified iteration limit or time has been exceeded, or (iv) a feasible point cannot be found or a feasible non-optimal point has been obtained but a direction of improvement cannot be found. If no solution can be found, the DM can be confident that the ranking of the two alternatives is robust (i.e. that no changes in the input parameters between the specified ranges will result in a reversal of the ranking). Random numbers are generated between the specified input parameter ranges for the PVs and CWs to be used as the starting values of the input parameters for the optimisation. GRG2 is not a global optimisation algorithm, therefore, to increase the chances of finding the global or near-global optima, the optimisation is repeated a number of times using different randomly generated starting values. This aims to minimise the influence the starting values have on the outcome of the analysis. A non-feasible (NF) outcome occurs when one or more of the constraints are violated. The output of the proposed approach is the minimum Euclidean distance for each pair of alternatives, which can be summarised in a matrix. A non-feasible outcome or a large Euclidean distance between two alternatives informs the DM that one alternative will predominantly be superior to another, regardless of the input parameter values.
Conversely, if the distance is small, slight changes in the input parameters will result in rank equivalence which indicates that the ranking of the alternatives is sensitive to the input parameter values. The most critical criteria input parameters can also be identified by examining the relative and absolute change in the original and optimised parameter values: Absolute ∆ x mn =
x mno − xmni
or Absolute ∆ w m =
wmo − wmi
(6)
Relative ∆ x mn =
x mno − x mni × 100 % xmni
or Relative ∆ wm =
wmo − wmi × 100 % w mi
(7)
The criteria that exhibit the smallest relative change in value are most critical to the reversal in ranking. The results of the proposed distance-based uncertainty analysis approach provide the DM with further information to aid in making a final decision, including the robustness of the ranking of the alternatives and information on the most critical input parameters, while not requiring the DM to specify generalised criterion functions for each of the criteria and allowing simultaneous variation of all of the input parameters. The proposed method of incorporating the uncertainty in the PVs by upper and lower bounds is less subjective than assigning generalised criterion functions, as determining the feasible range that the parameters can vary between (c.f. thresholds for generalised criterion functions) is more intuitive for the actor as actual data are often available and the values that have to be chosen have a physical meaning.
3.
Case Study
The proposed approach has been applied to a study undertaken by Martin et al. (1999) and details of the study, the problem formulation and the results are presented below. 3.1. Background The Saint Charles River, located in the Province of Quebec, Canada, is a source of drinking water for the surrounding municipalities, is approximately 35 km long, and crosses contrasting natural environments. The water quality of the river is deteriorating due to withdrawal during the summer months and uncontrolled or conflicting users near the river. The review of Quebec’s Urban Community’s land-use and development plan provided an opportunity for a MCDA to be undertaken by Martin et al. (1999) with the aim of selecting a development alternative that would meet the needs of the surrounding populations as well as take into account the natural environment. 12 actors were involved in elaborating the 8 development alternatives and 11 evaluation criteria which cover a range of categories including land-use, environment, social and economic. The alternatives (Table 1) were evaluated using available georeferenced data, ecological mapping and spatial information. The thresholds used for the generalised criterion functions were those suggested by the MCDA program PROMCALC based on the mean and standard deviation of the data. A description of the criteria, the generalised criterion functions and associated thresholds, the preference direction of each of the criteria and the criteria PVs used by Martin et al. (1999) for the MCDA are summarised in Table 2. In addition, each of the 12 actors provided weights for each criterion. The rankings of the alternatives for each set of actor CWs from the MCDA undertaken by Martin et al. (1999), using PROMCALC, are contained in Table 3. Altogether, the best performing alternatives are 2, 3, 6 and 8 and the rankings are very similar for each of the actors, with the exception of actors 9 and 10, whose preferences are different.
Table 1. Description of Development Alternatives Assessed by Martin et al. (1999) Alternative
Potential Development Actions
1. Green corridor
Priority residential developments, 20 m watercourse protection zone, green corridor along the river (recreation zoning)
2. Priority
Priority residential developments, 20 m watercourse protection zone, linear park along the river
3. Various
Same as Alternative 2 plus a golf course, two recycling facilities, a snow dump, modification of Cloutier sector (Loretteville) from ‘residential’to ‘recreo-tourism’
4. Maximum
Maximum residential development, 20 m watercourse protection zone, linear park
5. Maximum & golf
Same as Alternative 4 plus a golf course
6. Maximum & runoff
Same as Alternative 4 plus interventions aimed at reducing urban & stormwater runoff to the river
7. Maximum & 5 m zone
Maximum residential development, 5 m watercourse protection zone, linear park
8. Maximum & 30 m zone
Maximum residential development, 30 m watercourse protection zone, linear park
Table 2. Description of the Criteria and the Associated Values Utilised by Martin et al. (1999) Fn. Type
Threshold Values
Pref Dirn
1
2
3
4
5
6
7
8
Accessibility to Park & River
1
-
Min
10
8
9
8
9
8
8
8
Continuity
3
p = 102.52
Min
420
195
196
195
196
195
195
195
Recreational & Green Spaces
5
q = 1.11 p = 2.84
Max
-20.56
-16.64
-14.64
-19.82
-18.14
-19.82
-19.82
-19.82
Effect on Quality of Landscapes
1
-
Min
3
2
1
2
1
2
2
2
Effects on Water quality
3
p = 0.45
Max
-0.25
0.50
-1.25
0.50
-0.50
1.50
-1.50
1.00
Influence on Water flow
1
-
Min
3
3
1
4
4
2
4
4
Protection of Riparian Zone
1
-
Max
1
2
2
2
2
2
1
3
Sensitive elements
5
q = 4.52 p = 15.94
Max
0
35
34
35
34
35
35
35
Collective Lands
5
q = 0.47 p = 1.17
Max
23.0
24.0
23.5
22.1
21.6
22.1
22.1
22.1
Municipal Costs
4
q = 1.23 p = 2.85
Min
1
2
7
3
3
6
4
5
Fiscal Benefits
1
-
Min
4
4
3
2
1
2
2
2
Criterion
Alternative
3.2. Problem Formulation Generalised criterion functions are not required to be specified for each of the criteria in the proposed approach, therefore, the deterministic analysis is repeated using PROMETHEE as part of this research using the criteria PVs and CWs provided by Martin et al. (1999). Level I generalised criterion functions are used for each criterion to enable the outranking methodology to be undertaken. The feasible input parameter range for the actors’CWs and the PVs of the initially lower ranked alternative must be specified, as defined by Equations 4 and 5. No information was provided by Martin et al. (1999) on the uncertainty associated with the criteria PVs or the CWs, therefore, the upper and lower limits of the input parameters were based upon the range of PVs and CWs, to enable the proposed
approach to be illustrated. The optimisation was undertaken using the Microsoft Excel Add-In Solver Function. The main advantages of Solver are its wide availability and ease of use. Information on Solver and the options available can be obtained from Microsoft Excel or Stokes and Plummer (2004). The Microsoft Excel binomial random number generator (i.e. RANDBETWEEN function) was used to generate the random starting values of the input parameters. This operation was repeated a number of times for each pair of alternatives to sufficiently vary the starting values with the aim of increasing the chances of finding near globally optimal solutions. Table 3. Complete Rankings of Alter natives for each set of CWs by Martin et al. (1999) Rank
Actor 1
Actor 2
Actor 3
Actor 4
Actor 5
Actor 6
Actor 7
Actor 8
Actor 9
Actor 10
Actor 11
Actor 12
1
Alt 2
Alt 3
Alt 3
Alt 6
Alt 2
Alt 6
Alt 6
Alt 2
Alt 5
Alt 1
Alt 2
Alt 2
2
Alt 6
Alt 2
Alt 2
Alt 2
Alt 6
Alt 2
Alt 2
Alt 8
Alt 2
Alt 3
Alt 3
Alt 3
3
Alt 8
Alt 8
Alt 6
Alt 3
Alt 8
Alt 8
Alt 3
Alt 3
Alt 4
Alt 5
Alt 5
Alt 6
4
Alt 4
Alt 6
Alt 8
Alt 8
Alt 3
Alt 4
Alt 8
Alt 6
Alt 7
Alt 6
Alt 6
Alt 8
5
Alt 3
Alt 5
Alt 5
Alt 5
Alt 4
Alt 3
Alt 4
Alt 5
Alt 8
Alt 2
Alt 8
Alt 4
6
Alt 5
Alt 4
Alt 4
Alt 4
Alt 5
Alt 5
Alt 5
Alt 4
Alt 1
Alt 4
Alt 4
Alt 5
7
Alt 7
Alt 7
Alt 7
Alt 7
Alt 7
Alt 7
Alt 7
Alt 7
Alt 6
Alt 7
Alt 7
Alt 1
8
Alt 1
Alt 1
Alt 1
Alt 1
Alt 1
Alt 1
Alt 1
Alt 1
Alt 3
Alt 8
Alt 1
Alt 7
3.3. Results and Discussion The complete rankings and total flows of the alternatives for each set of actors’CWs using PROMETHEE with the Level I generalised criterion function for each criterion are contained in Table 4. The best performing alternatives are 2, 6 and 8, being consistently ranked in the top three alternatives by each of the actors. However, the total flow of each alternative varies quite markedly for each of the actors’CWs, as does the difference in total flows between two alternatives, in particular the two highest ranked alternatives. It would therefore be difficult for the DM to confidently select an optimal alternative due to the variability in the results. The rankings in Table 4 are slightly different from those obtained by Martin et al. (1999) (see Table 3), with the most significant difference being the relegation of Alternative 3 to a much lower position in the ranking for each of the actors’CWs when the Level I generalised criterion functions are used. The results of the proposed approach using Actor 2 CWs (Table 5) are presented in this paper, as space limitations restrict the presentation of the results for each actors’set of CWs. Different Euclidean distances were obtained with each of the random input parameter starting values for each pair of alternatives using Solver, which indicates that the solution space is very complex and that the starting values have an impact on the solution. The solution producing the minimum Euclidean distance from the ranges of solutions obtained with the random starting values was designated as the ‘best’solution. The Euclidean distance obtained for each pair of alternatives using Actor 2 CWs is summarised in Table 6. The initially lower ranked alternatives are listed down the leftmost column of Table 6, in rank order. The Euclidean distances presented are those which result in rank equivalence between each pair of alternatives, with the initially higher ranked alternatives listed across the top row, in rank order (Table 6). For example, a Euclidean distance of 5.58 is obtained when the input parameters are varied such that Alternative 8 (rank 2nd ) has the same total flow as Alternative 2, which is the initially highest ranked alternative. Further information and analysis would therefore be required before Alternative 2 can be confidently selected by the DM as being optimal due to the relatively small Euclidean distance. Alternatives 1 and 7 could be eliminated from the analysis due to comparatively large Euclidean distances with the higher ranked alternatives, signifying that a large change in the input parameter values of these alternatives is required before a reversal of the ranking occurs. Based on the deterministic PROMETHEE results using Actor 2 CWs (Table 4), the difference between the total flows of the initially highest ranked alternative, Alternative 2 (30), and Alternative 3 (6.29) is quite large, therefore, on viewing these results, the DM would naturally conclude that Alternative 3 would not be a preferred alternative. However, Alternative 3 (initially ranked 5th) has one of the lowest Euclidean Distances (8.47) when the proposed
approach is utilised to determine the change in input parameters required for each of the alternatives to outrank Alternative 2, which is the highest ranked alternative. The results indicate that the ranking of Alternative 2 is not very robust with respect to changes in the input parameter values of Alternative 3. As Alternative 3 is the highest ranked alternative obtained by Martin et al. (1999) (Table 3), this result is not surprising and demonstrates the benefits of assessing the impact that the simultaneous variation in the input parameters has on the ranking of the alternatives using the proposed approach. The results also demonstrate that utilising the parameter ranges is an adequate method for taking the uncertainty in the criteria PVs into account, as opposed to the uncertainty associated with specifying generalised criterion functions for each criterion. Table 4. Rankings & Total Flows of Alternatives: each actors CWs, Level I Generalised Criterion Functions Rank
Actor 1
Actor 2
Actor 3
Actor 4
Actor 5
Actor 6
Actor 7
Actor 8
Actor 9
Actor 10
Actor 11
Actor 12
1
Alt 2
Alt 2
Alt 2
Alt 2
Alt 2
Alt 2
Alt 6
Alt 2
Alt 2
Alt 6
Alt 2
Alt 2
2
Alt 6
Alt 8
Alt 6
Alt 6
Alt 6
Alt 6
Alt 2
Alt 8
Alt 5
Alt 2
Alt 6
Alt 3
3
Alt 8
Alt 6
Alt 8
Alt 8
Alt 8
Alt 8
Alt 8
Alt 6
Alt 4
Alt 8
Alt 8
Alt 6
4
Alt 4
Alt 4
Alt 3
Alt 4
Alt 4
Alt 4
Alt 3
Alt 4
Alt 1
Alt 4
Alt 4
Alt 8
5
Alt 3
Alt 3
Alt 4
Alt 3
Alt 3
Alt 3
Alt 4
Alt 3
Alt 7
Alt 7
Alt 3
Alt 4
6
Alt 5
Alt 5
Alt 5
Alt 5
Alt 5
Alt 5
Alt 5
Alt 5
Alt 8
Alt 3
Alt 7
Alt 5
7
Alt 7
Alt 7
Alt 7
Alt 7
Alt 7
Alt 7
Alt 7
Alt 7
Alt 6
Alt 5
Alt 5
Alt 7
8
Alt 1
Alt 1
Alt 1
Alt 1
Alt 1
Alt 1
Alt 1
Alt 1
Alt 3
Alt 1
Alt 1
Alt 1
Total Flows
Actor 1
Actor 2
Actor 3
Actor 4
Actor 5
Actor 6
Actor 7
Actor 8
Actor 9
Actor 10
Actor 11
Actor 12
1
42.14
30.00
27.00
28.57
32.71
32.57
35.71
37.00
31.29
37.29
34.71
51.29
2
17.43
25.86
22.71
28.43
28.86
30.14
31.86
24.71
29.14
31.29
16.00
19.29
3
10.86
24.29
19.57
12.43
24.14
20.43
22.00
14.43
16.43
31.00
14.29
14.86
4
9.71
10.14
6.57
4.43
10.00
9.43
10.43
14.00
-7.29
20.29
11.86
6.71
5
-6.71
6.29
5.71
2.14
-3.14
-4.29
2.71
-3.29
-7.57
-3.86
8.71
-0.86
6
-13.00
-9.29
-5.86
-16.71
-15.57
-17.71
-16.57
-7.43
-8.29
-19.43
2.57
-22.14
7
-21.86
-18.57
-28.71
-27.57
-25.86
-25.29
-37.71
-14.57
-18.86
-28.71
-2.43
-31.71
8
-38.57
-68.71
-47.00
-31.71
-51.14
-45.29
-48.43
-64.86
-34.86
-67.86
-85.71
-37.43
Table 5. Actor 2 Criteria Weights Criteria
1
2
3
4
5
6
7
8
9
10
11
CW
4
15
10
15
10
7
15
14
5
2
3
The results of the proposed approach can also be used to determine the absolute change (Equation 6) and percentage change (Equation 7) in the input parameters required to reverse the ranking of two alternatives. The changes in the input parameters required for Alternative 8 to equal Alternative 2 are contained in Table 7. From this it can be seen that the weights of criteria 4, 6 and 10 and the PVs of criteria 4, 5, 7 and 11 are the most critical, as only minimal changes to these parameters are required for rank equivalence to occur. The results of the analysis also illustrate that the criteria PVs play an equally important role in the decision analysis process, as only minor changes in all of the PVs are required for rank equivalence to occur. This is an important result, as existing sensitivity analysis methods generally ignore the uncertainty in the PVs, or they consider uncertainty only in CWs or PVs, but not both. The implication of this is that a complete and accurate understanding of the uncertainty associated with the ranking of the alternatives is not obtained when existing sensitivity analysis methods are used.
Table 6. Euclidean distances of alternatives for Actor 2 Criteria Weights
Initially Lower Ranked Alternatives
Initially Higher Ranked Alternatives Alt 2 (R1)
Alt 8 (R2)
Alt 6 (R3)
Alt 4 (R4)
Alt 3 (R5)
Alt 5 (R6)
Alt 7 (R7)
Alt 1 (R8)
Alt 2 (R1)
-
NA
NA
NA
NA
NA
NA
NA
Alt 8 (R2)
5.58
-
NA
NA
NA
NA
NA
NA
Alt 6 (R3)
7.58
4.32
-
NA
NA
NA
NA
NA
Alt 4 (R4)
12.01
7.57
12.18
-
NA
NA
NA
NA
Alt 3 (R5)
8.47
5.44
4.23
2.64
-
NA
NA
NA
Alt 5 (R6)
6.49
11.76
8.68
3.22
13.20
-
NA
NA
Alt 7 (R7)
16.37
22.43
17.10
20.84
20.59
18.78
-
NA
Alt 1 (R8)
19.18
24.38
21.40
19.66
15.52
17.64
14.79
-
Note: R = Rank, NA = not applicable as the alternative is already ranked higher than its paired alternative Table 7. Changes in the Input Parameters for Alt 8 = Alt 2, CWs of Actor2 CWs of Actor 2 Criteria
4.
PVs Alternative 8
wmi
wmo
Absolute ∆
% Relative ∆
xm,8,i
xm,8,o
Absolute ∆
% Relative ∆
1
4.00
3.05
-0.95
-23.63%
8.00
8.03
0.03
0.38%
2
15.00
13.27
-1.73
-11.52%
195.00
195.04
0.04
0.02%
3
10.00
9.07
-0.93
-9.26%
-19.82
-19.75
0.07
-0.37%
4
15.00
15.00
0.00
0.00%
2.00
1.99
-0.01
-0.60%
5
10.00
11.12
1.12
11.20%
1.00
1.00
0.00
0.00%
6
7.00
6.92
-0.08
-1.08%
4.00
3.91
-0.09
-2.32%
7
15.00
17.00
2.00
13.33%
3.00
3.00
0.00
0.00%
8
14.00
12.18
-1.82
-13.01%
35.00
34.00
-1.00
-2.86%
9
5.00
3.55
-1.45
-29.09%
22.10
22.17
0.07
0.34%
10
2.00
2.00
0.00
0.00%
5.00
4.95
-0.05
-1.09%
11
3.00
6.83
3.83
127.72%
2.00
1.99
-0.01
-0.40%
Summary and Conclusions
MCDA, and in particular PROMETHEE, is utilised extensively to assess many types of decision analysis problems, however, the uncertainty associated with the input parameters values is rarely, or ambiguously, considered. A distance-based uncertainty analysis approach is proposed in this paper, which calculates the magnitude of the change in the initial data that is required to result in rank equivalence between any two alternatives. Applying the proposed methodology to the study undertaken by Martin et al. (1999) illustrated that different rankings can be obtained when
different generalised criterion functions are used. The results of the proposed approach (of Alternative 3 in particular) also demonstrate that the complete rankings and the difference between the total flows should not be relied upon in selecting an optimal alternative. Undertaking uncertainty analysis by varying the input parameters simultaneously between their expected ranges is essential to determine how robust the ranking of the alternatives is to the input parameter values. Identifying the most critical input parameters also provides the DM with valuable information which can provide direction for further analysis or confidence that a large change in the ni put parameters is required before a reversal in the ranking occurs. The results of the proposed approach for the case study also demonstrated that both the CWs and PVs have an impact on the ranking of the alternatives and therefore the uncertainty in the values of the CWs and PVs should be considered in the decision analysis concurrently.
References Al-Kloub, B., Al-Shemmeri, T. and Pearman, A. (1997) "The role of weights in multi-criteria decision aid, and the ranking of water projects in Jordan" European Journal of Operational Research, 99, 278-288. Barron, H. and Schmidt, P. (1988) "Sensitivity analysis of additive multiattribute value models" Operations Research, 36, 122-127. Brans, J. P., Vincke, P. and Mareschal, B. (1986) "How to select and how to rank projects: the PROMETHEE method" European Journal of Operational Research, 24, 228-238. Felli, J. C. and Hazen, G. B. (1998) "Sensitivity analysis and the expected value of perfect information" Medical decision making: an international journal of the society for medical decision making, 18, 95-109. Georgopoulou, E., Sarafidis, Y. and Diakoulaki, D. (1998) "Design and implementation of a group DSS for sustaining renewable energies exploitation" European Journal of Operational Research, 109, 483-500. Haralambopoulos, D. A. and Polatidis, H. (2003) "Renewable energy projects: structuring a multi-criteria group decision-making framework" Renewable Energy, 28, 961-973. Hokkanen, J. and Salminen, P. (1997) "Locating a Waste Treatment Facility by Multicriteria Analysis" Journal of Multi-Criteria Decision Analysis, 6, 175-184. Isaacs, H. H. (1963) "Sensitivity of decisions to probability estimation errors" Operations Research, 11, 536-552. Mareschal, B. (1988) "Weight stability intervals in mu lticriteria decision aid" European Journal of Operational Research, 33, 54-64. Martin, N. J., St Onge, B. and Waaub, J. P. (1999) "An integrated decision aid system for the development of Saint Charles River alluvial plain, Quebec, Canada" International Journal of Environment and Pollution, 12, 264-279. Mousseau, V., Figueira, J., Dias, L., Gomes da Silva, C. and Climaco, J. (2003) "Resolving inconsistencies among constraints on the parameters of an MCDA model" European Journal of Operational Research, 147, 72-93. Rios Insua, D. and French, S. (1991) "A framework for sensitivity analysis in discrete multi-objective decisionmaking" European Journal of Operational Research, 54, 176-190. Salminen, P., Hokkanen, J. and Lahdelma, R. (1998) "Comparing multicriteria methods in the context of environmental problems" European Journal of Operational Research, 104, 485-496. Schneller, G. O. and Sphicas, G. (1985) "On sensitivity analysis in decision theory" Decision Sciences, 16, 399-409. Soofi, E. S. (1990) "Generalized entropy-based weights for multiattribute value models" Operations Research, 38, 362-363. Stokes, S. and Plummer, J. (2004) "Using spreadsheet solvers in sample design" Computational statistics & data analysis, 44, 527-546. Triantaphyllou, E. and Sanchez, A. (1997) "A sensitivity analysis approach for some deterministic multi-criteria decision-making methods" Decision Sciences, 28, 151-194. Wolters, W. T. M. and Mareschal, B. (1995) "Novel types of sensitivity analysis for additive MCDM methods" European Journal of Operational Research, 81, 281-290.
