A DEA-neural network approach

Expert Systems with Applications 42 (2015) 6758–6766

Contents lists available at ScienceDirect

Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa

Two-stage production modeling of large U.S. banks: A DEA-neural network approach He-Boong Kwon a,⇑, Jooh Lee b a b

Hasan School of Business, Colorado State University-Pueblo, 2200 Bonforte Blvd., Pueblo, CO 81001, USA William G. Rohrer College of Business, Rowan University, 201 Mullica Hill Rd., Glassboro, NJ 08028, USA

a r t i c l e

i n f o

Article history: Available online 30 April 2015 Keywords: Banking Data envelopment analysis (DEA) Neural network Two-stage DEA

a b s t r a c t The purpose of this paper is to explore an innovative performance model for a two-stage sequential production process by combining data envelopment analysis (DEA) and back propagation neural network (BPNN). Recent literature shows a growing interest on performance modeling of two-stage production process using DEA. But, most previous studies on the scope of two-stage modeling are still limited to the efficiency measurement and also have neglected the progressive direction of predictive value and capacity. As an optimization technique, two-stage DEA model lacks predictive capacity. Despite an adaptive prediction model being a practical necessity, this area has rarely been addressed in the previous studies. This paper demonstrates an integrative approach to constructive performance modeling of a two-stage sequential production process by exploring predictive capacity of BPNN in conjunction with DEA. The effectiveness of our jointly integrated performance model through this study is empirically supported by its practical application to the financial banking operations across large U.S. banks. Ó 2015 Elsevier Ltd. All rights reserved.

1. Introduction Data envelopment analysis (hereafter called DEA) has been a popular performance analysis tool for decades. Since its inception in the 1970s, DEA has shown consistent growth in theoretical and empirical studies alike according to the extant literature (Emrouznejad, Parker, & Tavares, 2008; Liu, Lu, Lu, & Lin, 2013a, 2013b). Recent advancements have extended DEA to explore two-stage production processes with intermediate variables to bridge inputs and outputs (Kao & Hwang, 2014; Wang, Lu, Huang, & Lee, 2013; Zha & Liang, 2010). In a study assessing profitability and (stock) marketability of the top 55 U.S. commercial banks, Seiford and Zhu (1999) first introduced a two-stage DEA model and demonstrated the strategic interaction between initial inputs (e.g., employees, assets, and equity) and final outputs (e.g., market value, return to investors, and earnings per share) through intermediate factors such as revenues and profits. Despite each process being treated as independent, their pilot study laid a foundation for recent multi-stage production modeling for banks and other industries. A bank utilizes its resources to produce outputs such as loans, deposits, and other service activities as a production unit, and subsequently, exploits these intermediate outputs to ⇑ Corresponding author. Tel.: +1 719 290 5021; fax: +1 719 549 2909. E-mail addresses: [email protected] (H.-B. Kwon), [email protected] (J. Lee). http://dx.doi.org/10.1016/j.eswa.2015.04.062 0957-4174/Ó 2015 Elsevier Ltd. All rights reserved.

generate a profit by assuming an intermediary role (Berger & Humphrey, 1997; Piot-Lepetit & Nzongang, 2014; Wanke & Barros, 2014). At its core, bank production processes are a two-stage sequential process, thus becoming an attractive choice for two-stage DEA studies (Barros & Wanke, 2014; Du, Liang, Chen, Cook, & Zhu, 2011; Kao & Hwang, 2008; Liang, Cook, & Zhu, 2008; Wang, Huang, Wu, & Liu, 2014; Zha & Liang 2010). In two-stage DEA modeling, rigid projections from two sub-models cause conflicts in the intermediate layer, hence, relaxation of this constraint has been a challenging task for measuring overall efficiency of decision making units (DMUs). The primary focus of two-stage DEA applications still remains on the scope of efficiency measurement, but not many progressive attempts to attain predictive capacity have been made in the performance modeling related studies up to this date. Consequently, emergence of a flexible model is of significant academic interest and a practical demand. In this sense, the empirical exploration of an integrative model which retains deterministic measurement and adaptive prediction capacity is timely and meaningful. Although DEA solutions are mathematically sound in drawing optimal goals, they do not always provide implementable solutions for the strategic allocation of potential resources across different business organizations. In fact, two-stage performance modeling using standalone DEA retains the shortfall of lacking predictive capacity as in a single stage application. Furthermore, practical implementation of DEA results, frequently involves a tradeoff

H.-B. Kwon, J. Lee / Expert Systems with Applications 42 (2015) 6758–6766

process between theoretical suggestions and operational constraints of a DMU, while seeking actionable options, which demands managerial intuitions for better decision making. That is, two-stage production modeling should be further directed toward development of a practical system in support of identification of productive operational resources and achievable goals. In spite of the exploration of a new standardized paradigm for optimizing performance becoming a pragmatic research need, most existing literature rarely addresses this crucial issue. Accordingly, this study is designed to introduce an innovative and adaptive model by jointly using back propagation neural network (hereafter BPNN) and DEA model for goal oriented managerial decision making for productive operations processes. Very few studies, until now, have attempted to explore the practical implications of a combined approach of BPNN and DEA models. Additionally, the main attempt of their studies was limited to efficiency predictions on a single stage setting (Barros & Wanke, 2014; Kwon, 2014; Mostafa, 2009a; Wu, Yang, & Liang, 2006). Quite differently from previous studies, this paper explores the predictive potential of BPNN in estimating direct outputs beyond efficiency, a surrogate measure of performance. As a pilot application to a two-stage process, the proposed model enables prediction of achievable final outputs of DMUs for committed resources and intermediate outcomes in prior stages through sequential prediction modeling. In so doing, this paper presents an integrated adaptive model which facilitates incremental performance improvement, thus potentially advancing a superiority-driven DEA framework into a flexible better-practice paradigm. The remainder of this paper is organized as follows: Section 2 reviews the related literature on DEA and BPNN. Section 3 describes methodology and the empirical model for this study. Results of the empirical analysis and discussions are presented in Section 4 followed by concluding remarks in Section 5.

2. Related studies 2.1. Two-stage production process using DEA Followed by Seiford and Zhu’s (1999) initial application to the bank production process, researchers have reported prospective usage of two-stage DEA by presenting methodological advancements with supporting empirical evidence (Du et al., 2011; Liang et al., 2008; Luo, 2003; Wang et al., 2014; Wanke & Barros, 2014; Yu & Shi, 2014; Zha & Liang, 2010). These explorations added value to the bank performance literature and reinvigorated DEA applications to model multiple production processes (Berger & Humphrey, 1997; Fethi & Pasiouras, 2010; Liang, Li, Cook, & Zhu, 2011; Wang et al., 2013; Wu, Liang, Yang, & Yan, 2009). In a sequential two-stage process, the intermediate variables serve as outputs for the 1st stage and input for the 2nd stage as well. As a consequence, potential conflicts are inevitable; the 1st model increases the outputs while the 2nd model decreases the inputs to become efficient during the optimization process. For this reason, coordination between two sub-stages has been a central point in developing two-stage models (Cook, Liang, & Zhu, 2010; Kao & Hwang, 2008, 2011; Wang et al., 2013, 2014; Wanke & Barros, 2014). For example, Wang et al. (2013) focused on finding an optimal level of intermediate variables, but at the cost of reducing levels of intermediate variables such as sales to streamline two sequential processes; mathematically sound, but practically controversial. Kao and Hwang (2008, 2011) suggested an input oriented model for the 1st stage and an output oriented model for the 2nd stage to avoid misalignment, however, any inconsistency between the outputs of the 1st model and inputs of the 2nd model can break relations between the two stages.

6759

According to Kao and Hwang (2008, 2011), two-stage DEA can be classified as independent, connected, and relational model depending on treatment schemes of two sub-processes toward efficiency measurement (Chen, Cook, Li, & Zhu, 2009; Tavana & Damghani, 2014; Wang & Chin, 2010). As a superiority driven model, two-stage DEA is not capable of predicting outputs, especially incremental outputs on a two-stage setting. Indeed, lack of generalized learning and prediction capability diminish the practical utility of the model (Barros & Wanke, 2014; Kwon, 2014; Mostafa, 2009b; Wu et al., 2006). From this perspective, introduction of a general model that retains predictive capacity is a demanding and practical research agenda (Kao & Hwang, 2008, 2014). No predictive analysis of two-stage production processes has been attempted until this date of writing; the primary usage of two-stage DEA has been limited to efficiency analysis. 2.2. Integrated scheme of DEA–BPNN BPNN is an intelligent information processing technique with nonlinear pattern learning capabilities. Literature shows a growing trend in utilizing ANNs in many business sectors, but their potential value needs to be further explored. Especially in the banking and financial sector, most of the applications remain within the traditional classification problems such as credit scoring, loan evaluation, and failure predictions (Baesens, Setiono, Mues, & Vanthienen, 2003; Doumpos & Zopounidis, 2011; Ioannidis, Pasiouras, & Zopounidis, 2010; Malhotra & Malhotra, 2003; Ravi & Ravi, 2007; Tam, 1991). Very few papers have explored BPNN, as a standardized ANN model, particularly for efficiency analysis and benchmarking by jointly using DEA. Athanassopoulos and Curram (1996) were the first to investigate the possibility of using BPNN as an alternative to DEA in assessing the efficiency of banks, and proposed potential benefits of combining the two methods. Since then, a limited number of research papers have reported successful implementation of the combined DEA–BPNN method in the applications of healthcare operations (Pendharkar, 2005; Pendharkar & Rodger, 2003), supplier selection (Wu et al., 2009; Çelebi & Bayraktar, 2008), performance of the technology industry (Hsiang-Hsi, Tser-Yieth, Yung-Ho, & Fu-Hsiang, 2013; Kwon, 2014), and other tasks (Pendharkar, 2011; Santin 2008). Application on bank operations is also a rarity (Azadeh, Saberi, Moghaddam, & Javanmardi, 2011; Mostafa, 2009a; Mostafa, 2009b; Wu et al., 2006). Azadeh et al. (2011) applied BPNN to predict the efficiency of 102 bank branches. Mostafa (2009a), Mostafa (2009b) and Wu et al. (2006) used probabilistic neural network (PNN) with DEA to classify banks based upon efficiency predictions. Up to this date, only a few scholars have explored combined usage of DEA and neural networks despite the popularity and computational advantage of each method as a standalone technique. Though being rare, the majority of combined studies were devoted to the prediction of DEA efficiency scores using neural networks. These applications can be further divided into estimation and classification problems according to the implementation of neural networks. The first group focuses on estimating efficiency scores of an individual DMU (Azadeh, Saberi, & Anvari, 2010; Azadeh et al., 2011; Sreekumar & Mahapatra, 2011; Ülengin et al.; 2011; Vaninsky, 2004; Wu et al., 2009) and the second intends to classify DMUs into different categories according to efficiency levels (Mostafa, 2009a; Mostafa, 2009b; Mostafa, 2009c; Wu et al., 2006; Wu et al., 2009). Some scholars explored the opposite sequence of neural network-DEA for data processing purposes (Liao & Li, 2008; Samoilenko & Osei-Bryson, 2010; Çelebi & Bayraktar, 2008). Overall, these studies took advantage of the combined technique and reported encouraging outcomes even for small data sets. Indeed, previous applications demonstrated effectiveness of the combined approach for a wide range of DMUs including 23

6760


suppliers (Wu et al., 2009), 62 companies (Mostafa, 2009c), 162 bank branches (Wu et al., 2006), and 10,000 simulated DMUs (Emrouznejad & Shale, 2009). Most recently, Barros and Wanke (2014) reported successful application of the combined approach for insurance firms. Their model used BPNN to predict performance of insurance firms by predicting DEA efficiency and confirmed effectiveness of the combined approach. Thorough exploration on the capability of DEA–BPNN was presented by Kwon (2014). By utilizing eight year time series data from eight major smartphone providers, he demonstrated the capability of BPNN to learn production functions from various returns-to-scale assumptions (e.g., increasing-returns-to-scale, decreasing returns-to-scale, variablereturns-to scale, and super-efficiency). He further expanded the combined model to estimate efficiency frontiers by using DEA projections as inputs and promoted neural networks to learn best practice patterns. However, his efforts were centered on predicting best outputs only on a single-stage setting. Thus far, there have been very few reports that demonstrate credible evidence of combined prediction modeling on a two-stage production process. Even for single stage modeling, attempts to predict incremental outcomes are scarcely found in the literature. Therefore, this paper fills the research gap by advancing the DEA–BPNN method and the resulting model can accommodate incremental performance predictions in a two-stage production process. 3. Description of the methodology 3.1. Brief overview of DEA DEA is a nonparametric linear programming method for measuring the efficiency of DMUs, banks in this study, by formulating multidimensional input and output vectors. DEA (Charnes, Cooper, & Rhodes, 1978), as a frontier technology, bases its theoretical foundation on Farrell’s work (1957) and determines an efficient frontier by enveloping best practice DMUs. DEA, then, measures the relative distance of an individual DMU from the frontier and the relevant deviation indicates the inefficiency of a DMU. Eventually, DEA determines the potential adjustment of input and output variables for inefficient DMUs to become efficient. Assuming n-DMUs with r-input and s-output vectors, the efficiency of DMUk using the conventional Cooper-Charnes-Rhodes (CCR) model (Charnes et al., 1978) can be formulated as: s X Ek ¼ max oj Y jk j¼1

s:t

s X

oj Y jp

j¼1

,

,

r X qi X ik i¼1

r X qi X ip 1;

Xi,k (i = 1...r)

3.2. Brief overview of BPNN The artificial neural network, including BPNN, is designed to mimic the human thinking paradigm as an intelligent information processing system. BPNN as a nonparametric model does not require a priori assumptions in model building and is particularly useful for handling multidimensional input–output models under unknown complex relationships as an adaptive mining technique (Fausett, 1994; Lam, 2004; Samoilenko & Osei-Bryson, 2010). The motivation for using the BPNN model is in the exploitation of its adaptive learning and nonlinear pattern mapping capabilities by using a small subset of data. As an adaptive learning technique, a neural network allows presentation of data to the model for an update of learned information which is encoded in weights connecting neurons in a highly parallel structure. Fig. 2 illustrates a schematic diagram of a simple BPNN model with a 3-2-1 structure. As shown in the figure, BPNN learning occurs through an iterative process including input presentation, information feed forward, error calculation, and backward propagation of error for sequential weight adjustments. During the information feed forward process, neurons in hidden and output layers conduct inner products of input and weight vectors before activation of net outputs in a sequential manner. Let Vij and Wjk denote weights of hidden neuron j connecting input (i) and output (k) neurons. Then, outputs of hidden neuron J and output neuron K can be described as follows:

j¼1

! ð4Þ

ð2Þ

X Y K ¼ f ðynetK Þ ¼ f Hj WjK

!

Information flow

i¼1

, h X ~ c Z ck W c¼1

ð3Þ

ð5Þ

j

Calculate error

h X ¼ max oj Y jk

X Xi ViJ

Present input

c¼1

r X qi X ik

Output variable

i

E1k , and the subsequent 2nd stage, E2k (Kao & Hwang, 2008; Seiford & Zhu, 1999):

,

Intermediate variable

~ c are weights associated with Z ck . where, wc and W In this research, two output oriented CCR models are used as a preprocessor to subsequent BPNN modules. Detailed discussions on two-stage DEA models are found in the literature (Du et al., 2011; Kao & Hwang, 2008, 2014; Liang et al., 2008; Wang et al., 2014; Wanke & Barros, 2014; Zha & Liang, 2010).

i¼1

h X E1k ¼ max wc Z ck

Yj,k (j= 1... s)

2nd stage

Fig. 1. Two-stage sequential process.

HJ ¼ f ðynetJ Þ ¼ f

p ¼ 1; . . . ; n

Zc,k (c= 1...h)

1st stage

Input variable

ð1Þ

oj ; qi q > 0, j = 1,...,s, i = 1,. . ., r, q: a positive infinitesimal value,where, Yjp (Xip) is output (input) of DMUp and oj(qi) is a nonnegative weight. The computational scheme of a single stage process can be extended to a two-stage sequential production model by embedding h-dimensional intermediate variables, Zck, c = 1,...,h, which assume a dual role of output to the 1st stage input (Xik) and input to the 2nd stage output (Yjp) as shown in Fig. 1. Therefore, the conventional two-stage model yields the efficiencies of the 1st stage,

E2k

DMUk (k=1...p)

Error backpropagation Fig. 2. Schematic diagram of a simple BPNN model.

6761


1 1 þ ex

ð6Þ

At the end of the feed forward process, the network calculates Euclidean distance, E, between the target (Tk) and the activated (Yk) output of all training pairs, then backward error propagation ensues for weight adjustment in a way to minimize E as controlled by a fractional learning rate qðtÞ.

E¼

1X kT k Y k k2 2 k

mij ðt þ 1Þ ¼ mij ðtÞ qðtÞ

ð7Þ

@E @ mij ðtÞ

wjk ðt þ 1Þ ¼ wjk ðtÞ qðtÞ

@E @ jk ðtÞ

Process model

f ðxÞ ¼

Profit earning process

Production process

Empirical model

A nonlinear transfer function, f(.), is applied before activating net outputs of neurons in hidden and output layers, and logistic sigmoid functions are commonly used as in this research.

Employee Equity Expenses

Loans Deposits Investments

DEA 1

Profit

DEA 2 Measurement Prediction

BPNN 1

BPNN 2

ð8Þ Fig. 3. Schematic diagram: two-stage production modeling.

ð9Þ

The weight vectors after training preserve a key code to associate new incoming inputs to the closest prototype. In forecasting applications, BPNN can conduct functional approximations for unseen inputs or variations of learned inputs. BPNN, which requires only a subset of data, possesses an adaptive and generalized learning property that distinguishes it from batch type modeling where reconstruction of the model is required to accommodate new information (Lam, 2004). In this sense, BPNN is suitable for building a flexible model under varying scenarios on input–output assumptions. The BPNN model learns central tendency of the data presented to the network as contrasted to DEA which determines extreme data points. In this combined approach, BPNN learns monotonicity-preserving production patterns of DMUs in support of incremental performance improvement.

levels of intermediate variables. In so doing, the present study proposes an innovative approach for two-stage production modeling and overcomes limits of standalone DEA. The empirical data used for this study is from Uniform Bank Performance Reports (UBPRs) produced by the Federal Financial Institutions Examination Council (FFIEC) for the year of 2013. The sample data include financial information from Peer Group1 institutions as classified by the FFEIC, which represent insured commercial banks with asset size in excess of $3billion. For this study, 181 banks were used for empirical modeling and negative profit banks were not included for modeling but saved as a separate test set for prediction experiments. Table 1 shows descriptive statistics of selected variables.

4. Experimental results and discussion 4.1. DEA efficiency assessment

3.3. Design of the empirical model The proposed two-stage bank production model shows a 3-3-1 structure by forming an input layer (employees, equity, expenses), an intermediate layer (deposits, loans, and investments), and an output layer (profit), where expenses are a summation of interest and noninterest expenses and profit represents net income as a final output. In so doing, the 1st stage represents the production sub-process while the 2nd stage represents the profit earning sub-process by reflecting sequential production and intermediation approaches. Fig. 3 visualizes the design scheme of employing DEA–BPNN for this explorative modeling. In this pilot study, the scope of the research is not confined to measuring efficiency levels as in conventional DEA approaches but is extended to estimate the performance of DMUs in terms of final outputs. Consequently, the empirical model incorporates measurement and prediction modules as shown in the figure. The objective of the empirical model is to predict incremental profits in accordance with desired performance levels in a two-stage production environment. The empirical process starts by applying two independent DEA models to measure efficiency of DMUs in both stages and the resulting efficiency scores play a key role in developing sequential prediction modules. The first BPNN module is trained to predict DEA efficiency scores for given or new inputs, which serve as a target performance for the 2nd stage. The 2nd stage BPNN model is trained by the data stream of DMUs (e.g., intermediate and output variables in combination with efficiency scores) to learn the profit earning pattern of DMUs under evaluation. In this scheme, the BPNN model is designed to predict incremental outputs as a function of desired efficiency and varying

As a first step to building a two-stage performance model, two separate DEA analyses have been conducted by using the CCR-O model and the experiment results are summarized in Table 2. As mentioned earlier, three intermediate variables (i.e., loans, deposits, investments) are used as output variables for the 1st stage (production process) and input for the 2nd stage (profit earning process) as well. The results show higher efficiency scores and more efficient DMUs in the 1st stage, thus exposing relatively lower efficiency in the 2nd stage. The discrepancy between the two model outcomes indicates that most of the DMUs do not properly translate production efficiency into the equivalent level of profit earning efficiency. With this notion considered, the main focus of the present model is centered on predicting incremental final outputs for ‘inefficient’ DMUs in the 2nd stage to achieve target performance equivalent to the 1st stage in support of stepwise improvement as illustrated in Fig. 4. Fig. 4 displays two-stage efficiency plots

Table 1 Descriptive statistics of variables. Layer Input

Variables ($ in USM)

Employee Equity Expenses Intermediate Loans Deposits Investments Output Profit

Mean

SD

Maximum Minimum

8,025 6,649 1,806 30,300 46,203 12,529 603

28,767 23,771 6,739 105,086 169,608 43,196 2,449

223,040 178,693 56,033 778,519 1,326,036 333,904 19,075

222 194 57 1,098 2,035 14 8

6762


where efficiency analysis through alignment of two stages is a main drive, this study acknowledges two different production functions and the model utilizes given inputs and intermediate outputs to build a prediction model. As discussed earlier, standalone DEA cannot estimate the appropriate scale of final output (e.g. profit) to match 1st stage efficiency. Furthermore, in a situation where a DMU intends to improve 2nd stage efficiency to a certain level while maintaining scales of inputs and intermediate outputs, DEA cannot provide appropriate solutions for these what-if scenarios. With the notion of deficiencies of existing methods, the present model intends to provide feasible solutions and predict relevant final outputs of DMUs to match 1st stage efficiencies or desired performance levels. From this perspective, the empirical model proposed in this study is quite different from the conventional two-stage DEA. Through the adaptive prediction capabilities embedded in the model, this technique advances the current best oriented model into a flexible ‘‘better practice’’ paradigm.

Table 2 DEA experiment results.

1 2 3

DEA

1st Stage production process

2nd Stage profit earning process

Efficient DMUs Average efficiency Average projections (%) Employee1 Equity1 Expenses1 Loans2 Deposits2 Investments2 Profit3

11 0.680

4 0.301

26.81 (input) 0.00 (input) 4.03 (input) +59.11 (output) +56.76 (output) +79.25 (output)

1.92 (input) 4.62 (input) 11.42 (input) +313.77 (output)

Input variables. Intermediate variables. Output variable.

1.0 4.2. BPNN prediction experiments

1st stage

0.8 0.6

Average level

Target level

4.2.1. Network design and data sets selection The performance of BPNN models is determined by several design elements which include network structures (e.g. number of hidden layers and hidden neurons), learning algorithms, transfer functions, the quantity and quality of data sets, and etc. Selection of a good model is still a challenging task and involves a certain level of heuristics. In this research, a commercial software package, NeuralWare Predict (NeuralWare, 2003), was used to search for the best possible model. Although a detailed discussion of the software is beyond the scope of this paper, a brief introduction to the utilization of built-in functions and model parameters may reduce any potential burden of efforts in searching for good models. This paper follows three key design factors in considering the performance of BPNN models. First of all, network structure and learning – incremental learning is adopted to build up a proper network structure as a key design consideration, particularly in determining a suitable number of hidden neurons. The network finds the optimal number of neurons by adding a neuron and testing the network each time

0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0

2nd stage Fig. 4. Two-dimensional efficiency plot.

and indicates average (current) and equivalent (target) performance. Table 2 further exhibits average projections of each variable obtained from the independent DEA models and shows inconsistency in the intermediate layer variables, thus exposing a coordination issue prevalent in traditional two-stage DEA models (Cook et al., 2010; Kao & Hwang, 2008, 2011; Wang et al., 2013, 2014; Wanke & Barros, 2014). In contrast to traditional DEA studies

DEA1

I1

DEA2

M1

I2

ES

I3

M2

P

ES

M3

BPNN1

BPNN2 ES

I’1 I’2 I’3

M’1 M’2 M’3

Note: I1, I2, I3 : Input variables (I’1, I’2, I’3: unseen DMUs) ES: Efficiency score, P: Profit (P’: Profit predicted) M1, M2, M3: Intermediate variables (M’1, M’2, M’3: unseen DMUs) Fig. 5. Sequential flow diagram and input–output variables.

P`

6763


Network architecture

Mode

DMUs

R

MAE

MAPE

LT 15% Error

15–9-1

Train Test Valid

108 37 36

0.948 0.906 0.880

0.034 0.048 0.047

5.78 5.42 6.17

106 (98%) 37 (100%) 34 (94%)

6–30-1

Train Test Valid

108 37 36

0.959 0.954 0.870

0.029 0.035 0.049

4.23 5.20 7.30

106 (100%) 36 (97%) 33 (92%)

Output Y

Table 3 BPNN training results.

ES

Input X

Note. R: Correlation coefficient between actual and predicted efficiency scores. MAE: Mean absolute error. MAPE: Mean absolute percentage error.

Fig. 7. Monotonicity preserved by efficiency score (ES).

4.2.2. Prediction of 1st stage efficiency The sequential prediction experiments include estimation of efficiency in the 1st stage, prediction of target output in the 2nd stage, and integrated prediction on unseen DMUs. In this series of prediction experiments, DEA measurement modules are functioning as a preprocessor for the subsequent BPNN forecasting modules. In predicting efficiency of the 1st stage, the efficiency score (ES) is treated as a target output for BPNN1 as shown in Fig. 5, therefore the following functional relationships hold:

until no further improvement is made. The number of hidden layers is set to one in advance. In most nonlinear pattern learning applications, one hidden layer is deemed sufficient and a certain number of hidden neurons can properly capture complex patterns of the presented data (Fausett, 1994; Golmohammadi, 2011; Liao & Fildes, 2005). The Predict software also enables selection of learning rules, input transform, and other parameters. In this research, a gradient-decent learning rule was employed to adjust weights and sigmoid functions were used as an activation function for hidden and output neurons. Secondly, data set selection - The total data set used for this study includes 181 DMUs. For the effective training and validation of the model, the data set was split into training, test, and validation sets. The training set is used to build a model while the test set is used to monitor network learning and prevent over-training of the network. The validation data set is unseen to the trained network. In this study, 60% of the data set (108 DMUs) was used for training, 20% (37 DMUs) for testing, and 20% (36 DMUs) of the data set was saved for network validation throughout the experiment. After completion of the network validation the trained model is used to predict outputs for new DMUs which were excluded from the modeling by assuming real world scenarios. The implementation of the BPNN model in our paper includes four phases of training, testing, validation, and extended applications. Lastly, quantity and quality of the data sample - A large data set is often preferred for the generalization of BPNN models, however, small scale data sets were commonly used, especially for explorative modeling and experiments (Kwon, 2014; Wu et al., 2009; Çelebi & Bayraktar, 2008). There is no certain rule to determine the optimal size of the data for neural networks. As a rule of thumb, training data sets of 10 times the number of independent variables are considered sufficient for nonparametric modeling (Trout, Rai, & Zhang, 1995; Wu et al., 2006). Two sequential experiments in this study utilize 6 and 4 independent variables, hence, the size of the training data meet the criteria. A reliable data set is another important factor in developing adaptive learning models such as BPNN. In this approach, BPNN exploits the monotonicity preserved by DEA efficiencies which contributes to stable learning and less fluctuation of BPNN (Pendharkar & Rodger, 2003).

40%

0%

-40%

4.2.3. Prediction of 2nd stage output The next prediction experiment aims to predict the output of the 2nd stage by using three intermediate variables as inputs. The scheme of this experiment design is different from the previous stage, in that the 2nd prediction module intends to learn the

Network: 15-9-1

20%

-20%

where, I and M represent input and intermediate vectors used for DEA1. Eventually, the BPNN module learns the production function of DMUs by using three dimensional input and output variables and predicts the scalar value of efficiency. Two BPNN models were implemented for this experiment. The first model utilizes an input transform function supported by the Predict software and resulted in a 15-9-1 network. The second model was built on original inputs without input transform and yielded a 6-30-1 structure, by generating more hidden neurons. The performance of both models is comparable with a marginal improvement in the 15-9-1 model for the validation data set in terms of performance statistics such as R (correlation between actual and predicted efficiency), mean absolute error (MAE), and mean absolute percentage error (MAPE). From a computational perspective, the first model can be considered more economic with less connected weights. The training results summarized in Table 3 demonstrate the strong prediction capabilities of the BPNN models both on seen and unseen data. Fig. 6 further exhibits the strong predictive potential of the BPNN model with most of DMUs showing less than 15% prediction errors. Interestingly, bidirectional error patterns reveal characteristics of the BPNN model to learn central tendency of data.

1

61

121

181

DMUs (sorted by error scale)

Prediction error

Prediction error

40%

ES ¼ f ðIemployees ; Iequity ; Iexpenses ; Mloans ; Mdeposits ; Minv estments Þ

Network: 6-30-1

20% 0% 1 -20%

61

121

181

DMUs (sorted by error scale)

-40% Fig. 6. Efficiency prediction results (1st stage).

6764


Table 4 BPNN learning results. ES

Network structure

Mode

DMUs

R

MAE

MAPE

LT 15% Error

Included

15–3-1

Train Test Valid

108 37 36

0.996 0.998 0.998

87,972 16,415 36,085

8.96 5.91 9.62

93 (86%) 34 (92%) 29 (81%)

Excluded

15–3-1

Train Test Valid

108 37 36

0.994 0.950 0.977

156,595 60,699 106,882

41.0 36.9 63.1

37 (34%) 17 (46%) 11 (31%)

Note. R: Correlation coefficient value between actual and predicted profits. MAE: Mean absolute error. MAPE: Mean absolute percentage error.

Prediction error

1000% w. ES

800%

wo. ES

600% 400% 200% 0% -200%

1

21

41

61

81

101 121 141 161 181

if ES1 > ES2. This monotonicity preserving property can be a precondition to build a reliable prediction model, which minimizes fluctuations and improves predictability (Archer & Wang, 1993; Pendharkar, 2005). Fig. 7 shows a simple illustration of monotonicity preserved between univariate input (X) and output (Y) as controlled by ES, a desired level of efficiency in this study setting. Table 4 summarizes the training and test results of BPNN for this prediction modeling. To demonstrate the effect of monotonicity, the BPNN output is compared with the result obtained from the model which does not include ES as part of training inputs. The results show a remarkable difference between the two models. The model output obtained from monotonicity preserving data demonstrates superior performance with far less error scales. Fig. 8 further visualizes the impact of monotonicity on BPNN learning. Advantages of using monotonic data were previously proposed by Pendharkar and Rodger (2003) and Wu et al. (2006). In their approach, they selected a subset of DMUs with relatively higher efficiency scores for neural network training. Consequently, DEA’s role was limited to prescreening of a training subset, which is ‘approximately’ monotonic, thus reducing the availability of data for network training. In contrast, by incorporating efficiency score as a key variable, the presented method enables full utilization of training data and is beneficial for model building with limited data sets.

DMUs (sorted by scale) Note. w. ES: monotonicity preserved (ES included for trainning) wo.ES: monotonicity not preserved (ES excluded from training) Fig. 8. Impact of monotonicity on BPNN learing.

efficiency frontier and approximate subsequent frontiers represented by efficiency scores. Functional representation of the prediction scheme can be expressed as follows:

Pprofit ¼ f ðMloans ; Mdeposits ; Minv estments ; ESÞ where, Pprofit and M represent a single dimensional output of profit and three dimensional intermediate variables and ES denotes efficiency score. In this process, ES is a key variable which establishes a monotonic relationship between input and output vectors. For all efficient DMUs holding ES = 1, f () is considered a monotonicity preserving function, where f (X1) > f (X2) for two single dimensional inputs if X1 > X2. Therefore, for DMUs holding the same level of inputs, the functional relationship of f (ES1) > f (ES2) still holds true

4.2.4. Integrated prediction for unknown targets The empirical results presented thus far signify promising usage of neural networks in conjunction with DEA for modeling a two-stage sequential production process. To further demonstrate performance of the proposed method, an additional experiment has been conducted by using 13 banks with negative profit, which were excluded from the previous DEA experiments. The experiment aims to confirm the capability of the model to streamline the two-stage production process assuming a stepwise improvement scenario for the underperforming and unseen DMUs. For these new test cases, the proposed model performs sequential tasks as shown in Fig. 5: (1) 1st BPNN module estimates 1st stage efficiency of DMUs. (2) 1st stage efficiency is set for a target goal for 2nd stage performance. (3) 2nd BPNN module predicts profit level relevant to the target goal. (4) 2nd BPNN module predicts incremental profits for desired targets.

Table 5 Combined prediction outcomes for stepwise improvement. DMUs

Profit

Emp

Eqt

Exp

Lns

Dps

Inv

Prediction results 1st stage: ES

RBS FBP DRB CAN BNM FMB SNB GFB BBC BBG EVB UCB AMB

(3,000) (154) (80) (64) (56) (16) (9) (7) (3) (2) (2) (2) (1)

15,595 2,424 1,327 690 310 472 657 167 506 474 315 334 421

15,450 1,404 700 153 224 33 317 36 172 58 107 137 306

6,583 513 405 154 118 77 142 46 101 75 51 94 152

70,853 9,620 6,065 864 415 560 2,137 604 1,317 615 657 642 1,963

71,716 9,921 5,022 1,470 965 926 2,639 889 1,408 933 870 1,257 2,580

Note. Emp: No. of employees, Eqt: Equity, Exp: Expenses, Lns: Loans, Dps: Deposits, Inv: Investments.

10,104 1,978 204 275 112 123 441 274 150 166 270 322 1,451

0.453 0.670 0.569 0.535 0.537 0.668 0.647 0.788 0.636 0.679 0.772 0.653 0.656

2nd stage: profit ES1/3

ES2/3

ES1

493 117 46 15 15 19 26 22 17 19 22 19 28

1,014 214 76 26 24 27 48 34 29 28 33 29 58

1,486 395 126 34 32 38 79 45 43 39 45 42 99


Table 5 presents the test data of the 13 DMUs represented by their initials and summarizes the overall prediction results. As discussed, the BPNN model first predicts the efficiency of the DMUs in the 1st stage by using three inputs and three outputs. The BPNN model then predicts the profit level required for each DMU to achieve its target performance (denoted as ES1). It also predicts the incremental profits necessary to attain desired performance levels (denoted as ES1/3, ES2/3). For example, the 1st BPNN module predicts the efficiency of RBS, which marked negative profit of 3,000$USM, as 0.453. Then, the 2nd BPNN module predicts incremental profit of 493 (1,014, 1,486) for the DMU to achieve 1/3 (2/3, equivalent) level of 1st stage performance. The table shows incremental outputs in accordance with the increasing performance targets across all DMUs. The present model proposes a streamlined process of two-stage production modeling and the empirical results evidently support effectiveness of the presented adaptive model. In its core, the proposed model integrates measurement and prediction frameworks, accommodates seen and unseen data, and forecasts best and better outputs. The demonstrated empirical results affirm the effectiveness of the BPNN model in complementing and augmenting deterministic DEA models. The embedded flexibility and hypothetical test capacity demonstrated throughout the experiments adds meaningful features to the proposed method and can support decision making processes during implementation of improvement initiatives for banks and other industries as well.

5. Concluding remarks The primary purpose of this study is to explore an innovative model to streamline a two-stage production process, thus enabling incremental performance predictions. As an optimization tool, two-stage DEA has shown a limited scope of application for efficiency analysis. Moreover, the lack of predictive capacity has been a serious drawback of DEA, which hinders advancement of the method toward a more generalized and flexible platform in performance modeling. Combining the complementary features of BPNN with DEA, this paper demonstrates an exploratory finding of an innovative two-stage production modeling through an empirical application to 181 large U.S. banks. In a practical sense, the proposed model can drastically enhance the managerial decision making process as a pragmatic and innovative technique. Although managers may frequently rely on quantitative tools such as DEA in their initiatives to promote performance improvement, their solutions are not always actionable (Mostafa, 2009b, 2009c, Wu et al., 2006) mainly due to limitations of practical implementation. Rather, available options for the pursuit of improvement are contingent on firm specific factors and environmental constraints. From this perspective, a sound methodology which can provide managers with tradeoff analysis capabilities is a pragmatic demand. In an empirical modeling of a two-stage process across large U.S. banks, the operational capability to estimate appropriate final output (e.g. profit) for committed inputs (e.g. employee, equity, expenses) and intermediate outputs (e.g., loans, deposit, investments) is crucial for the initiation of improvement options. In addition, the operational capacity to predict optimal outputs using hypothetical inputs and other intermediate factors is a key requirement for setting improvement goals and monitoring progress under changing scenarios. The proposed DEA–BPNN model provides plausible solutions for this practical necessity which standalone DEA models cannot accommodate. Nonetheless, a series of intriguing questions has risen: ‘What is the appropriate final output for non-profit earning DMUs?’ ‘Is the estimated profit an achievable goal? If not, what are the actionable small goals?’ By utilizing the 13 banks which were excluded from the DEA

6765

analysis, the proposed model demonstrated its desirable operational capability to answer these questions. In other words, the proposed model is capable of dealing with unseen DMUs through its predictive capacity. Thus, our newly proposed model can support managers in their pursuit of performance improvement. The essence of the proposed model is in its capacity to accommodate stepwise performance improvement setting achievable targets through adaptive prediction capabilities. The methodological innovations and contributions through this paper can also be applied to the development of robust strategic paradigms for performance improvement, particularly through an integration of a measurement and prediction model, streamlined predictions in support of incremental performance improvement, goal setting in terms of direct output besides indirect measurement of efficiency, adaptive learning and flexible modeling to deal with unseen inputs, and accommodation of negative income DMUs. This study further contributes to the theoretical foundation for the conceptual development of better performance benchmarking while enriching benchmarking contexts by expanding from ‘single best way’ to ‘good or better ways’ (Francis & Holloway, 2007). Like other studies, the major findings of this paper are not warranted without limitations that may preserve further advancement in the future. In this paper, the objective of the model design was limited to the prediction of final outputs leveraged by 1st stage efficiency in a two-stage production process. Therefore, this pilot study can invite further scrutiny of the method and extended applications. First, a potential opportunity of advancing this study would be to build an integrative model to predict both intermediate and final outputs in sequence. In particular, BPNN can be implemented to learn patterns of efficient frontiers of both production processes and also to estimate data envelopment surfaces (Kwon, 2014). This input oriented prediction can help determine the proper level of resources and their impact on sequential outputs. Second, expansion of the proposed model for other industries or different production processes will open the door for a promising research agenda and efforts to develop a more constructive empirical model. For example, expansion of the model into R&D value chain analysis by using R&D variables and market performance as sequential outputs can contribute to the generation of more interesting results (Wang et al., 2013) across different industry settings. Third, the proposed prediction scheme can be further expanded by integrating classification techniques (Wu et al., 2006). In other words, BPNN can be implemented to learn and predict target classes and relevant outputs for DMUs as well. The resulting model would be capable of selecting benchmark targets and predicting performance goals, thus covering two crucial aspects of the benchmarking process. Fourth, DEA can be used as a clustering preprocessor (Po, Guh, & Yang, 2009) for BPNN modules. By implementing an integrated model of DEA and BPNN, BPNN can be capable of generating synergistic effects by identifying peer DMUs and estimating expected outputs in terms of average performance. As the model and data are improved, the integrated model in this paper can also be extended with different neural network models such as probabilistic neural network to explore a robust synergistic performance prediction model.

References Archer, N., & Wang, S. (1993). Application of the back propagation neural network algorithm with monotonicity constraints for two-group classification problems. Decision Sciences, 24(1), 60–75. Athanassopoulos, A. D., & Curram, S. P. (1996). A comparison of data envelopment analysis and artificial neural networks as tools for assessing. Journal of the Operational Research Society, 47(8), 1000–1016. Azadeh, A., Saberi, M., & Anvari, M. (2010). An integrated artificial neural network algorithm for performance assessment and optimization of decision making units. Expert Systems with Applications, 36(8), 5688–5697.

6766


Azadeh, A., Saberi, M., Moghaddam, R. T., & Javanmardi, L. (2011). An integrated data envelopment analysis-artificial neural network-rough set algorithm for assessment of personnel efficiency. Expert Systems with Applications, 38(3), 1364–1373. Baesens, B., Setiono, R., Mues, C., & Vanthienen, J. (2003). Using neural network rule extraction and decision tables for credit-risk evaluation. Management Science, 49(3), 312–329. Barros, C., & Wanke, P. (2014). Insurance companies in Mozambique: A two-stage DEA and neural networks on efficiency and capacity slacks. Applied Economics, 46(29), 3591–3600. Berger, A. N., & Humphrey, D. B. (1997). Efficiency of financial institutions: International survey and directions for future research. European Journal of Operational Research, 98(2), 175–212. Çelebi, D., & Bayraktar, D. (2008). An integrated neural network and data envelopment analysis for supplier evaluation under incomplete information. Expert Systems with Applications, 35(4), 1698–1710. Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the efficiency of decision making units. European Journal of Operational Research, 2(6), 429–444. Chen, Y., Cook, W. D., Li, N., & Zhu, J. (2009). Additive efficiency decomposition in two-stage DEA. European Journal of Operational Research, 196(3), 1170–1176. Cook, W. D., Liang, L., & Zhu, J. (2010). Measuring performance of two-stage network structures by DEA: A review and future perspective. Omega, 38(6), 423–430. Doumpos, M., & Zopounidis, C. (2011). A Multicriteria Outranking modeling approach for credit rating. Decision Sciences, 42(3), 721–742. Du, J., Liang, L., Chen, Y., Cook, W. D., & Zhu, J. (2011). A bargaining game model for measuring performance of two-stage network structures. European Journal of Operational Research, 210(2), 390–397. Emrouznejad, A., Parker, B., & Tavares, G. (2008). Evaluation of research in efficiency and productivity: A survey and analysis of the first 30 years of scholarly literature in DEA. Socio-Economic Planning Sciences, 42, 151–157. Emrouznejad, A., & Shale, E. (2009). A combined neural network and DEA for measuring efficiency of large scale data sets. Computers & Industrial Engineering, 56(1), 249–254. Fausett, L. (1994). Fundamentals of neural networks: Architectures, algorithms, and applications. Englewood Cliffs, NJ: Prentice Hall. Fethi, M. D., & Pasiouras, F. (2010). Assessing bank efficiency and performance with operational research and artificial intelligence techniques: A survey. European Journal of Operational Research, 204(2), 189–198. Francis, G., & Holloway, J. (2007). What have we learned? Themes from the literature on best-practice benchmarking. International Journal of Management Reviews, 9(3), 171–189. Golmohammadi, D. (2011). Neural network application for fuzzy multi-criteria decision making problems. International Journal of Production Economics, 131, 490–504. Hsiang-Hsi, L., Tser-Yieth, C., Yung-Ho, C., & Fu-Hsiang, K. (2013). A comparison of three-stage DEA and artificial neural network on the operational efficiency of semi-conductor firms in Taiwan. Modern Economy, 4(1), 20–31. Ioannidis, C., Pasiouras, F., & Zopounidis, C. (2010). Assessing bank soundness with classification techniques. Omega, 38(5), 345–357. Kao, C., & Hwang, S. (2008). Efficiency decomposition in two-stage data envelopment analysis: An application to non-life insurance companies in Taiwan. European Journal of Operational Research, 185(3), 418–429. Kao, C., & Hwang, S. (2011). Decomposition of technical and scale efficiencies in two-stage production systems. European Journal of Operational Research, 211(3), 515–519. Kao, C., & Hwang, S. (2014). Multi-period efficiency and Malmquist productivity index in two-stage production systems. European Journal of Operational Research, 232(3), 512–521. Kwon, H. (2014). Performance modeling of mobile phone providers: A DEA-ANN combined approach. Benchmarking: An International Journal, 21(6), 1120–1144. Lam, M. (2004). Neural network techniques for financial performance prediction: Integrating fundamental and technical analysis. Decision Support Systems, 37, 567–581. Liang, L., Cook, W. D., & Zhu, J. (2008). DEA Models for Two-Stage Processes: Game Approach and Efficiency Decomposition. Naval Research Logistics, 55(7), 643–653. Liang, L., Li, Z.-Q., Cook, W. D., & Zhu, J. (2011). Data envelopment analysis efficiency in two-stage networks with feedback. IIE Transactions, 43(5), 309–322. Liao, K.-P., & Fildes, R. (2005). The accuracy of a procedural approach to specifying feedforward neural networks for forecasting. Computers & Operations Research, 32, 2151–2169. Liao, H., & Li, Z. (2008). Multiobjective design of equivalent accelerated life testing plans. International Journal of Reliability, Quality & Safety Engineering, 15(6), 515–538. Liu, J. S., Lu, L. Y., Lu, W., & Lin, B. Y. (2013a). A survey of DEA applications. Omega, 41(5), 893–902.

Liu, J. S., Lu, L. Y., Lu, W., & Lin, B. Y. (2013b). Data envelopment analysis 1978–2010: A citation-based literature survey. Omega, 41(1), 3–15. Luo, X. (2003). Evaluating the profitability and marketability efficiency of large banks: An application of data envelopment analysis. Journal of Business Research, 56(8), 627–635. Malhotra, R., & Malhotra, D. (2003). Evaluating consumer loans using neural networks. Omega, 31(2), 83. Mostafa, M. M. (2009a). A probabilistic neural network approach for modeling and classifying efficiency of GCC banks. International Journal of Business Performance Management, 11(3), 236–258. Mostafa, M. M. (2009b). Modeling the efficiency of top Arab banks: A DEA-neural network approach. Expert Systems with Applications, 36(1), 309–320. Mostafa, M. (2009c). Modeling the competitive market efficiency of Egyptian companies: A probabilistic neural network. Expert Systems with Applications, 36(5), 8839–8848. NeuralWare. (2003). Neuralware Predict manual, Pittsburg, PA. Pendharkar, P. (2011). A hybrid radial basis function and data envelopment analysis neural network for classification. Computers & Operations Research, 38(1), 256–266. Pendharkar, P. C., & Rodger, J. A. (2003). Technical efficiency-based selection of learning cases to improve forecasting accuracy of neural networks under monotonicity assumption. Decision Support Systems, 36(1), 117–136. Pendharkar, P. (2005). A data envelopment analysis-based approach for data preprocessing. IEEE Transactions on Knowledge and Data Engineering, 17b(10), 1379–1388. Piot-Lepetit, I., & Nzongang, J. (2014). Financial sustainability and poverty outreach within a network of village banks in Cameroon: A multi-DEA approach. European Journal of Operational Research, 234(1), 319–330. Po, R.-W., Guh, Y. Y., & Yang, M.-S. (2009). A new clustering approach using data envelopment analysis. European Journal of Operational Research, 199, 276–284. Ravi, K. P., & Ravi, V. (2007). Bankruptcy prediction in banks and firms via statistical and intelligent techniques – a review. European Journal of Operational Research, 180(1), 1–28. Samoilenko, S., & Osei-Bryson, K.-M. (2010). Determining sources of relative inefficiency in heterogeneous samples: Methodology using cluster analysis, DEA and neural networks. European Journal of Operational Research, 206(2), 479–487. Santin, D. (2008). On the approximation of production functions: A comparison of artificial neural networks frontiers and efficiency techniques. Applied Economics Letters, 15(8), 597–600. Seiford, L. M., & Zhu, J. (1999). Profitability and marketability of the top 55 U.S. commercial banks. Management Science, 45(9), 1270–1288. Sreekumar, S., & Mahapatra, S. S. (2011). Performance modeling of Indian business schools: A DEA-neural network approach. Benchmarking: An International Journal, 18(2), 221–239. Tam, K. Y. (1991). Neural network models and the prediction of bank bankruptcy. Omega, 19(5), 429–445. Tavana, M., & Damghani, K. (2014). A new two-stage Stackelberg fuzzy data envelopment analysis model. Measurement, 53, 277–296. Trout, M. D., Rai, A., & Zhang, A. (1995). The potential use of DEA for credit applicant acceptance systems. Computers and Operations Research, 4, 405–408. Ülengin, F., Kabak, Ö., Önsel, S., Aktas, E., & Parker, B. R. (2011). The competitiveness of nations and implications for human development. Socio-Economic Planning Sciences, 45(1), 16–27. Vaninsky, A. (2004). Combining data envelopment analysis with neural networks: Application to analysis of stock prices. Journal of Information and Optimization Sciences, 25(3), 589–611. Wang, Y.-M., & Chin, K.-S. (2010). Some alternative DEA models for two-stage process. Expert Systems with Applications, 37(12), 8799–8808. Wang, K., Huang, W., Wu, J., & Liu, Y. (2014). Efficiency measures of the Chinese commercial banking system using an additive two-stage DEA. Omega, 44, 5–20. Wang, C., Lu, Y., Huang, C., & Lee, J. (2013). R&D, productivity, and market value: An empirical study from high-technology firms. Omega, 41(1), 143–155. Wanke, P., & Barros, C. (2014). Two-stage DEA: An application to major Brazilian banks. Expert Systems with Applications, 41(5), 2337–2344. Wu, J., Liang, L., Yang, F., & Yan, H. (2009). Bargaining game model in the evaluation of decision making units. Expert Systems with Applications, 36(3), 4357–4362. Wu, D., Yang, Z., & Liang, L. (2006). Using DEA-neural network approach to evaluate branch efficiency of a large Canadian bank. Expert Systems with Applications, 31(1), 108–115. Yu, Y., & Shi, Q. (2014). Two-stage DEA model with additional input in the second stage and part of intermediate products as final outputs. Expert Systems with Applications, 41(15), 6570–6574. Zha, Y., & Liang, L. (2010). Two-stage cooperation model with input freely distributed among the stages. European Journal of Operational Research, 205(2), 332–338.