A percentile transition ranking algorithm applied to

0 downloads 0 Views 391KB Size Report
A percentile transition ranking algorithm applied to binarization of continuous swarm intelligence metaheuristics. José Garcıa1,2, Broderick Crawford2, Ricardo ...
A percentile transition ranking algorithm applied to binarization of continuous swarm intelligence metaheuristics Jos´e Garc´ıa1,2 , Broderick Crawford2 , Ricardo Soto2 , and Gino Astorga2,3 1

2

Telef´ onica Investigaci´ on y Desarrollo, Santiago, Chile [email protected], Pontificia Universidad Cat´ olica de Valpara´ıso, Valpara´ıso, Chile [email protected], [email protected] 3 Universidad de Valpara´ıso, 2361864 Valpara´ıso, Chile [email protected]

Abstract. The binarization of continuous swarm-intelligence metaheuristics is an area of great interest in operational research. This interest is mainly due to the application of binarized metaheuristics to combinatorial problems. In this article we propose a general binarization algorithm called Percentil Transition Ranking Algorithm (PTRA). PTRA uses the percentile concept as a binarization mechanism. In particular we apply this mechanism to the Cuckoo Search metaheuristic to solve the Set Covering Problem (SCP). We provide necessary experiments to investigate the role of key ingredients of the algorithm. Finally to demonstrate the efficiency of our proposal, Set Covering benchmark instances of the literature show that PTRA competes with the state-of-the-art algorithms. Keywords: Combinatorial Optimization; Set covering Problem; Binary Metaheuristics; Percentile Ranking.

1

Introduction

In recent years, the areas of physics and particle intelligence have generated a large number of metaheuritic algorithms [4]. These metaheuristics are suitable for solving a broad class of complex optimization problems. Examples of these algorithms are Firefly Algorithm (FA) [24], magnetic optimization [22], gravitational search algorithm (GSA) [19], Cuckoo Search Algorithm (CSA)[25], Particle Swarm Optimization (PSO) [15]. Part of the success of modern metaheuristics is because they simple to understand and to implement given their nature inspired behaviour. However many of these algorithms are specifically designed to solve continuous problems. On the other hand combinatorial problems arise in many areas of computer science and application domains. For example in protein structure prediction, grouping routing, planning, scheduling and timetabling problems to mention only some examples. It is natural to try to apply these algorithms inspired by physics and particle intelligence in combinatorial problems. In the process of

2

Jos´e Garc´ıa et al.

adaptation, a series of difficulties arise when moving from continuous spaces to discrete spaces. Examples of these difficulties are spacial disconnect, hamming cliffs, loss of precision and curse of dimension [17]. This has the consequence that binarizations are not always effective and efficient [16, 7]. In this paper, a Percentile Transition Ranking Algorithm (PTRA) is proposed as a general mechanism to binarize continuous metaheuristics. This algorithm is composed of two operators. The main operator corresponds to the percentile ranking transition operator. This operator transfers the exploration / exploitation capabilities of continuous metaheuristics to their discrete or binary adapted version. The second operator corresponds to a perturbation operator. The main goal of this work corresponds to evaluate our binarization algorithm when dealing with an well-known NP-hard combinatorial optimization problem such as the SCP. To develop the evaluation, we used the Cuckoo Search metaheuristic. Experiments were developed that shed light on the contribution of the different operators to the effectiveness of the algorithm. Moreover our algorithm was compared with algorithms that use a specific Teaching-learning binarization [18], a transfer function Binary Shuffled Frog Leaping Algorithm (BSFLA) [6], and a Lagrangian-based heuristic algorithm [3]. For this purpose we use tests problems from the OR-Library1 and the instances from Balas and Carrera [1]. The numerical results show that PTRA achieves highly competitive results. The remainder of this paper is organized as follows. Section 2 briefly introduces the Set Covering problem. In Section 3 we explain the transition ranking binarization algorithm. The results of numerical experiment are presented in Section 4. Finally we provide the conclusions of our work.

2

Set Covering Problem

The SCP is one of the oldest and most studied optimization problems. SCP is well-known to be NP-hard [10]. Nevertheless, different algorithms for solving it have been developed. There exist exact algorithms who generally rely on the branch-and-bound and branch-and-cut methods to obtain optimal solutions. These methods, however, need an effort for solving an SCP instance that grows exponential with the problem size. Then, even medium-sized problem instances often become intractable and cannot be solved any more using exact algorithms. To overcome this issue, the use of different heuristics have been proposed. For example, [13] presented a number of greedy algorithms based on a Lagrangian relaxation (called the Lagrangian heuristics), Caprara et al. [3] introduced relaxation-based Lagrangian heuristics applied to the set covering problem. Metaheuristics also have been applied to solve SCP, some examples are genetic algorithm [26], simulated annealing [2] and ant colony optimization [23]. More recently swarm based metaheuristics as cuckoo search [20],black hole [8], and a Meta-optimization approach [5] were also proposed. 1

OR-Library: http://www.brunel.ac.uk/ mastjjb/jeb/orlib/mknapinfo.html.

PTRA

3

SCP has many practical applications in engineer, e.g., vehicle routing, facility location, railway, and airline crew scheduling [14, 12, 21, 5] problems. The SCP can be formally defined as follows. Let A = (aij ), be a n × m zeroone matrix, where a column j cover a row i if aij = 1, besides a column j is associated with a non-negative real cost cj . Let I = {1, ..., n} and J = {1, ...m}, be the row and column set of A, respectively. The SCP consists in searching a minimum cost subset S ⊂ J for which every row i ∈ I is covered by at least one column j ∈ J, i.e,: Minimize f (x) =

m X

cj x j

(1)

j=1

Subject to

m X

aij xj ≥ 1, ∀i ∈ I, and xj ∈ {0, 1}, ∀j ∈ J

(2)

j=1

where xj = 1 if j ∈ S, xj = 0 otherwise.

3

Percentile transition ranking algorithm

The application of statistics and data mining, can be found in many areas such as transports, smart cities, agriculture and computational intelligence [7, 9, 11]. In this section, we explore the application of the percentile concept to the binarization of continuous metaheuristics. The Proposed PTRA algorithm has three modules. The first module corresponds to the initialization of the feasible solutions. Once the initialization of the particles is performed, it is consulted if the detention criterion is satisfied. This criterion includes a maximum of iterations. Subsequently if the criterion is not satisfied, the percentile transition ranking operator is executed (section 3.1). This module is responsible for performing the iteration of solutions. Once the transitions of the different solutions are made, we compare the resulting solutions with the best solution previously obtained. In the event that a superior solution is found, this replaces the previous one. Finally having met a number of iterations where there has not been a replacement for the best solution, a perturbation operator is used. The general algorithm scheme is detailed in figure 1. In the following subsection we will explain in detail the percentile transition ranking operator. A detail explanation of the other operators will be left for an extended version. 3.1

Percentile transition ranking operator

Considering that our metaheuristic is a continuous and swarm intelligence. Due to its iterative nature, it needs to update the position of particles at each iteration. When the metaheuristic is continuous, this update is performed in Rn space. In equation 3, the update position is presented in a general manner. The x(t + 1) variable represents the x position of the particle at time t+1. This position is obtained from the position x at time t plus a ∆ function calculated at

4

Jos´e Garc´ıa et al.

Fig. 1. Flowchart of the percentile transition ranking algorithm.

time t+1. The function ∆ is proper to each metaheuristic and produces values in Rn . For example in Cuckoo Search ∆(x) = α ⊕ Levy(λ)(x), in Black Hole ∆(x) = rand × (xbh (t) − x(t)) and in the Firefly, Bat and PSO algorithms ∆ can be written in simplified form as ∆(x) = v(x). x(t + 1) = x(t) + ∆t+1 (x(t))

(3)

In the percentile transition ranking operator, we considering the movements generated by the metaheuristic in each dimension for all particles. ∆i (x) corresponds to the magnitude of the displacement ∆(x) in the i-th position for the particle x. Subsequently these displacement are grouped using ∆i (x), the magnitude of the displacement. This grouping is done using the percentile list. In our case the percentile list used the values {20, 40, 60, 80, 100}. The percentile operator has as entry the parameters percentile list (percentileList) and the list of values (valuesList). Given an iteration, the list of values corresponds to the magnitude ∆i (x) of the displacements of the particles in each dimension. As a first step the operator uses the valueList and obtains the values of the percentiles given in the percentileList. Later, each value in the valueList is assigned the group of the smallest percentile to which the value belongs. Finally, the list of the percentile to which each value belongs is returned (percentileGroupValue). The algorithm is shown in 1. A transition probability through the function Ptr is assigned to each element of the valueList. This assignment is done using the percentile group assigned to each value. For the case of this study, we particularly use the Step function given in Equation 4.

PTRA

Ptr (xi ) =



0.1, if xi ∈ group {0, 1} 0.5, if xi ∈ group {2, 3, 4}

5

(4)

Afterwards the transition of each particle is performed. In the case of Cuckoo search the Equation 5 is used to perform the transition, where x ˆi is the complei ment of x . Finally, each solution is repaired using the repair operator.  i x ˆ (t), if rand < Ptg (xi ) xi (t + 1) := (5) xi (t), otherwise

Algorithm 1 percentile ranking operator 1: 2: 3: 4: 5: 6: 7: 8:

4

Function percentileRankingTransition(valueList, percentileList) Input valueList, percentileList Output percentileGroupValue percentileValue = getPercentileValue(valueList, percentileList) for each value in valueList do percetileGroupValue = getPercentileGroupValue(percentileValue,valueList) end for return percetileGroupValue

Results

In this section, we present computational experiments with the proposed PTRA. For the execution of the instances we use a PC with windows 10, Intel Core i74770 processor with 16GB in RAM, and programmed in Python 2.7. Experiments were developed to analyze the key ingredients of our algorithm. To perform the statistical analysis in this study, the non-parametric test of wilcoxon signed-rank test and And violin charts are used. The analysis is performed by comparing the dispersion, median and the interquartile range of the distributions. Finally, comparisons were made with different algorithms, a Binary Teaching-Learning Based Optimization [18], a Binary Shuffled Frog Leaping Algorithm (BSFLA) [6], and a Lagrangian-based heuristic algorithm [3] were selected. 4.1

Insight of PTRA algorithm

In this section we investigate some important ingredients of PTRA to get insight into the behavior of the proposed algorithm. To carry out this comparison the the instances from Balas and Carrera were chosen. The contribution of the percentile transition ranking operator on the final performance of the algorithm was studied. The contribution of the perturbation operator will be developed in an extended version. To compare the distributions of the results of the different experiments we use violin Chart. The horizontal axis corresponds to the problems. While Y axis uses the measure % - Gap defined in equation 6

6

Jos´e Garc´ıa et al.

% − Gap = 100

SolutionV alue − BestKnown BestKnown

(6)

Furthermore, a non-parametric test, Wilcoxon signed-rank test is carried out to determine if the results of PTRA with respect to other algorithms have significant difference or not. The parameter settings and browser ranges are shown in Table 1.

Table 1. Setting of parameters for Cuckoo Search Algorithm Parameters

Description

ν

Coefficient for the perturbation operator N Number of Nest G Number of percentiles γ Step Length κ Levy distribution parameter Iteration Number Maximum iterations

Value

Range

20%

[15, 20, 25]

20 5 0.01 1.5 500

[15, 20, 25] [4,5,6] [0.009,0.01,0.011] [1.4,1.5,1.6] [500]

Evaluation of percentile transition ranking operator: To evaluate the contribution of the percentile transition ranking operator to the final result, we designed a random operator. This random operator executes the transition with a fixed probability (0.5) without considering the ranking of the particle in each dimension. In the evaluation the perturbation operator is included. PTRA corresponds to our standard algorithm. 05.pe is the random variant that includes the perturbation operator. When we compared the best values and averages between PTRA and 05.pe algorithms in table 2. PTRA outperforms to 05.pe in all problems. The comparison of distributions is shown in figure 2. We see the dispersion of the 05.pe distributions are bigger than the dispersions of PTRA. In particular this can be appreciated in the problems AA06, AA11, AA13, AA15, AA16 and AA19. When the interquartile ranges are compared, it is observed that the PTRA ranges are closer to 0 than those of 05.pe. All this suggests that the percentile transition ranking operator together with perturbation operator, contribute to improve the precision and the quality of the results. When we evaluate the behaviour of the algorithms through the Wilcoxon test, this indicates that there is a significant difference between the two algorithms. 4.2

PTRA Comparisons

In this section, we describe the comparisons that were perform to evaluate our algorithm. Two groups of problems were chosen. The first group was obtained from the OR-library. The TBLO [18] and BSFLA [6] algorithms were chosen to perform the comparison. The second group corresponds to the instances of

PTRA

7

Fig. 2. Evaluation of percentile transition operator with perturbation operator

Table 2. Balas and Carrera instances Instance

row

col

Density

AA03 AA04 AA05 AA06 AA11 AA12 AA13 AA14 AA15 AA16 AA17 AA18 AA19 AA20

106 106 105 105 271 272 265 266 267 265 264 271 263 269

8661 8002 7435 6951 4413 4208 4025 3868 3701 3558 3425 3314 3202 3095

4.05% 4.05% 4.05% 4.11% 2.53% 2.52% 2.60% 2.50% 2.58% 2.63% 2.61% 2.55% 2.63% 2.58%

33155 34573 31623 37464 35384 30809 33211 33219 34409 32752 31612 36782 32317 34912

BUS1 BUS2

454 681

2241 9524

1.88% 0.51%

Average p-value

Best PTRA Known[3] (best)

PTRA (avg)

std

Time (Secs)

05.pe (best)

05.pe (avg)

std

33155 34573 31623 37464 35392 31036 33712 33530 34508 32927 31714 37043 32725 35431

33335.1 34573.0 31623.0 37479.5 35463.4 31136.1 33842.7 33694.8 34673.8 33011.6 31776.8 37100.2 32772.1 35466.0

182.3 0 0 30.5 72.2 133.1 122.2 137.2 118.3 94.6 72.1 73.8 34.9 64.3

127 138 146 144 186 136 141 122 126 135 143 134 125 137

33468 34573 31623 37468 35596 31099 33829 33664 34508 32927 31743 37103 32832 35627

33555.5 34573.0 31623.0 37684.7 35879.0 31427.7 34129.3 33897.7 34755.2 33204.6 31925.2 37322.8 32960.0 35674.4

53.7 0 0 107.6 174.9 163.6 203.9 148.5 128.4 126.1 97.8 134.2 104.2 34.5

27947 67760

28034 68051

28110.8 68113.3

81.8 72.2

167 187

28108 68143

28194.2 68255.2

44.7 88.9

35651.6

35850.8

35922.4

73.8

143.4

35922.8

36100.4 1.6e-07

103.8

8

Jos´e Garc´ıa et al.

Balas and Carrera [1], and a Lagrangian-based heuristics algorithm [3] was used to perform the comparison. Comparison with TLBO and BSFLA: The TLBO algorithm adapts in a specific way the principles of the Teaching Learning metaheuristic which works naturally in a continuous space, to solve combinatorial problems such as set covering. In the case of BSFLA, it uses transfer functions that correspond to a general binarization mechanism. The results are shown in the table 3. PTRA found 13 of the 20 Best Known values. When we compare the best values of PTRA with respect to TBLO and BSFLA, PTRA is superior in all problems. When we compare the averages TLBO has better results in problems E.4, F.3, F.5, G.3, G.5, H.1 and H.4. Regarding convergence times despite not being directly comparable due to differences in implementation and hardware, TBLO turns out to be more efficient than PTRA. Comparison with Lagrangian-based heuristics: In the case of Lagrangianbased heuristics algorithm, the results are shown in the table 2. The Lagrangianbased heuristics completely outperforms PTRA regarding the quality of the solutions. the difference in % - Gap was 0.57 for the Best Value and 0.78 for the average. However the Lagrangian-based heuristic approach is not nearly as straightforward for practitioners to implement and use. On the other hand PTRA uses mechanisms quite simple to implement, besides easily adapting to different combinatorial problems and any continuous swarm-intelligence metaheuristics.

Table 3. OR-Library benchmarks E, F, G, H Instance

row

col

Densi- ty

Best Known

PTRA (best)

PTRA (avg)

PTRA (time)

TBLO

BSFLA (Best)

E.1 E.2 E.3 E.4 E.5

500 500 500 500 500

5000 5000 5000 5000 5000

10% 10% 10% 10% 10%

29 30 27 28 28

29 30 27 28 28

29 30 27.8 28.1 28

12 16 31 7 21

29 31 28 28 28

29 31 28 29 28

F.1 F.2 F.3 F.4 F.5

500 500 500 500 500

5000 5000 5000 5000 5000

20% 20% 20% 20% 20%

14 15 14 14 13

14 15 14 14 13

14 15 14.5 14 13.8

21.5 11.2 31.5 21.3 16.4

14 15 14 14 13

15 15 16 15 15

G.1 G.2 G.3 G.4 G.5

1000 1000 1000 1000 1000

10000 10000 10000 10000 10000

2% 2% 2% 2% 2%

176 154 166 168 168

176 155 167 170 168

177.9 155.9 168.7 171.1 169.7

192 143 176 198 143

179 156 168 172 168

182 161 173 173 174

H.1 H.2 H.3 H.4 H.5

1000 1000 1000 1000 1000

10000 10000 10000 10000 10000

5% 5% 5% 5% 5%

63 63 59 58 55

64 64 60 59 55

64.9 64 60.8 60.4 55.4

185 165 185 154 120

64 64 61 59 56

68 66 62 63 59

67.1

67.5

68.2

92.5

68.1

70.1

Average

PTRA

5

9

Conclusion and Future Work

In this article, we proposed a algorithm whose main function is to binarize continuous swarm-intelligence metaheuristics. To evaluate the performance of our algorithm, the set covering problem was used together with the Cuckoo Search metaheuristic. The contribution of the main operator of the algorithm was evaluated, finding that the percentile transition ranking operator contributes significantly to improve the precision and quality of the solutions. Finally, in comparison with state of the art algorithms our algorithm showed a good performance. As a future works we want to investigate the behaviour of other metaheuristics. Furthermore, the algorithm must be verified with other NP-hard problems. Moreover to simplify the choice of the appropriate configuration, it is important to explore adaptive techniques. From an understanding point of view of how the framework performs binarization, it is interesting to understand how the algorithm alters the properties of exploration and exploitation. Also is interesting to study how the velocities and positions generated by continuous metaheuristics are mapped to positions in the discrete space.

References 1. Egon Balas and Maria C Carrera. A dynamic subgradient-based branch-and-bound procedure for set covering. Operations Research, 44(6):875–890, 1996. 2. Michael J Brusco, Larry W Jacobs, and Garry M Thompson. A morphing procedure to supplement a simulated annealing heuristic for cost-andcoverage-correlated set-covering problems. Annals of Operations Research, 86:611–627, 1999. 3. Alberto Caprara, Matteo Fischetti, and Paolo Toth. A heuristic method for the set covering problem. Operations research, 47(5):730–743, 1999. 4. Broderick Crawford, Ricardo Soto, Gino Astorga, Jos´e Garc´ıa, Carlos Castro, and Fernando Paredes. Putting continuous metaheuristics to work in binary search spaces. Complexity, 2017, 2017. 5. Broderick Crawford, Ricardo Soto, Eric Monfroy, Gino Astorga, Jos´e Garc´ıa, and Enrique Cortes. A meta-optimization approach for covering problems in facility location. In Workshop on Engineering Applications, pages 565–578. Springer, 2017. 6. Broderick Crawford, Ricardo Soto, Cristian Pe˜ na, Wenceslao Palma, Franklin Johnson, and Fernando Paredes. Solving the set covering problem with a shuffled frog leaping algorithm. In Intelligent Information and Database Systems, pages 41–50. Springer, 2015. 7. Jos´e Garc´ıa, Broderick Crawford, Ricardo Soto, Castro Carlos, and Fernando Paredes. A k-means binarization framework applied to multidimensional knapsack problem. Applied Intelligence, pages 1–24, 2017. 8. Jos´e Garc´ıa, Broderick Crawford, Ricardo Soto, and Pablo Garc´ıa. A multi dynamic binary black hole algorithm applied to set covering problem. In International Conference on Harmony Search Algorithm, pages 42–51. Springer, 2017. 9. Jos´e Garc´ıa, Christopher Pope, and Francisco Altimiras. A distributed k-means segmentation algorithm applied to lobesia botrana recognition. Complexity, 2017, 2017. 10. Michael R Gary and David S Johnson. Computers and intractability. A Guide to the Theory of NP-Completeness, 1979.

10

Jos´e Garc´ıa et al.

11. Eduardo Graells-Garrido and Jos´e Garc´ıa. Visual exploration of urban dynamics using mobile data. In International Conference on Ubiquitous Computing and Ambient Intelligence, pages 480–491. Springer, 2015. 12. Mark´ o Horv´ ath and Tam´ as Kis. Computing strong lower and upper bounds for the integrated multiple-depot vehicle and crew scheduling problem with branchand-price. Central European Journal of Operations Research, pages 1–29, 2017. 13. Beasley John. A lagrangian heuristic for set-covering problems. Naval Research Logistics (NRL), 37(1):151–164, 1990. 14. Atoosa Kasirzadeh, Mohammed Saddoune, and Fran¸cois Soumis. Airline crew scheduling: models, algorithms, and data sets. EURO Journal on Transportation and Logistics, 6(2):111–137, 2017. 15. James Kennedy. Particle swarm optimization. In Encyclopedia of machine learning, pages 760–766. Springer, 2011. 16. Jose M Lanza-Gutierrez, Broderick Crawford, Ricardo Soto, Natalia Berrios, Juan A Gomez-Pulido, and Fernando Paredes. Analyzing the effects of binarization techniques when solving the set covering problem through swarm optimization. Expert Systems with Applications, 70:67–82, 2017. 17. Barend J Leonard, Andries P Engelbrecht, and Christopher W Cleghorn. Critical considerations on angle modulated particle swarm optimisers. Swarm Intelligence, 9(4):291–314, 2015. 18. Yun Lu and Francis J Vasko. An or practitioner’s solution approach for the set covering problem. International Journal of Applied Metaheuristic Computing (IJAMC), 6(4):1–13, 2015. 19. Esmat Rashedi, Hossein Nezamabadi-Pour, and Saeid Saryazdi. Gsa: a gravitational search algorithm. Information sciences, 179(13):2232–2248, 2009. 20. Ricardo Soto, Broderick Crawford, Rodrigo Olivares, Jorge Barraza, Franklin Johnson, and Fernando Paredes. A binary cuckoo search algorithm for solving the set covering problem. In Bioinspired Computation in Artificial Systems, pages 88–97. Springer, 2015. 21. Mirela Stojkovi´c. The operational flight and multi-crew scheduling problem. Yugoslav Journal of Operations Research, 15(1), 2016. 22. Akbarzadeh Totonchi and Mohammad Reza. Magnetic optimization algorithms, a new synthesis. In IEEE International Conference on Evolutionary Computations, 2008. 23. Claudio Valenzuela, Broderick Crawford, Ricardo Soto, Eric Monfroy, and Fernando Paredes. A 2-level metaheuristic for the set covering problem. International Journal of Computers Communications & Control, 7(2):377–387, 2014. 24. Xin-She Yang. Firefly algorithm, stochastic test functions and design optimisation. International Journal of Bio-Inspired Computation, 2(2):78–84, 2010. 25. Xin-She Yang and Suash Deb. Cuckoo search via l´evy flights. In Nature & Biologically Inspired Computing, 2009. NaBIC 2009. World Congress on, pages 210–214. IEEE, 2009. ˙ 26. Belma Yelbay, S ¸ Ilker Birbil, and Kerem B¨ ulb¨ ul. The set covering problem revisited: an empirical study of the value of dual information. European Journal of Operational Research, 2012.