Lagrangian Based Heuristics for the Linear Ordering ... - CiteSeerX

MIC’2001 - 4th Metaheuristics International Conference

445

Lagrangian Based Heuristics for the Linear Ordering Problem Alexandre Belloni∗

Abilio Lucena†

∗

Instituto de Matem´ atica Pura e Aplicada Estrada Dona Castorina 110, Rio de Janeiro - RJ, 22460-320, Brazil Email: [email protected] †

1

Departamento de Administra¸ca õ, Universidade Federal do Rio de Janeiro Av. Pasteur 250, Rio de Janeiro - RJ, 22290-240, Brazil Email: [email protected]

Introduction

Assume that the elements of a finite set N are to be ordered and that some associated precedence costs, {cij : i, j ∈ N, i = j}, are given. For any ordering of the elements of N and any pair of elements i, j ∈ N , whenever i is placed prior to j, one incurs a cost cij . Otherwise, a cost cji is incurred. The Linear Ordering Problem (LOP) is to find an ordering of the elements of N where the total sum of the costs incurred is as large as possible. The LOP can be cast, in graph theoretical terms, by associating every element of N with a different vertex in a companion graph, D. Accordingly, let D = (V, A) be a complete directed graph with a set V of vertices and a set A of arcs. Arc costs {cij : (i, j) ∈ A} are derived from the set of precedence costs introduced before. A subgraph D = (V, A ) is denoted a tournament if, for every pair of vertices i, j ∈ V , exactly one of the arcs (i, j) or (j, i)} is contained in A . It is then follows that an acyclic tournament of D, where the sum of all arc costs is as large as possible, implies an optimal solution to LOP. LOP has a number of relevant practical applications (see [6] for details). Among these, the most commonly cited is the triangulation of input-output matrices. An exact solution algorithm to LOP, based on the use of polyhedral cutting planes, can be found in [6]. More recently, [11] proposed another exact solution algorithm where Linear Programming (LP) relaxations are solved by combining an interior point algorithm with a simplex one. Two recent heuristics for LOP are [8] where Tabu Search [4] is used and [2] where Scatter Search (see [5] for details) is used. In this paper, two different LOP heuristics are proposed and computationally tested. Heuristics are incorporated into a Lagrangian Relaxation (LR) framework and repeatedly called under Lagrangian modified arc costs. As a result, a series of feasible LOP solutions are generated which attempt to incorporating LR upper bounding information. Due to the exceedingly large number of inequalities that would have to be dualized to generate Lagrangian LOP upper bounds, a Relax and Cut algorithm, in the style proposed in [9, 10], is used. Computational experiments indicate that the approach appears highly effective for LOPs associated with the triangulation of input-output matrices. Indeed, for all 43 instances taken from LOLIB [13], optimal solutions have been generated. On the other hand, for only 2 out of the 30 instances proposed in [11], optimal solutions could not be generated. Porto, Portugal, July 16-20, 2001

446

2


A Lagrangian Relaxation Upper Bound for LOP

A Linear Integer Programming formulation for LOP [6] uses binary variables xij , (i, j) ∈ A, to control the inclusion (xij = 1) or not (xij = 0) of an arc (i, j) into a tournament. In association, consider a polyhedral region, R, described as xij + xji = 1, for all i, j ∈ V

(1)

xij + xjk + xki ≤ 2, for all (i, j), (j, k), (k, i) ∈ A

(2)

0 ≤ xij ≤ 1, for all (i, j) ∈ A.

(3)

An LOP formulation is then given by Max {

cij xij : x ∈ R ∩ |A|},

(4)

(i,j)∈A

where (1) enforces that a tournament be formed and (2) ensures the tournament to be cycle free. Attaching multipliers Λ ≥ 0 to the inequalities in (2) and denoting by RL the polyhedral region described by (1) and (3), a valid upper bound for LOP is given by U B(Λ) = Maxλijk ≥0 { cij xij + 2 × λijk : x ∈ RL ∩ |A|}, (5) (i,j)∈A

i,j,k∈V,i=j,i=k,j=k

where {cij = cij − k∈V \{i,j} λijk : (i, j) ∈ A} is the set Lagrangian modified arc costs. An optimal solution to (5) is obtained by setting to one, for every pair i, j ∈ V , that variable, xij or xji , with the largest Lagrangian cost (the remaining variable being set to zero). Ties are broken arbitrarily. The best possible Lagrangian bound is thus given by MinΛ≥0 U B(Λ)

(6)

and might possibly be attained through the use of Subgradient Optimization (see [7] for the Subgradient 3 Method (SM) and [1] for Bundle Methods). Notice, nevertheless, that the O(|V | ) inequalities in (2) could number over 970, 000 for an LOP with as little as 100 vertices. In turn, that would make the use of a straightforward implementation of, say the SM, quite ineffective. The main reason being the extremely high number of nonzero subgradients, for the inequalities in (2), that would typically arise at a SM iteration. Consequently, the step size at a SM iteration (which depends on the norm of the array of subgradients) would result minute, thus jeopardizing convergence to (6). In this respect, a compromise could be reached by adapting the SM to operate as suggested in [9, 10] (in a procedure that resembles the use of cutting planes in a Branch and Cut [12] algorithm). This path has been followed in this work. The term Relax and Cut has been coined in [3] for an algorithm that, although different from the one in [9, 10], bears a number of resemblances with it. In this paper, one uses the term Relax and Cut to denote the whole class of LR algorithms where inequalities are dualized on the fly, as they become violated at the solution to a LR subproblem. This class therefore encompasses the algorithm in [9, 10] as well as the one in [3].

3

LOP heuristics

For any set S ⊂ N , denote by S the complement of S over N and compute, for every j ∈ S, position costs qj = i∈S cij + k∈S\{j} cjk . The basic idea behind the first heuristic, denoted Position Cost (PC), is to use position costs to sequentially build an ordering of the elements of N . Accordingly, at step p = |S| + 1 of the procedure (note that S can be the empty set), elements in S ⊂ N would have Porto, Portugal, July 16-20, 2001


447

been placed in the first p − 1 positions of the ordering. At this stage, that element j, j ∈ S, for which qj ≥ qk , for all k ∈ S \ {j}, is placed at position p of the ordering. The step is concluded by setting S := S ∪ {j}. The procedure is finished after |N | steps and has complexity O(|N |2 ). Procedure PC is to be called, throughout the SM, for different sets of Lagrangian modified arc costs. At every such iteration, one uses the corresponding Lagrangian modified arc costs to compute position costs. The second heuristic uses LR solutions (instead of Lagrangian modified arc costs) as input. Assume x to be the optimal solution to (5), for a given set of Lagrangian multipliers Λ, at an iteration of the SM. Clearly, such a solution must be a tournament, though not necessarily an acyclic one. For such a = x tournament compute, for every vertex i ∈ V , the out degree d+ i (i,j)∈A ij of i (i.e. the number of tournament arcs pointing outwards of the vertex). Notice that if x were an acyclic tournament (and therefore an optimal solution to LOP), exactly one of the vertices in V would have an out degree of |V | − 1. Accordingly, exactly one other vertex in V would have an out degree of l ∈ {0, . . . , |V | − 2}. This is the basic idea behind the second heuristic, denoted Vertex Degree (VP). One orders vertex out degrees associated with an optimal solution to (5) in decreasing value of magnitude (ties are broken arbitrarily). Such an ordering is clearly feasible to LOP. The complexity of VP is O(|N | log |N |).

3.1

Local Search

Given a feasible, suboptimal, LOP solution, the concept of k-optimality could be used in an attempt to generating better quality solutions. A simplest move in this line would be to change positions of two contiguously placed elements in the ordering (as long as an increase in the total cost of the ordering results). Such a move is denoted here contiguous 2-opt. A more elaborated move would be to change positions of any two distinct elements in an ordering (not necessarily contiguously placed ones). Such a move is denoted 2-opt. Clearly the concept could be extended to k distinct elements in an ordering (once again, not necessarily contiguously placed ones) in a k-opt move. In the computational experiments that have been conducted, 2-opt and 3-opt proved, computationally speaking, far too expensive. One should recall here that, for every iteration of the SM, a call (under Lagrangian modified arc costs) is made to a construction heuristic and that the feasible solution thus obtained is submitted to local improvement. For the overall Lagrangian heuristics, the alternative of restricting the search neighborhood associated with 2-opt and 3-opt moves did not appear to lead to significantly better results than those obtained by simply applying contiguous 2-opt. Therefore, only contiguous 2-opt was used for local improvement. Throughout the application of the Lagrangian heuristics, care should be exercised in order to avoid duplication of work. In this study, along the Subgradient Optimization, a hash table is kept for storing previously generated LOP solutions. In doing so, one avoids re-submitting a same feasible solution to local improvement.

4

Computational Results

In order to test the proposed heuristics, two different sets of LOP instances, from the literature, were used. The first set involves 43 instances from LOLIB [13] and relate with input-output matrices compiled for the economies of different European countries. The largest instance in this set involves 60 elements while the smallest one has 44. For the second set, instances were randomly generated in [11] to resemble input-output matrices. The 30 instances in the set ranged in size from 100 up to 250 elements. One should point out that the LOP is posed in [11] as a minimization problem. Porto, Portugal, July 16-20, 2001

448

MIC’2001 - 4th Metaheuristics International Conference # ITERS 1 100 5000

GAP 0.2103 0.0139 0.0083

# OPTS 9 30 40

CPU-TIME 0.097 0.541 25.83

Table 1: LOLIB instances: Lagrangian Heuristic PC

# ITERS 1 100 5000

GAP 0.2485 0.009 0.000

# OPTS 8 37 43

CPU-TIME 0.102 0.512 5.581

Table 2: LOLIB instances: Lagrangian Heuristic VP

Feasible solutions were generated by firstly defining the maximum number of iterations allowed for the SM. An associated LOP heuristic is called for every such iteration. Experiments have been conducted for a maximum of 1, 100, and 5000 iterations of the SM. Obviously, whenever optimality could be proven before reaching that number, the SM run is stopped. Since arc costs are integral, one possibility for proving optimality is to reach a difference of less than one unit between LOP upper and lower bounds. Another possibility arises by assuming that, at a given iteration of the SM, an acyclic tournament turns out to be the optimal solution to (5). Whenever the Lagrangian cost of this tournament equals the cost of the tournament under the original arc costs, the tournament is guaranteed to be an optimal LOP solution. Tables 1 through 4 summarize the results obtained. For any of these Tables, columns give, respectively, the maximum number of SM iterations allowed, the average percentage gap between optimal solution values and solution values obtained by the corresponding Lagrangian heuristic, the number of instances where an optimal solution has been generated, and the average CPU time involved (in seconds of a Pentium III 500 MHz based machine). All the coding was done in C++. Optimal solution values are not quoted for the instances in [11]. Nevertheless, Relax and Cut was capable of proving optimality for 28 out of the 30 instances involved. In the absence of a known optimal solution value for these two remaining instances, one has used the gap between Lagrangian upper and lower bounds for the statistics quoted in Tables 3 and 4. The two different heuristics appear to perform well for the LOP instances considered. The one involving VP appears to have a clear edge over the other one (generates better quality LOP solutions in less CPU time). Average CPU time for the 5, 000 iterations entry in Table 4 could have been significantly better if it were not for a single instance which took 1, 200 CPU seconds to complete. The dual bounds generated by the Relax and Cut algorithm also proved very good. These bounds, combined with the primal bounds generated by the Lagrangian heuristic VP, allowed optimality to be proven for all but two instances in Table 2 and another two instances in Table 4. Nevertheless, for these 4 instances, a nonzero gap exists between the LP relaxation of (4) and the corresponding optimal LOP solution values. One would thus have to resort to branching, anyway, to be capable of proving optimality. It should also be pointed out that at most 6090 inequalities, out of a total of over 205,000 inequalities in (2), were explicitly dualized, at any iteration of the Relax and Cut SM, for any of the LOLIB instances. For the instances in [11] the corresponding figures are, respectively, 70,000 and over 15,438,000. Porto, Portugal, July 16-20, 2001

MIC’2001 - 4th Metaheuristics International Conference # ITERS 1 100 5000

LB-GAP 0.0220 0.00262 0.00227

# OPTS 0 7 16

449

CPU-TIME 0.526 12.43 605.13

Table 3: Instances from [11]: Lagrangian Heuristic PC # ITERS 1 100 5000

LB-GAP 0.0151 0.00057 0.00014

# OPTS 2 22 28

CPU-TIME 0.504 19.56 105.46

Table 4: Instances from [11]: Lagrangian Heuristic VP

References [1] J.F. Bonnans, J.Ch. Gilbert, C. Lemaréchal, and C. Sagastizábal. Optimisation numérique: aspects théoriques et pratiques. Springer Verlag, 1997. [2] V. Campos, M. Laguna, and R. Marti. Scatter Search for the Linear Ordering Problem. Graduate School of Business Report, University of Colorado, Boulder, CO 80309, USA, 1998. [3] L. Escudero, M. Guignard, and K. Malik. A lagrangian relax and cut approach for the sequential ordering with precedence constraints. Annals of Operations Research, 50, pages 219–237, 1994. [4] F. Glover and M. Laguna. Tabu Search. Kluwer Academic Publisher, 1997. [5] F. Glover. A Template for Scatter Search and Path Relinking. In J.-K. Hao, E. Lutton, E. Ronald, M. Schoenauer, and D. Snyers, Artificial Evolution, Lecture Notes in Computer Science 1363, Springer, pages 13–54, 1998. [6] M. Gr¨ otschel, M. J¨ unger, and G. Reinelt. A cutting plane algorithm for the linear ordering problem. Operations Research, 32:1195-1220, 1984. [7] M. Held, P. Wolfe, and H.P. Crowder. Validation of subgradient optimization. Mathematical Programming, 6, pages 62–88,1974. [8] M. Laguna, R. Marti, and V. Campos. Intensification and Diversification with Elite Tabu Search Solutions for the Linear Ordering Problem. Graduate School of Business Report, University of Colorado, Boulder, CO 80309, USA, 1998. [9] A. Lucena. Steiner problem in graphs: Lagrangean relaxation and cutting-planes. COAL Bulletin, 21, pages 2–8, 1992. [10] A. Lucena. Tight bounds for the Steiner problem in graphs. Proceedings of NETFLOW93, pages 147–154, 1993. [11] J.E. Mitchell and B. Borchers. Solving linear ordering problems with a combined interior point/simplex cutting plane algorithm, Rensseler Institute Report, Troy, NJ 12180, USA, 1997. [12] M. Padberg and G. Rinaldi. A branch-and-cut algorithm for the resolution of large-scale symmetric traveling salesman problems. SIAM Review, 33, pages 60–100, 1991. [13] G. Reinelt. LOLIB http://www.iwr.uni-heidelberg.de/iwr/comopt/soft/LOLIB/LOLIB.html.

Porto, Portugal, July 16-20, 2001