1
date: June 6, 2005
LINEAR OPTIMIZATION: THEOREMS OF THE ALTERNATIVE, ThAlt If one has two systems of linear relations, where each relation is either an linear equation (or linear equality relation) or a linear inequality relation (of type >, ≥, 0 T y A = 0, 0 6= y ≥ 0
3 a b
J. Farkas (1902) [3] Ax = b, x ≥ 0 T y A ≥ 0, y T b < 0
4 a b
J. Farkas (1902) [3] Ax ≤ b T y ≥ 0, y A = 0, y T b < 0
5 a b
E. Stiemke (1915) [13] Ax = 0, x > 0 T y A ≥ 0, y T A 6= 0
6 a b
W.B. Carver (1921) [2] Ax < b T y A = 0, y ≥ 0, y T b ≤ 0, y 6= 0
7 a b
T.S. Motzkin (1936) [10] Ax ≤ 0, Bx < 0 T T y A + v B = 0, y ≥ 0, v ≥ 0, v 6= 0
8 a b
J. Ville (1938) [15] Ax > 0, x > 0 T y A ≤ 0, y ≥ 0, y 6= 0 or AT y 6= 0
9 a b
A.W. Tucker (1956) [14] Ax ≥ 0, Ax 6= 0, Bx ≥ 0, Cx = 0 T y A + v T B + wT C = 0, y > 0, v ≥ 0
10 a b
D. Gale (1960) [5] Ax ≤ b T y A = 0, y T b = −1, y ≥ 0 Ten pairs of alternative systems
that the implied inequality has no solution x if and only if y T A = 0 and y T b < 0. Together with y ≥ 0 these are precisely the relations in
file: ROOS1
date: June 6, 2005
the alternative system 4b. Thus it may be concluded that Farkas’ lemma can be restated by saying that the system Ax ≤ b is feasible if and only if it does not imply (in a linear fashion) the ‘contradiction’ 0T x < 0. The ‘if’-part is obvious: if the system has an implied inequality 0T x < 0 then it must be inconsistent. But the ‘only if’part is a very deep result: it states that if the system has no contradictory implied inequality then it has a solution. The other theorems of the alternative in the table admit a similar interpretation. The relevance of a theorem of the alternative is the following. Given some system S of relations the crucial question is whether the system has a solution or not. Knowing the answer to this question one is able to answer many other questions. For example, if one has a linear optimization problem LO in the standard form min cT x : Ax = b, x ≥ 0 , x
a given real number z is a strict lower bound for the optimal value of the problem if and only if the system Ax = b, cT x ≤ z, x ≥ 0 has no solution, i.e. is infeasible. On the other hand, a given real number z is an upper bound for the optimal value of the problem if and only if the system Ax = b, cT x ≥ z, x ≥ 0 has a solution, i.e. is feasible. If a system S has a solution then this is easy to certify, namely by giving a solution of the system. The solution then serves as a certificate for the feasibility of S. If S is infeasible, however, it is more difficult to give an easy certificate. One is then faced with the problem of how to certify a negative statement. This is in general a very nontrivial problem that also occurs in many real life situations. For example, when accused for murder, how should one prove his innocence? In linear optimization problem standard form infeasible feasible certificate linear optimization duality theorem for linear optimization
2
circumstances like these it may be impossible to find an easy to verify certificate for the negative statement ‘not guilty’. A practical solution is the rule ‘a person is innocent until his/her guilt is certified’. Clearly, from the mathematical point of view this approach is unsatisfactory. Now suppose that there is an alternative system T and there exists a theorem of the alternative for S and T . Then we know that exactly one of the two systems has a solution. Therefore, S has a solution if and only if T has no solution. In that case, any solution of T provides a certificate for the unsolvability of S. Thus it is clear that a theorem of the alternative provides an easy to verify certificate for the unsolvability of a system of linear relations. The proof of any theorem of the alternative consists of two parts. Assuming the existence of a solution of one system one needs to show that the other system is infeasible, and vice versa. It has been demonstrated above for Farkas’ lemma that one of the two implications is easy to prove. This seems to be true for each theorem of the alternative: in all cases one of the implications is almost trivial, but the other implication is highly nontrivial and very hard to prove. On the other hand, having proved one theorem of the alternative the other theorems of the alternative easily follow. In this sense one might say that all the listed theorems of the alternative are equivalent: accepting one of them to be true, the validity of each of the other theorems can be verified easily. The situation resembles a number of cities on a high plateau. Travel between them is not too difficult; the hard part is the initial ascent from the plains below [1]. It should be pointed out that Farkas’ lemma, or each of the other theorems of the alternative, is equivalent to the most deep result in linear optimization, namely the duality theorem for linear optimization: this theorem can be easily derived from Farkas’ lemma, and vice versa.
3
date: June 6, 2005
In fact, in many text books on linear optimization the duality theorem is derived in this way [5, 16], whereas in other text books the opposite occurs: the duality theorem is proved first and then Farkas’ lemma follows as a corollary [11]. This phenomenon is a consequence of a simple, and basic, logical principle that any duality theorem is actually equivalent to a theorem of the alternative, as has been shown in [9]. Both the Farkas’ lemma and the duality theorem for linear optimization can be derived from a more general result which states that for any skew-symmetric matrix K (i.e., K = −K T ) there exists a vector x such that Kx ≥ 0, x ≥ 0, x + Kx > 0. This result is due to Tucker [14] who also derives Farkas’ lemma from it, whereas Goldman and Tucker [6] show how this result implies the duality theorem for linear optimization. For recent proofs, see [12]. References [1] Broyden, C.G.: ‘A simple algebraic proof of Farkas’ lemma and related theorems’, Optimization Methods and Software (1998), To appear. [2] Carver, W.B.: ‘Systems of Linear Inequalities’, Annals of Mathematics 23 (1921), 212–220. [3] Farkas, J.: ‘Theorie der Einfachen Ungleichungen’, Journal f¨ ur die Reine und Angewandte Mathematik 124 (1902), 1–27. [4] Fourier, J.B.J.: ‘Solution d’une Question Particuli`ere du Calcul des In´egalit´es’, Nouveau Bulletin des Sciences par la Socie´et´e Philomathique de Paris (1826), 99–100. [5] Gale, D.: The Theory of Linear Economic Models, McGraw-Hill, New York, NY, USA., 1960. [6] Goldman, A.J., and Tucker, A.W.: ‘Theory of Linear Programming’, Linear Inequalities and Related Systems, in H.W. Kuhn and A.W. Tucker (eds.), Annals of Mathematical Studies, No. 38. Princeton University Press, Princeton, New Jersey, 1956, pp. 53–97. ¨ [7] Gordan, P.: ‘Uber die Aufl¨ osung Linearer Gleichungen mit Reelen Coefficienten’, Mathematische Annalen 6 (1873), 23–28. [8] Mangasarian, O.L.: Nonlinear Programming, No. 10 in Classics in Applied Mathematics. SIAM, Philadelphia, PA, USA, 1994. [9] McLinden, L.: ‘Duality theorems and theorems of the alternative’, Proceeedings of the American Mathematical Society 53 (1975), 172–175. skew-symmetric matrix
file: ROOS1
[10] Motzkin, T.S.: Beitr¨ age zur Theorie der Linearen Ungleichungen, PhD thesis, Basel, Azriel, Jerusalem, 1936. [11] Padberg, M.: Linear Optimization and Extensions, Vol. 12 of Algorithmis and Combinatorics, Springer Verlag, Berlin, West–Germany, 1995. [12] Roos, C., Terlaky, T., and J.-Ph.Vial: Theory and Algorithms for Linear Optimization. An Interior Approach, John Wiley & Sons, Chichester, UK, 1997. ¨ [13] Stiemke, E.: ‘Uber Positive L¨ osungen Homogener Linearer Gleichungen’, Mathematische Annalen 76 (1915), 340–342. [14] Tucker, A.W.: ‘Dual systems of homogeneous linear relations’, Linear Inequalities and Related Systems, Annals of Mathematical Studies, No. 38, in H.W. Kuhn and A.W. Tucker (eds.). Princeton University Press, Princeton, New Jersey, 1956, pp. 3–18. [15] Ville, J.: ‘Sur la th´eorie g´en´erale des jeux o` u intervient l’habilet´e des joueurs’, Applications aux Jeux de Hasard, in J. Ville (ed.). Gautheier-Villars, Paris, 1938, pp. 105–113. [16] Zoutendijk, G.: Mathematical Programming Methods, North-Holland Publishing Comp., Amsterdam, The Netherlands, 1976.
Kees Roos Delft Univ. of Technology Department ITS/TWI/SSOR P.O. Box 356, 2600 AJ Delft The Netherlands E-mail address:
[email protected]
AMS1991SubjectClassification: 15A39,90C05. Key words and phrases: inequality systems, duality, certificate, transposition theorem.
