Linear and Nonlinear Programming

The 5th edition of this classic textbook covers the central concepts of practical optimization techniques, with an emphasis on methods that are both state-of-the-art and popular. One major insight is the connection between the purely analytical character of an optimization problem and the behavior of algorithms used to solve that problem. End-of-chapter exercises are provided for all chapters.

 The material is organized into three separate parts. Part I offers a self-contained introduction to linear programming. The presentation in this part is fairly conventional, covering the main elements of the underlying theory of linear programming, many of the most effective numerical algorithms, and many of its important special applications. Part II, which is independent of Part I, covers the theory of unconstrained optimization, including both derivations of the appropriate optimality conditions and an introduction to basic algorithms. This part of the book explores the general properties of algorithms and defines various notions of convergence. In turn, Part III extends the concepts developed in the second part to constrained optimization problems. Except for a few isolated sections, this part is also independent of Part I. As such, Parts II and III can easily be used without reading Part I and, in fact, the book has been used in this way at many universities.

 New to this edition are popular topics in data science and machine learning, such as the Markov Decision Process, Farkas’ lemma, convergence speed analysis, duality theories and applications, various first-order methods, stochastic gradient method, mirror-descent method, Frank-Wolf method, ALM/ADMM method, interior trust-region method for non-convex optimization, distributionally robust optimization, online linear programming, semidefinite programming for sensor-network localization, and infeasibility detection for nonlinear optimization.

1. Introduction.- Part I: Linear Programming.- 2. Basic Properties of Linear Programs.- 3. Duality and Complementarity.- 4. The Simplex Method.- 5. Interior-Point Methods.- 6. Conic Linear Programming.- Part II: Unconstrained Problems.- 7. Basic Properties of Solutions and Algorithms.- 8. Basic Descent Methods.- 9. Conjugate Direction Methods.- 10. Quasi-Newton Methods.- Part III: Constrained Optimization.- 11. Constrained Optimization Conditions.- 12. Primal Methods.- 13. Penalty and Barrier Methods.- 14. Local Duality and Dual Methods.- 15. Primal–Dual Methods.

David G. Luenberger received his B.S. degree from the California Institute of Technology and his M.S. and Ph.D. degrees from Stanford University, all in Electrical Engineering.  Since 1963 he has served on the faculty of Stanford University.  He helped found the Department of Engineering-Economic Systems, which have since become the Department of Management Science and Engineering, where his is currently a professor.

He is a Member of the National Academy of Engineering (2008) and has received e.g. the Bode Lecture Prize of the Control Systems Society (1990), the Oldenburger Medal of the American Society of Mechanical Engineers (1995), and the Expository Writing Award of the Institute of Operations Research and Management Science (1999). He is a Fellow of the Institute of Electrical and Electronic Engineers (since 1975).

Yinyu Ye is currently the Kwoh-Ting Li Professor in the School of Engineering at the Department of Management Science and Engineering and Institute of Computational and Mathematical Engineering. He received his B.S. degree in System Engineering from Huazhong University of Science and Technology, China, and his M.S. and Ph.D. degrees in Engineering-Economic Systems and Operations Research from Stanford University. He is an INFORMS (The Institute for Operations Research and The Management Science). Fellow since 2012, and has received several academic awards including: the 2009 John von Neumann Theory Prize for fundamental sustained contributions to theory in Operations Research and the Management Sciences, the 2015 SPS Signal Processing Magazine Best Paper Award, the winner of the 2014 SIAM Optimization Prize awarded (every three years), the inaugural 2012 ISMP Tseng Lectureship Prize for outstanding contribution to continuous optimization (every three years), the inaugural 2006 Farkas Prize on Optimization, the 2009 IBM Faculty Award.