Zhang X.-S. Neural Networks in Optimization

Springer, 2000. — 369 p.People are facing more and more NP-complete or NP-hard problems of combinatorial nature and of a continuous nature in economic, military and management practice. There are two ways in which one can enhance the efficiency of searching for the solutions of these problems. The first is to improve the speed and memory capacity of hardware. We all have witnessed the computer industry's amazing achievements with hardware and software developments over the last twenty years. On one hand many computers, bought only a few years ago, are being sent to elementary schools for children to learn the ABC's of computing. On the other hand, with economic, scientific and military developments, it seems that the increase of intricacy and the size of newly arising problems have no end. We all realize then that the second way, to design good algorithms, will definitely compensate for the hardware limitations in the case of complicated problems.
It is the collective and parallel computation property of artificial neural networks that has activated the enthusiasm of researchers in the field of computer science and applied mathematics. It is hard to say that artificial neural networks are solvers of the above-mentioned dilemma, but at least they throw some new light on the difficulties we face. We not only anticipate that there will be neural computers with intelligence but we also believe that the research results of artificial neural networks might lead to new algorithms on von Neumann's computers.
Sometimes the idea of the artificial neural network makes one recall the time when analog computers were employed to solve different problems. We should be aware that there will be revolutionary concepts and methods in advanced artificial neural network research which will absolutely effect other scientific fields. Among them optimization theory as well as algorithms may be one of the most effected.
In fact it is not only the potential high computation rate of the artificial neural nets, due to the massive parallelism that has attracted the attentions of scientists in various fields, but also a great degree of robustness of the computation process in two respects: (1) From the point of view of hardware and software, there is a high degree of fault tolerance as compared with the case of von Neumann computers because there are many more processing elements. (2) From the point of view of the problem data, neural algorithms are not sensitive to incompleteness or data errors. In terms of neural network terminology, adaptation and learning are major features of neural net study.
Neural network study has been interfused into different theoretical and applied sciences. Being the first to be effected, technical branches, such as signal processing, object recognition, process identification, robot kinematics control and intelligent system for diverse purposes, have been the areas where the neural network research has displayed its tremendous power. Recently neural network theory has infiltrated some regular science disciplines, such as statistics, control theory and optimization theory.
The motivation of this book is to review the relevant materials in the current literature, which include both the applications of mathematical programming theory and algorithms in artificial neural network study and applications of novel neural network methods in developing efficient algorithms for solving optimization problems. The review is presented in the language and thinking mode familiar to the researchers, practitioners and graduate students with background in the area of optimization research. I hope that some of them may find this book helpful, and I will feel satisfied if someone is inspired by this book to think more deeply about the further inter-penetration between the optimization theory and neural network study.Part I Concepts and Models of Optimization
Preliminaries
Introduction to Mathematical Programming
Unconstrained Nonlinear Programming
Constrained Nonlinear Programming
Part II Basic Artificial Neural Network Models
Introduction to Artificial Neural Network
Feedforward Neural Networks
Feedback Neural Networks
Self-Organized Neural Networks
Part III Neural Algorithms for Optimization
NN Models for Combinatorial Problems
NN for Quadratic Programming Problems
NN for General Nonlinear Programming
NN for Linear Programming
A Review on NN for Continuous Optimization