Medsker L.R., Jain L.C. (eds.) Recurrent Neural Networks. Design and Applications

CRC Press, 2000. — 391 p.Recurrent neural networks have been an interesting and important part of neural network research during the 1990's. They have already been applied to a wide variety of problems involving time sequences of events and ordered data such as characters in words. Novel current uses range from motion detection and music synthesis to financial forecasting. This book is a summary of work on recurrent neural networks and is exemplary of current research ideas and challenges in this subfield of artificial neural network research and development. By sharing these perspectives, we hope to illuminate opportunities and encourage further work in this promising area.
Two broad areas of importance in recurrent neural network research, the architectures and learning techniques, are addressed in every chapter. Architectures range from fully interconnected to partially connected networks, including recurrent multilayer feedforward. Learning is a critical issue and one of the primary advantages of neural networks. The added complexity of learning in recurrent networks has given rise to a variety of techniques and associated research projects. A goal is to design better algorithms that are both computationally efficient and simple to implement.
Another broad division of work in recurrent neural networks, on which this book is structured, is the design perspective and application issues. The first section concentrates on ideas for alternate designs and advances in theoretical aspects of recurrent neural networks. Some authors discuss aspects of improving recurrent neural network performance and connections with Bayesian analysis and knowledge representation, including extended neuro-fuzzy systems. Others address real-time solutions of optimization problems and a unified method for designing optimization neural network models with global convergence.
The second section of this book looks at recent applications of recurrent neural networks. Problems dealing with trajectories, control systems, robotics, and language learning are included, along with an interesting use of recurrent neural networks in chaotic systems. The latter work presents evidence for a computational paradigm that has higher potential for pattern capacity and boundary flexibility than a multilayer static feedforward network. Other chapters examine natural language as a dynamic system appropriate for grammar induction and language learning using recurrent neural networks. Another chapter applies a recurrent neural network technique to problems in controls and signal processing, and other work addresses trajectory problems and robot behavior.
The next decade should produce significant improvements in theory and design of recurrent neural networks, as well as many more applications for the creative solution of important practical problems. The widespread application of recurrent neural networks should foster more interest in research and development and raise further theoretical and design questions.Recurrent Neural Networks for Optimization: The State of the Art
Efficient Second-Order Learning Algorithms for Discrete-Time Recurrent Neural Networks
Designing High Order Recurrent Networks for Bayesian Belief Revision
Equivalence in Knowledge Representation: Automata, Recurrent Neural Networks, and Dynamical Fuzzy Systems
Learning Long-Term Dependencies in NARX Recurrent Neural Networks
Oscillation Responses in a Chaotic Recurrent Network
Lessons From Language Learning
Recurrent Autoassociative Networks: Developing Distributed Representations of Structured Sequences by Autoassociation
Comparison of Recurrent Neural Networks for Trajectory Generation
Training Algorithms for Recurrent Neural Nets that Eliminate the Need for Computation of Error Gradients with Application to Trajectory Production Problem
Training Recurrent Neural Networks for Filtering and Control
Remembering How To Behave: Recurrent Neural Networks for Adaptive Robot Behavior