Salem F.M. Recurrent Neural Networks: From Simple to Gated Architectures

Springer, 2022. — 130 p. — ISBN13: 9783030899288.This textbook provides a compact but comprehensive treatment that provides analytical and design steps to recurrent neural networks from scratch. It provides a treatment of the general recurrent neural networks with principled methods for training that render the (generalized) backpropagation through time (BPTT). This author focuses on the basics and nuances of recurrent neural networks, providing technical and principled treatment of the subject, to use coding and deep learning computational frameworks, e.g., Python and Tensorflow-Keras. Recurrent neural networks are treated holistically from simple to gated architectures, adopting the technical machinery of adaptive non-convex optimization with dynamic constraints to leverage its systematic power in organizing the learning and training processes. This permits the flow of concepts and techniques that provide grounded support for design and training choices. The author’s approach enables strategic co-training of output layers, using supervised learning, and hidden layers, using unsupervised learning, to generate more efficient internal representations and accuracy performance. As a result, readers will be enabled to create designs tailoring proficient procedures for recurrent neural networks in their targeted applications.Basic Elements of Neural Networks
Network Architectures
Learning Processes
Recurrent Neural Networks (RNN)
Recurrent Neural Networks (RNN)
Gated Recurrent Neural Networks: The LSTM RNN
Gated RNN: The Long Short-Term Memory (LSTM) RNN
Gated RNN: The Gated Recurrent Unit (GRU) RNN
Gated RNN: The Minimal Gated Unit (MGU) RNN