Lee T.-S. Structure Level Adaptation for Artificial Neural Networks

Springer, 1991. — 224 p.Current artificial neural network (ANN) models allow the networks to adjust their behavior by changing the interconnection weights associating neurons to each other, but the number of neurons and the structural relationship between neurons must be set up by system designers and once the structure is designed, it is fixed throughout the life cycle of the system. This sets quite a constraint on the applicability of artificial neural networks.
This monograph proposes a general framework for Structural Level Adaptation (SLA) of ANN to allow a neural network to change its structure in addition to weights in the adaptation process. Basic requirements and criteria for SLA of a ANN are identified and analyzed and a general paradigm, called Activity Based Structure Level Adaptation (ASLA), is developed for SLA of ANN (chapter 2). Two structural level adaptable neural network models, FUNNET (FUNction NETwork) (chapter 3) and SPAN (Space PArtition Network) (chapter 4), are introduced to demonstrate the proposed SLA framework.
Software simulators for SPAN and FUNNET based on doubly linked lists were implemented and tested. Simulation results for the two different network models show that we can initially put a small number of seed neurons in the network, then let the neurons replicate and orga- nize the structural relationships between one another according to the training patterns; finally the network grows to a configuration suitable to the class of problems characterized by the training patterns. In other words, the neural network self-synthesizes to fit the problem space. If the statistics of the problem space changes with time, the network will adapt its structure to follow the variations.
An adaptive source coding system based on SPAN is developed and computer simulation demonstrates that fast encoding/decoding, good rate-distortion performance, and smooth & incremental adaptation can be achieved using SPAN model (chapter 5). Potential applications of this coding scheme are speech and image coding as well as data compression for HDTV and ISDN systems.
The main contribution of this work is that we have provided a general framework for structure level adaptation of artificial neural networks, and we have demonstrated the validity of the proposed framework through two structural level adaptable neural network models (SPAN and FUNNET), and the application of SPAN to some practical problems.Basic Framework
Multi-Layer Feed-Forward Networks
Competitive Signal Clustering Networks
Application Example: An Adaptive Neural Network Source CoderA: Mathematical Background
B: Fluctuated Distortion Measure
C: SPAN Convergence Theory
D: Operational Measures
E: Glossary of Symbols and Acronyms