Hyvärinen A., Karhunen J., Oja E. Independent Component Analysis

John Wiley, 2001. — 505 p.Independent component analysis (ICA) is a statistical and computational technique for revealing hidden factors that underlie sets of random variables, measurements, or signals. ICA defines a generative model for the observed multivariate data, which is typically given as a large database of samples. In the model, the data variables are assumed to be linear or nonlinear mixtures of some unknown latent variables, and the mixing system is also unknown. The latent variables are assumed nongaussian and mutually independent, and they are called the independent components of the observed data. These independent components, also called sources or factors, can be found by ICA.
ICA can be seen as an extension to principal component analysis and factor analysis. ICA is a much more powerful technique, however, capable of finding the underlying factors or sources when these classic methods fail completely. The data analyzed by ICA could originate from many different kinds of application fields, including digital images and document databases, as well as economic indicators and psychometric measurements. In many cases, the measurements are given as a set of parallel signals or time series; the term blind source separation is used to characterize this problem. Typical examples are mixtures of simultaneous speech signals that have been picked up by several microphones, brain waves recorded by multiple sensors, interfering radio signals arriving at a mobile phone, or parallel time series obtained from some industrial process.
The technique of ICA is a relatively new invention. It was for the first time in- introduced in early 1980s in the context of neural network modeling. In mid-1990s, some highly successful new algorithms were introduced by several research groups, together with impressive demonstrations on problems like the cocktail-party effect, where the individual speech waveforms are found from their mixtures. ICA became one of the exciting new topics, both in the field of neural networks, especially unsupervised learning, and more generally in advanced statistics and signal processing. Reported real-world applications of ICA on biomedical signal processing, audio signal separation, telecommunications, fault diagnosis, feature extraction, financial time series analysis, and data mining began to appear.
Many articles on ICA were published during the past 20 years in a large number of journals and conference proceedings in the fields of signal processing, artificial neural networks, statistics, information theory, and various application fields. Several special sessions and workshops on ICA have been arranged recently [70, 348], and some edited collections of articles [315, 173, 150] as well as some monographs on ICA, blind source separation, and related subjects [105, 267, 149] have appeared. However, while highly useful for their intended readership, these existing texts typically concentrate on some selected aspects of the ICA methods only. In the brief scientific papers and book chapters, mathematical and statistical preliminaries are usually not included, which makes it very hard for a wider audience to gain full understanding of this fairly technical topic.
A comprehensive and detailed text book has been missing, which would cover both the mathematical background and principles, algorithmic solutions, and practical applications of the present state of the art of ICA. The present book is intended to fill that gap, serving as a fundamental introduction to ICA. It is expected that the readership will be from a variety of disciplines, such as statistics, signal processing, neural networks, applied mathematics, neural and cognitive sciences, information theory, artificial intelligence, and engineering. Both researchers, students, and practitioners will be able to use the book. We have made every effort to make this book self-contained, so that a reader with a basic background in college calculus, matrix algebra, probability theory, and statistics will be able to read it. This book is also suitable for a graduate level university course on ICA, which is facilitated by the exercise problems and computer assignments given in many chapters.Mathematical Preliminaries
Random Vectors and Independence
Gradients and Optimization Methods
Estimation Theory
Information Theory
Principal component Analysis and Whitening
Basic Independent Component Analysis
What is Independent Component Analysis?
ICA by Maximization of Nongaussianity
ICA by Maximum Likelihood Estimation
ICA by Minimization of Mutual Information
ICA by Tensorial Methods
ICA by Nonlinear Decorrelation and Nonlinear PCA
Practical Considerations
Overview and Comparison of Basic ICA methods
Extensions and Related Methods
Noisy ICA
ICA with Overcomplete Bases
Nonlinear ICA
Methods using Time Structure
Convolutive Mixtures and Blind Deconvolution
Other Extensions
Applications of ICA
Feature Extraction by ICA
Brain Imaging Applications
Telecommunications
Other Applications