McLachlan G.J., Krishnan T. The EM Algorithm and Extensions

Wiley-Interscience, 1997. — 304 p. — (Wiley Series in Probability and Statistics).This book presents an organized and well-knit account of the theory, methodology, extensions, and major applications of the Expectation-Maximization (EM) algorithm. It includes applications in the standard statistical contexts such as regression, factor analysis, variance components estimation, repeated-measures designs, categorical data analysis, survival analysis and survey sampling.The EM Algorithm and Extensions
General Introduction
Maximum Likelihood Estimation
Newton-Type Methods
Introductory Examples
Formulation of the EM Algorithm
EM Algorithm for Maximum a Posteriori and Maximum Penalized Likelihood Estimation
Brief Summary of the Properties of the EM Algorithm
History of the EM Algorithm
Overview of the BookExamples of the EMAlgorithm
Multivariate Data with Missing Values
Least Squares with Missing Data
Example 2.4: Multinomial with Complex Cell Structure
Example 2.5: Analysis of PET and SPECT Data
Example 2.6: Multivariate t-Distribution with Known Degrees of Freedom
Finite Normal Mixtures
Example 2.9: Grouped and Truncated Data
Basic Theory of the EMAlgorithm
Monotonicity of the EM Algorithm
Monotonicity of a Generalized EM Algorithm
Convergence of an EM Sequence to a Stationary Value
Convergence of an EM Sequence of Iterates
Examples of Nontypical Behavior of an EM (GEM) Sequence
Score Statistic
Missing Information
Rate of Convergence of the EM Algorithm
Standard Errors and Speeding Up Convergence
Observed Information Matrix
Approximations to the Observed Information Matrix: i.i.d. Case
Observed Information Matrix for Grouped Data
Supplemented EM Algorithm
Bootstrap Approach to Standard Error Approximation
Acceleration of the EM Algorithm via Aitken's Method
An Aitken Acceleration-Based Stopping Criterion
Conjugate Gradient Acceleration of the EM Algorithm
Hybrid Methods for Finding Maximum Likelihood Estimate
A Generalized EM Algorithm Based on One Newton-Raphson Step
EM Gradient Algorithm
A Quasi-Newton Acceleration of the EM Algorithm
Extensions of the EM Algorithm
ECM Algorithm
Multicycle ECM Algorithm
Example 5.1: Normal Mixtures with Equal Correlations
Example 5.2: Mixture Models for Survival Data
Example 5.3: Contingency Tables with Incomplete Data
ECME Algorithm
Example 5A: Maximum Likelihood Estimation of t-Distribution with Unknown Degrees of Freedom
Example 5.5: Variance Components
Example 5.6: Factor Analysis
Efficient Data Augmentation
Alternating ECM Algorithm
EMS Algorithm
One-Step-Late Algorithm
Variance Estimation for Penalized EM and OSL Algorithms
Linear Inverse Problems
Miscellaneous Topics
Iterative Simulation Algorithms
Monte Carlo E-Step
Stochastic EM Algorithm
Data Augmentation Algorithm
Multiple Imputation
Sampling-Importance Resampling
Gibbs Sampler
Missing-Data Mechanism and Ignorability
Competing Methods and Some Comparisons with the EM Algorithm
The Delta Algorithm
Image Space Reconstruction Algorithm
Further Applications of the EM Algorithm