Theodoridis S. Machine Learning. A Bayesian and Optimization Perspective

Academic Press, 2015, -1072 p.Machine Learning is a name that is gaining popularity as an umbrella formethods that have been studied and developed for many decades in different scientific communities and under different names, such as Statistical Learning, Statistical Signal Processing, Pattern Recognition, Adaptive Signal Processing, Image Processing and Analysis, System Identification and Control, Data Mining and Information Retrieval, Computer Vision, and Computational Learning. The name Machine Learning indicates what all these disciplines have in common, that is, to learn from data, and then make predictions. What one tries to learn from data is their underlying structure and regularities, via the development of a model, which can then be used to provide predictions.
To this end, a number of diverse approaches have been developed, ranging from optimization of cost functions, whose goal is to optimize the deviation between what one observes from data and what the model predicts, to probabilistic models that attempt to model the statistical properties of the observed data.
The goal of this book is to approach the machine learning discipline in a unifying context, by presenting the major paths and approaches that have been followed over the years, without giving preference to a specific one. It is the author’s belief that all of them are valuable to the newcomer who wants to learn the secrets of this topic, from the applications as well as from the pedagogic point of view. As the title of the book indicates, the emphasis is on the processing and analysis front of machine learning and not on topics concerning the theory of learning itself and related performance bounds. In other words, the focus is on methods and algorithms closer to the application level.
The book is the outgrowth of more than three decades of the author’s experience on research and teaching various related courses. The book is written in such a way that individual (or pairs of) chapters are as self-contained as possible. So, one can select and combine chapters according to the focus he/she wants to give to the course he/she teaches, or to the topics he/she wants to grasp in a first reading. Some guidelines on how one can use the book for different courses are provided in the introductory chapter.
Each chapter grows by starting from the basics and evolving to embrace the more recent advances. Some of the topics had to be split into two chapters, such as sparsity-aware learning, Bayesian learning, probabilistic graphical models, and Monte Carlo methods. The book addresses the needs of advanced graduate, postgraduate, and research students as well as of practicing scientists and engineers whose interests lie beyond black-box solutions. Also, the book can serve the needs of short courses on specific topics, e.g., sparse modeling, Bayesian learning, probabilistic graphical models, neural networks and deep learning.
Most of the chapters include MatLAB exercises, and the related code is available from the book’s website. The solutions manual as well as PowerPoint lectures are also available from the book’s website.Probability and Stochastic Processes
Learning in Parametric Modeling: Basic Concepts and Directions
Mean-Square Error Linear Estimation
Stochastic Gradient Descent: The LMS Algorithm and its Family
The Least-Squares Family
Classification: A Tour of the Classics
Parameter Learning: A Convex Analytic Path
Sparsity-Aware Learning: Concepts and Theoretical Foundations
Sparsity-Aware Learning: Algorithms and Applications
Learning in Reproducing Kernel Hilbert Spaces
Bayesian Learning: Approximate Inference and Nonparametric Models
Monte Carlo Methods
Probabilistic Graphical Models: Part I
Probabilistic Graphical Models: Part II
Particle Filtering
Neural Networks and Deep Learning
Dimensionality Reduction
A Linear Algebra
B Probability Theory and Statistics
C Hints on Constrained Optimization