Simeone Osvaldo. A Brief Introduction to Machine Learning for Engineers

UK, King’s College London, 2018. — 237 p.This monograph is an attempt to offer a basic and compact reference that describes key ideas and principles in simple terms and within a unified treatment, encompassing also more recent developments and pointers to the liter- ature for further study.This monograph aims at providing an introduction to key concepts, algorithms, and theoretical results in machine learn- ing. The treatment concentrates on probabilistic models for supervised and unsupervised learning problems. It in- troduces fundamental concepts and algorithms by building on first principles, while also exposing the reader to more advanced topics with extensive pointers to the literature, within a unified notation and mathematical framework. The material is organized according to clearly defined categories, such as discriminative and generative models, frequentist and Bayesian approaches, exact and approximate inference, as well as directed and undirected models. This monograph is meant as an entry point for researchers with an engineer- ing background in probability and linear algebra.Basics
Introduction
What is Machine Learning?
When to Use Machine Learning?
Goals and Outline
A Gentle Introduction through Linear Regression
Supervised Learning
Inference
Frequentist Approach
Bayesian Approach
Minimum Description Length (MDL)
Information-Theoretic Metrics
Interpretation and CausalityProbabilistic Models for Learning
Preliminaries
The Exponential Family
Frequentist Learning
Bayesian Learning
Supervised Learning via Generalized Linear Models (GLM)
Maximum Entropy Property
Energy-based Models
Some Advanced TopicsSupervised Learning
Classification
Preliminaries: Stochastic Gradient Descent
Classification as a Supervised Learning Problem
Discriminative Deterministic Models
Discriminative Probabilistic Models: Generalized Linear Models
Discriminative Probabilistic Models: Beyond GLM
Generative Probabilistic Models
BoostingStatistical Learning Theory
A Formal Framework for Supervised Learning
PAC Learnability and Sample Complexity
PAC Learnability for Finite Hypothesis Classes
VC Dimension and Fundamental Theorem of PAC LearningUnsupervised Learning
Unsupervised Learning
Unsupervised Learning
K-Means Clustering
ML, ELBO and EM
Directed Generative Models
Undirected Generative Models
Discriminative Models
Autoencoders
RankingAdvanced Modeling and Inference
Probabilistic Graphical ModelsBayesian Networks
Markov Random Fields
Bayesian Inference in Probabilistic Graphical ModelsApproximate Inference and Learning
Monte Carlo Methods
Variational Inference
Monte Carlo-Based Variational Inference
Approximate LearningConclusions
Concluding RemarksAppendix A: Information Measures
Appendix B: KL Divergence and Exponential Family