Lan Guanghui. First-order and Stochastic Optimization Methods for Machine Learning

New York: Springer, 2020. — 591 p.This book covers not only foundational materials but also the most recent progress made during the past few years in the area of Machine Learning algorithms. Despite the intensive research and development in this area, there does not exist a systematic treatment to introduce the fundamental concepts and recent progress on Machine Learning algorithms, especially on those based on stochastic optimization methods, randomized algorithms, nonconvex optimization, distributed and online learning, and projection free methods. This book will benefit the broad audience in the area of Machine Learning, Artificial Intelligence, and the mathematical programming community by presenting these recent developments in a tutorial style, starting from the basic building blocks to the most carefully designed and complicated algorithms for Machine Learning.In the past few years, deep learning has generated much excitement in Machine Learning, especially in the industry, due to many breakthrough results in speech recognition, computer vision, and text processing. For many researchers, deep learning is another name for a set of algorithms that use a neural network as an architecture. Even though neural networks have a long history, they became more successful in recent years due to the availability of inexpensive, parallel hardware (GPUs, computer clusters), massive amounts of data, and the recent development of efficient optimization algorithms, especially those designed for population risk minimization.