Steinwart I., Christmann A. Support Vector Machines

Springer, 2008, -610 p.The goal of this book is to explain the principles that made support vector machines (SVMs) a successful modeling and prediction tool for a variety of applications. We try to achieve this by presenting the basic ideas of SVMs together with the latest developments and current research questions in a unified style. In a nutshell, we identify at least three reasons for the success of SVMs: their ability to learn well with only a very small number of free parameters, their robustness against several types of model violations and outliers, and last but not least their computational efficiency compared with several other methods.
Although there are several roots and precursors of SVMs, these methods gained particular momentum during the last 15 years since Vapnik (1995, 1998) published his well-known textbooks on statistical learning theory with a special emphasis on support vector machines. Since then, the field of machine learning has witnessed intense activity in the study of SVMs, which has spread more and more to other disciplines such as statistics and mathematics. Thus it seems fair to say that several communities are currently working on support vector machines and on related kernel-based methods. Although there are many interactions between these communities, we think that there is still room for additional fruitful interaction and would be glad if this textbook were found helpful in stimulating further research. Many of the results presented in this book have previously been scattered in the journal literature or are still under review. As a consequence, these results have been accessible only to a relatively small number of specialists, sometimes probably only to people from one community but not the others. In view of the importance of SVMs for statistical machine learning, we hope that the unified presentation given here will make these results more accessible to researchers and users from different communities (e.g.; from the fields of statistics, mathematics, computer science, bioinformatics, data and text mining, and engineering).
As in most monographs, the selection of topics treated in this textbook is biased for several reasons. We have of course focused on those that particularly interest us and those that we have been working on during the last decade. We also decided to concentrate on some important and selected topics, so for these topics we can offer not only the results but also the proofs. This is in contrast to some other textbooks on SVMs or statistical machine learning in general, but we try to follow the path described by Devroye et al. (1996) and Gyorfi et al. (2002). Moreover, some topics, such as the robustness properties of SVMs, a detailed treatment of loss functions and reproducing kernel Hilbert spaces, recent advances in the statistical analysis of SVMs, and the relationship between good learning properties and good robustness properties such as a bounded influence function and a bounded maxbias, are not covered by other currently available books on SVMs. On the other hand, the area of statistical machine learning is nowadays so rich and progressing so rapidly that covering all aspects in detail in a single book hardly seems possible. The consequence is of course that several important and interesting topics of SVMs and related methods are not covered in this monograph. This includes, for example, SVMs for anomaly detection, kernel principal component analysis, kernel-based independence measures, structured estimation, recent progress in computational algorithms, boosting, Bayesian approaches, and the analysis of time series or text data. A reader interested in these topics will get useful information in the books by Vapnik (1995, 1998), Cristianini and Shawe-Taylor (2000), Hastie et al. (2001), Scholkopf and Smola (2002), Shawe-Taylor and Cristianini (2004), and Bishop (2006), among others. Moreover, many of the most recent developments can be found in journals such as Journal of Machine Learning Research, Machine Learning, and Annals of Statistics, or in the proceedings to conferences such as NIPS or COLT.
The process of writing this book took about four years. Springer asked one of us (A.C.), after a talk on robustness properties of SVMs, whether he was willing to write a book on this topic. After a few weeks to convince Ingo to write a joint textbook, our plan was to write a very condensed book of about 200–250 p. within one-and-a-half years. However, this soon turned out to be unrealistic because we aimed to include some of our own research results that were partially written simultaneously to the book. Moreover, we totally underestimated the richness of the available literature on SVMs and the field’s speed of progress.Loss Functions and Their Risks
Surrogate Loss Functions
Kernels and Reproducing Kernel Hilbert Spaces
Infinite-Sample Versions of Support Vector Machines
Basic Statistical Analysis of SVMs
Advanced Statistical Analysis of SVMs
Support Vector Machines for Classification
Support Vector Machines for Regression
Robustness
Computational Aspects
Data Mining