Friedman C., Sandow S. Utility-Based Learning from Data

Chapman & Hall/CRC, 2011. — 417 p.
ISBN: 1584886226, 978-1584886228, e-ISBN: 978-1420011289.Utility-Based Learning from Data provides a pedagogical, self-contained discussion of probability estimation methods via a coherent approach from the viewpoint of a decision maker who acts in an uncertain environment. This approach is motivated by the idea that probabilistic models are usually not learned for their own sake; rather, they are used to make decisions. Specifically, the authors adopt the point of view of a decision maker who
(i) operates in an uncertain environment where the consequences of possible outcomes are explicitly monetized,
(ii) bases his decisions on a probabilistic model, and
(iii) builds and assesses his models accordingly.
These assumptions are naturally expressed in the language of utility theory, which is well known from finance and decision theory. By taking this point of view, the book sheds light on and generalizes some popular statistical learning approaches, connecting ideas from information theory, statistics, and finance. It strikes a balance between rigor and intuition, conveying the main ideas to as wide an audience as possible.Notions from Utility Theory, Model Performance Measurement, Model Estimation, The Viewpoint of This Book, Organization of This Book, Examples
Mathematical Preliminaries
Some Probabilistic Concepts: Probability Space, Random Variables, Probability Distributions,
Univariate Transformations of Random Variables, Multivariate Transformations of Random Variables,
Expectations, Some Inequalities; Joint, Marginal, and Conditional Probabilities; Conditional Expectations
Convex Optimization
Entropy and Relative Entropy
The Horse Race
The Basic Idea of an Investor in a Horse Race, The Expected Wealth Growth Rate, The Kelly Investor,
Entropy and Wealth Growth Rate, The Conditional Horse Race
Elements of Utility Theory
Beginnings: The St. Petersburg Paradox, Axiomatic Approach, Risk Aversion, Some Popular Utility Functions,
Field Studies, Our Assumptions, Blowup and Saturation
The Horse Race and Utility
The Discrete Unconditional Horse Races, Discrete Conditional Horse Races, Continuous Unconditional Horse Races,
Continuous Conditional Horse Races
Select Methods for Measuring Model Performance
Rank-Based Methods for Two-State Models, Likelihood, Performance Measurement via Loss Function
A Utility-Based Approach to Information Theory
Interpreting Entropy and Relative Entropy in the Discrete Horse Race Context,
(U,O)-Entropy and Relative (U,O)-Entropy for Discrete Unconditional Probabilities,
Conditional (U,O)-Entropy and Conditional Relative (U,O)-Entropy for Discrete Probabilities,
U-Entropy for Discrete Unconditional Probabilities
Utility-Based Model Performance Measurement
Utility-Based Performance Measures for Discrete Probability Models, Revisiting the Likelihood Ratio,
Utility-Based Performance Measures for Discrete Conditional Probability Models,
Utility-Based Performance Measures for Probability Density Models,
Utility-Based Performance Measures for Conditional Probability Density Models,
Monetary Value of a Model Upgrade, Some Proofs
Select Methods for Estimating Probabilistic Models
Classical Parametric Methods, Regularized Maximum-Likelihood Inference, Bayesian Inference,
Minimum Relative Entropy (MRE) Methods
A Utility-Based Approach to Probability Estimation
Discrete Probability Models, Conditional Density Models, Probability Estimation via Relative U-Entropy Minimization,
Expressing the Data Constraints in Purely Economic Terms, Some Proofs
Extensions
Model Performance Measures and MRE for Leveraged Investors,
Model Performance Measures and MRE for Investors in Incomplete Markets,
Utility-Based Performance Measures for Regression Models
Select Applications
Three Credit Risk Models, The Gail Breast Cancer Model, A Text Classification Model; A Fat-Tailed, Flexible, Asset Return Model