Jin Y. (ed.) Multi-Objective Machine Learning

Springer, 2006. — 656.Feature selection and model selection are two major elements in machine learning. Both feature selection and model selection are inherently multi-objective optimization problems where more than one objective has to be optimized. For example in feature selection, minimization of the number of features and maximization of feature quality are two common objectives that are likely conflicting with each other. It is also widely realized that one has to deal with the trade-off between approximation or classification accuracy and model complexity in model selection.
Traditional machine learning algorithms try to satisfy multiple objectives by combining the objectives into a scalar cost function. A good example is the training of neural networks, where the main target is to minimize a cost function that accounts for the approximation or classification error on given training data. However, reducing the training error often leads to overfitting, which means that the error on unseen data will become very large, though the neural network performs perfectly on the training data. To improve the generalization capability of neural networks, i.e., to improve their ability to perform well on unseen data, a regularization term, e.g., the complexity of neural networks weighted by a hyper-parameter (regularization coefficient) has to be included in the cost function. One major challenge to implement the regularization technique is how to choose the regularization coefficient appropriately, which is non-trivial for most machine learning problems.
This book presents a collection of most representative research work on multi-objective machine learning.Part I Multi-Objective Clustering, Feature Extraction and Feature Selection
Feature Selection Using Rough Sets
Multi-Objective Clustering and Cluster Validation
Feature Selection for Ensembles Using the Multi-Objective Optimization Approach
Feature Extraction Using Multi-Objective Genetic Programming
Part II Multi-Objective Learning for Accuracy Improvement
Regression Error Characteristic Optimisation of Non-Linear Models
Regularization for Parameter Identification Using Multi-Objective Optimization
Multi-Objective Algorithms for Neural Networks Learning
Generating Support Vector Machines Using Multi-Objective Optimization and Goal Programming
Multi-Objective Optimization of Support Vector Machines
Multi-Objective Evolutionary Algorithm for Radial Basis Function Neural Network Design
Minimizing Structural Risk on Decision Tree Classification
Multi-objective Learning Classifier Systems
Part III Multi-Objective Learning for Interpretability Improvement
Simultaneous Generation of Accurate and Interpretable Neural Network Classifiers
GA-Based Pareto Optimization for Rule Extraction from Neural Networks
Agent Based Multi-Objective Approach to Generating Interpretable Fuzzy Systems
Multi-objective Evolutionary Algorithm for Temporal Linguistic Rule Extraction
Multiple Objective Learning for Constructing Interpretable Takagi-Sugeno Fuzzy Model
Part IV Multi-Objective Ensemble Generation
Pareto-Optimal Approaches to Neuro-Ensemble Learning
Trade-Off Between Diversity and Accuracy in Ensemble Generation
Cooperative Coevolution of Neural Networks and Ensembles of Neural Networks
Multi-Objective Structure Selection for RBF Networks and Its Application to Nonlinear System Identification
Fuzzy Ensemble Design through Multi-Objective Fuzzy Rule Selection
Part V Applications of Multi-Objective Machine Learning
Multi-Objective Optimisation for Receiver Operating Characteristic Analysis
Multi-Objective Design of Neuro-Fuzzy Controllers for Robot Behavior Coordination
Fuzzy Tuning for the Docking Maneuver Controller of an Automated Guided Vehicle
A Multi-Objective Genetic Algorithm for Learning Linguistic Persistent Queries in Text Retrieval Environments
Multi-Objective Neural Network Optimization for Visual Object Detection