Gentle Introduction To Support Vector Machines In Biomedicine, A - Volume 1: Theory And Methods

by Alexander Statnikov, Constantin F Aliferis, Douglas P Hardin, Isabelle Guyon

Overview

Support Vector Machines (SVMs) are among the most important recent developments in pattern recognition and statistical machine learning. They have found a great range of applications in various fields including biology and medicine. However, biomedical researchers often experience difficulties grasping both the theory and applications of these important methods because of lack of technical background. The purpose of this book is to introduce SVMs and their extensions and allow biomedical researchers to understand and apply them in real-life research in a very easy manner. The book is to consist of two volumes: theory and methods (Volume 1) and case studies (Volume 2).

Product Details

ISBN-13:	9789814324380
Publisher:	World Scientific Publishing Company, Incorporated
Publication date:	03/02/2011
Pages:	200
Product dimensions:	7.60(w) x 9.10(h) x 0.70(d)

Table of Contents

Preface ix

About the Authors xiii

1 Introduction 1

Classes of Data-Analytic Problems Considered in This Book 1

Basic Principles of Classification 6

Main Ideas of the Support Vector Machine (SVM) Classification Algorithm 12

History of SVMs and Their Use in the Literature 16

2 Necessary Mathematical Concepts 19

Geometrical Representation of Objects 19

Basic Operations on Vectors 24

Hyperplanes as Decision Surfaces 29

Basics of Optimization 34

3 Support Vector Machines (SVMs) for Binary Classification: Classical Formulation 40

Hard-Margin Linear SVM for Linearly Separable Data 40

Soft-Margin Linear SVM for Data That is not Exactly linearly Separable Due to Noise or Outliers 49

Non-Linear SVM and Kernel Trick For Linearly Non-Separable Data 57

4 Basic Principles of Statistical Machine Learning 64

Generalization and Overfitting 64

"Loss + Penalty" Paradigm for Learning to Avoid Overfitting and Ensure Generalization 68

5 Model Selection for SVMs 73

Motivation of Model Selection Strategy 74

Commonly Used Parameters/Kernels of SVM Classifiers 79

Cross-Validation for Accuracy Estimation 81

Cross-Validation for Accuracy Estimation and Model Selection 85

Statistical Considerations 90

6 SVMs for Multi-Category Classification 91

One-Versus-Rest SVMs 91

One-Versus-One SVMs 91

Methods by Crammer and Singer and by Weston and Watkins 96

7 Support Vector Regression (SVR) 97

Hard-Margin Linear ε-Insensitive SVR for Modeling linear Relations 97

Soft-Margin Linear ε-Insensitive SVR for Modeling Almost Linear Relations 106

Non-Linear ε-Insensitive SVR for Modeling Non-Linear Relations 111

Comparing ε-Insensitive SVR with Other Popular Regression Methods 113

On Model Selection for ε-Insensitive SVR 118

8 Novelty Detection with SVM-Based Methods 119

Hard-Margin Linear One-Class SVM 123

Soft-Margin Linear One-Class SVM 125

Non-linear One-Class SVM 129

On Model Selection for One-Class SVM 135

9 Support Vector Clustering (Contributed by Nikita I. Lytkin) 136

The Minimal Enclosing Hyper-Sphere 140

Cluster Assignment in SVC 144

Dealing with Noise in the Data 148

Relationship Between the Minimal Enclosing Hyper-Sphere and One-Glass SVM 153

10 SVM-Based Variable Selection 154

Understanding the SVM Weight Vector 156

Simple SVM-Based Variable Selection Algorithm 161

SVM-RPE Variable Selection Algorithm 164

Variable Selection and Estimation of Generalization Accuracy 166

11 Computing Posterior Class Probabilities for SVM Classifiers 168

Simple Binning Method for Posterior Probability Estimation 168

Piatt's Method for Posterior Probability Estimation 171

12 Conclusions 174

Appendix 176

Bibliography 178

Index 181