Learning from Data: Concepts, Theory, and Methods / Edition 2

Hardcover (Print)
Used and New from Other Sellers
Used and New from Other Sellers
from $79.22
Usually ships in 1-2 business days
(Save 47%)
Other sellers (Hardcover)
  • All (9) from $79.22   
  • New (7) from $79.22   
  • Used (2) from $79.95   


An interdisciplinary framework for learning methodologies-covering statistics, neural networks, and fuzzy logic, this book provides a unified treatment of the principles and methods for learning dependencies from data. It establishes a general conceptual framework in which various learning methods from statistics, neural networks, and fuzzy logic can be applied-showing that a few fundamental principles underlie most new methods being proposed today in statistics, engineering, and computer science. Complete with over one hundred illustrations, case studies, and examples making this an invaluable text.

Read More Show Less

Editorial Reviews

From the Publisher
"I think Learning From Data is a very valuable volume. Iwill recommend it to my graduate students." (Journal of theAmerican Statistical Association, March 2009)

"The broad spectrum of information it offers is beneficial tomany field of research. The selection of topics is good, and Ibelieve that many researchers and practioners will find this bookuseful." (Technometrics, May 2008)

"The authors have succeeded in summarizing some of the recenttrends and future challenges in different learning methods,including enabling technologies and some interesting practicalapplications." (Computing Reviews, May 22, 2008)

This book contains considerable information on the concept of statistical learning theory.... However, some may find its presentation difficult to follow...
This book contains considerable information on the concept of statistical learning theory.... However, some may find its presentation difficult to follow...
Read More Show Less

Product Details

  • ISBN-13: 9780471681823
  • Publisher: Wiley
  • Publication date: 8/17/2007
  • Edition description: REV
  • Edition number: 2
  • Pages: 538
  • Sales rank: 1,280,161
  • Product dimensions: 6.46 (w) x 9.33 (h) x 1.28 (d)

Meet the Author

Vladimir CherKassky, PhD, is Professor of Electrical andComputer Engineering at the University of Minnesota. He isinternationally known for his research on neural networks andstatistical learning.

Filip Mulier, PhD, has worked in the software field for the lasttwelve years, part of which has been spent researching, developing,and applying advanced statistical and machine learning methods. Hecurrently holds a project management position.

Read More Show Less

Table of Contents



1 Introduction.

1.1 Learning and Statistical Estimation.

1.2 Statistical Dependency and Causality.

1.3 Characterization of Variables.

1.4 Characterization of Uncertainty.

1.5 Predictive Learning versus Other Data AnalyticalMethodologies.

2 Problem Statement, Classical Approaches, and AdaptiveLearning.

2.1 Formulation of the Learning Problem.

2.1.1 Objective of Learning.

2.1.2 Common Learning Tasks.

2.1.3 Scope of the Learning Problem Formulation.

2.2 Classical Approaches.

2.2.1 Density Estimation.

2.2.2 Classification.

2.2.3 Regression.

2.2.4 Solving Problems with Finite Data.

2.2.5 Nonparametric Methods.

2.2.6 Stochastic Approximation.

2.3 Adaptive Learning: Concepts and Inductive Principles.

2.3.1 Philosophy, Major Concepts, and Issues.

2.3.2 A Priori Knowledge and Model Complexity.

2.3.3 Inductive Principles.

2.3.4 Alternative Learning Formulations.

2.4 Summary.

3 Regularization Framework.

3.1 Curse and Complexity of Dimensionality.

3.2 Function Approximation and Characterization ofComplexity.

3.3 Penalization.

3.3.1 Parametric Penalties.

3.3.2 Nonparametric Penalties.

3.4 Model Selection (Complexity Control).

3.4.1 Analytical Model Selection Criteria.

3.4.2 Model Selection via Resampling.

3.4.3 Bias–Variance Tradeoff.

3.4.4 Example of Model Selection.

3.4.5 Function Approximation versus Predictive Learning.

3.5 Summary.

4 Statistical Learning Theory.

4.1 Conditions for Consistency and Convergence of ERM.

4.2 Growth Function and VC Dimension.

4.2.1 VC Dimension for Classification and RegressionProblems.

4.2.2 Examples of Calculating VC Dimension.

4.3 Bounds on the Generalization.

4.3.1 Classification.

4.3.2 Regression.

4.3.3 Generalization Bounds and Sampling Theorem.

4.4 Structural Risk Minimization.

4.4.1 Dictionary Representation.

4.4.2 Feature Selection.

4.4.3 Penalization Formulation.

4.4.4 Input Preprocessing.

4.4.5 Initial Conditions for Training Algorithm.

4.5 Comparisons of Model Selection for Regression.

4.5.1 Model Selection for Linear Estimators.

4.5.2 Model Selection for k-Nearest-Neighbor Regression.

4.5.3 Model Selection for Linear Subset Regression.

4.5.4 Discussion.

4.6 Measuring the VC Dimension.

4.7 VC Dimension, Occam’s Razor, and Popper’sFalsifiability.

4.8 Summary and Discussion.

5 Nonlinear Optimization Strategies.

5.1 Stochastic Approximation Methods.

5.1.1 Linear Parameter Estimation.

5.1.2 Backpropagation Training of MLP Networks.

5.2 Iterative Methods.

5.2.1 EM Methods for Density Estimation.

5.2.2 Generalized Inverse Training of MLP Networks.

5.3 Greedy Optimization.

5.3.1 Neural Network Construction Algorithms.

5.3.2 Classification and Regression Trees.

5.4 Feature Selection, Optimization, and Statistical LearningTheory.

5.5 Summary.

6 Methods for Data Reduction and DimensionalityReduction.

6.1 Vector Quantization and Clustering.

6.1.1 Optimal Source Coding in Vector Quantization.

6.1.2 Generalized Lloyd Algorithm.

6.1.3 Clustering.

6.1.4 EM Algorithm for VQ and Clustering.

6.1.5 Fuzzy Clustering.

6.2 Dimensionality Reduction: Statistical Methods.

6.2.1 Linear Principal Components.

6.2.2 Principal Curves and Surfaces.

6.2.3 Multidimensional Scaling.

6.3 Dimensionality Reduction: Neural Network Methods.

6.3.1 Discrete Principal Curves and Self-Organizing MapAlgorithm.

6.3.2 Statistical Interpretation of the SOM Method.

6.3.3 Flow-Through Version of the SOM and Learning RateSchedules.

6.3.4 SOM Applications and Modifications.

6.3.5 Self-Supervised MLP.

6.4 Methods for Multivariate Data Analysis.

6.4.1 Factor Analysis.

6.4.2 Independent Component Analysis.

6.5 Summary.

7 Methods for Regression.

7.1 Taxonomy: Dictionary versus Kernel Representation.

7.2 Linear Estimators.

7.2.1 Estimation of Linear Models and Equivalence ofRepresentations.

7.2.2 Analytic Form of Cross-Validation.

7.2.3 Estimating Complexity of Penalized Linear Models.

7.2.4 Nonadaptive Methods.

7.3 Adaptive Dictionary Methods.

7.3.1 Additive Methods and Projection Pursuit Regression.

7.3.2 Multilayer Perceptrons and Backpropagation.

7.3.3 Multivariate Adaptive Regression Splines.

7.3.4 Orthogonal Basis Functions and Wavelet SignalDenoising.

7.4 Adaptive Kernel Methods and Local Risk Minimization.

7.4.1 Generalized Memory-Based Learning.

7.4.2 Constrained Topological Mapping.

7.5 Empirical Studies.

7.5.1 Predicting Net Asset Value (NAV) of Mutual Funds.

7.5.2 Comparison of Adaptive Methods for Regression.

7.6 Combining Predictive Models.

7.7 Summary.

8 Classification.

8.1 Statistical Learning Theory Formulation.

8.2 Classical Formulation.

8.2.1 Statistical Decision Theory.

8.2.2 Fisher’s Linear Discriminant Analysis.

8.3 Methods for Classification.

8.3.1 Regression-Based Methods.

8.3.2 Tree-Based Methods.

8.3.3 Nearest-Neighbor and Prototype Methods.

8.3.4 Empirical Comparisons.

8.4 Combining Methods and Boosting.

8.4.1 Boosting as an Additive Model.

8.4.2 Boosting for Regression Problems.

8.5 Summary.

9 Support Vector Machines.

9.1 Motivation for Margin-Based Loss.

9.2 Margin-Based Loss, Robustness, and Complexity Control.

9.3 Optimal Separating Hyperplane.

9.4 High-Dimensional Mapping and Inner Product Kernels.

9.5 Support Vector Machine for Classification.

9.6 Support Vector Implementations.

9.7 Support Vector Regression.

9.8 SVM Model Selection.

9.9 Support Vector Machines and Regularization.

9.10 Single-Class SVM and Novelty Detection.

9.11 Summary and Discussion.

10 Noninductive Inference and Alternative LearningFormulations.

10.1 Sparse High-Dimensional Data.

10.2 Transduction.

10.3 Inference Through Contradictions.

10.4 Multiple-Model Estimation.

10.5 Summary.

11 Concluding Remarks.

Appendix A: Review of Nonlinear Optimization.

Appendix B: Eigenvalues and Singular Value Decomposition.



Read More Show Less

Customer Reviews

Be the first to write a review
( 0 )
Rating Distribution

5 Star


4 Star


3 Star


2 Star


1 Star


Your Rating:

Your Name: Create a Pen Name or

Barnes & Noble.com Review Rules

Our reader reviews allow you to share your comments on titles you liked, or didn't, with others. By submitting an online review, you are representing to Barnes & Noble.com that all information contained in your review is original and accurate in all respects, and that the submission of such content by you and the posting of such content by Barnes & Noble.com does not and will not violate the rights of any third party. Please follow the rules below to help ensure that your review can be posted.

Reviews by Our Customers Under the Age of 13

We highly value and respect everyone's opinion concerning the titles we offer. However, we cannot allow persons under the age of 13 to have accounts at BN.com or to post customer reviews. Please see our Terms of Use for more details.

What to exclude from your review:

Please do not write about reviews, commentary, or information posted on the product page. If you see any errors in the information on the product page, please send us an email.

Reviews should not contain any of the following:

  • - HTML tags, profanity, obscenities, vulgarities, or comments that defame anyone
  • - Time-sensitive information such as tour dates, signings, lectures, etc.
  • - Single-word reviews. Other people will read your review to discover why you liked or didn't like the title. Be descriptive.
  • - Comments focusing on the author or that may ruin the ending for others
  • - Phone numbers, addresses, URLs
  • - Pricing and availability information or alternative ordering information
  • - Advertisements or commercial solicitation


  • - By submitting a review, you grant to Barnes & Noble.com and its sublicensees the royalty-free, perpetual, irrevocable right and license to use the review in accordance with the Barnes & Noble.com Terms of Use.
  • - Barnes & Noble.com reserves the right not to post any review -- particularly those that do not follow the terms and conditions of these Rules. Barnes & Noble.com also reserves the right to remove any review at any time without notice.
  • - See Terms of Use for other conditions and disclaimers.
Search for Products You'd Like to Recommend

Recommend other products that relate to your review. Just search for them below and share!

Create a Pen Name

Your Pen Name is your unique identity on BN.com. It will appear on the reviews you write and other website activities. Your Pen Name cannot be edited, changed or deleted once submitted.

Your Pen Name can be any combination of alphanumeric characters (plus - and _), and must be at least two characters long.

Continue Anonymously

    If you find inappropriate content, please report it to Barnes & Noble
    Why is this product inappropriate?
    Comments (optional)