Density Ratio Estimation in Machine Learning

by Masashi Sugiyama, Taiji Suzuki, Takafumi Kanamori

Overview

Machine learning is an interdisciplinary field of science and engineering that studies mathematical theories and practical applications of systems that learn. This book introduces theories, methods, and applications of density ratio estimation, which is a newly emerging paradigm in the machine learning community. Various machine learning problems such as non-stationarity adaptation, outlier detection, dimensionality reduction, independent component analysis, clustering, classification, and conditional density estimation can be systematically solved via the estimation of probability density ratios. The authors offer a comprehensive introduction of various density ratio estimators including methods via density estimation, moment matching, probabilistic classification, density fitting, and density ratio fitting as well as describing how these can be applied to machine learning. The book also provides mathematical theories for density ratio estimation including parametric and non-parametric convergence analysis and numerical stability analysis to complete the first and definitive treatment of the entire framework of density ratio estimation in machine learning.

Product Details

ISBN-13:	9780521190176
Publisher:	Cambridge University Press
Publication date:	02/20/2012
Edition description:	New Edition
Pages:	342
Product dimensions:	6.18(w) x 9.21(h) x 0.91(d)

About the Author

Masashi Sugiyama is an Associate Professor in the Department of Computer Science at the Tokyo Institute of Technology.

Taiji Suzuki is an Assistant Professor in the Department of Mathematical Informatics at the University of Tokyo, Japan.

Takafumi Kanamori is an Associate Professor in the Department of Computer Science and Mathematical Informatics at Nagoya University, Japan.

Table of Contents

Part I. Density Ratio Approach to Machine Learning: 1. Introduction; Part II. Methods of Density Ratio Estimation: 2. Density estimation; 3. Moment matching; 4. Probabilistic classification; 5. Density fitting; 6. Density-ratio fitting; 7. Unified framework; 8. Direct density-ratio estimation with dimensionality reduction; Part III. Applications of Density Ratios in Machine Learning: 9. Importance sampling; 10. Distribution comparison; 11. Mutual information estimation; 12. Conditional probability estimation; Part IV. Theoretical Analysis of Density Ratio Estimation: 13. Parametric convergence analysis; 14. Non-parametric convergence analysis; 15. Parametric two-sample test; 16. Non-parametric numerical stability analysis; Part V. Conclusions: 17. Conclusions and future directions.

Editorial Reviews

"There is no doubt that this book will change the way people think about machine learning and stimulate many new directions for research."
From the Foreword by Thomas G. Dietterich

"The book is well written and produced, and will probably be seen in retrospect as a significant addition to the literature in this important area—at least to the extent that density ratio estimation as a technique proves useful in real-world applications. Future work and applications using the theory presented should indicate to what extent this happens."
Shrisha Rao, Computing Reviews

"This book is clear and well written, and it is an excellent introduction to density ratio estimation in both theory and practice. It presents the state-of-the-art methodology on this topic and in this regard it is really nice that the bibliography is so exhaustive and well commented."
Pierre Alquir, Mathematical Reviews

From the Publisher