On-Line Learning in Neural Networks

by David Saad (Editor)

Overview

On-line learning is one of the most commonly used techniques for training neural networks. Though it has been used successfully in many real-world applications, most training methods are based on heuristic observations. The lack of theoretical support damages the credibility as well as the efficiency of neural networks training, making it hard to choose reliable or optimal methods. This book presents a coherent picture of the state of the art in the theoretical analysis of on-line learning. An introduction relates the subject to other developments in neural networks and explains the overall picture. Surveys by leading experts in the field combine new and established material and enable nonexperts to learn more about the techniques and methods used. This book, the first in the area, provides a comprehensive view of the subject and will be welcomed by mathematicians, scientists and engineers, both in industry and academia.

Product Details

ISBN-13:	9780521652636
Publisher:	Cambridge University Press
Publication date:	01/28/1999
Series:	Publications of the Newton Institute , #17
Pages:	412
Product dimensions:	6.26(w) x 9.29(h) x 1.18(d)

Table of Contents

Foreword C. Bishop; 1. Introduction D. Saad; 2. On-line learning and stochastic approximations Léon Bottou; 3. Exact and perturbative solutions for the ensemble dynamics Todd Leen; 4. A statistical study of on-line learning Noboru Murata; 5. On-line learning in switching and drifting environments Klaus-Robert Mueller, Andreas Ziehe, Noboru Murata and Shun-ichi Amari; 6. Parameter adaptation in stochastic optimization Luis B. Almeida, Thibault Langlois, José D. Amaral and Alexander Plakhov; 7. Optimal on-line learning for multilayer neural networks David Saad and Magnus Rattray; 8. Universal asymptotics in committee machines with tree architecture Mauro Copelli and Nestor Caticha; 9. Incorporating curvature information in on-line learning Magnus Rattray and David Saad; 10. Annealed on-line learning in multilayer networks Siegfried Bös and Shun-ichi Amari; 11. On-line learning of prototypes and principal components Michael Biehl, Ansgar Freking, Matthias Hölzer, Georg Reents and Enno Schlösser; 12. On-line learning with time-correlated patterns Tom Heskes and Wim Wiegerinck; 13. On-line learning from finite training sets David Barber and Peter Sollich; 14. Dynamics of supervised learning with restricted training sets Anthony C. C. Coolen and David Saad; 15. On-line learning of a decision boundary with and without queries Yoshiyuki Kabashima and Shigeru Shinomoto; 16. A Bayesian approach to on-line learning Manfred Opper; 17. Optimal perception learning: an on-line Bayesian approach Sara A. Solla and Ole Winther.

Editorial Reviews

"I recommend this book to readers with a theoretical, analytical, or mathematical interest in neural networks, especially online learning." Computing Reviews

"The introduction gives a nice overview of on-line learning in neural networks and relates the subject to other developments in neural networks. The material provides a comprehensive view of the subject and is accessible to mathematicians, statisticians, and engineers in both industry and academia." Journal of the American Statistical Association

From the Publisher