Neural Networks And Computing: Learning Algorithms And Applications

by Tommy Wai-shing Chow, David Siu-yeung Cho

Overview

This book covers neural networks with special emphasis on advanced learning methodologies and applications. It includes practical issues of weight initializations, stalling of learning, and escape from a local minima, which have not been covered by many existing books in this area. Additionally, the book highlights the important feature selection problem, which baffles many neural networks practitioners because of the difficulties handling large datasets. It also contains several interesting IT, engineering and bioinformatics applications.

Product Details

ISBN-13:	9781860947582
Publisher:	Imperial College Press
Publication date:	08/14/2007
Series:	Series In Electrical And Computer Engineering , #7
Pages:	324
Product dimensions:	6.30(w) x 9.30(h) x 0.90(d)

Table of Contents

Preface     V
Introduction     1
Background     1
Neuron Model     2
Historical Remarks     4
Network architecture     6
Supervised Neural Networks     6
McCulloh and Pitts Model     7
The Perceptron Model     11
Multi-layer Feedforward Network     14
Recurrent Networks     15
Unsupervised Neural Networks     17
Modeling and Learning Mechanism     19
Determination of Parameters     20
Gradient Descent Searching Method     26
Exercises     28
Learning Performance and Enhancement     31
Fundamental of Gradient Descent Optimization     32
Conventional Backpropagation Algorithm     35
Convergence Enhancement     42
Extended Backpropagation Algorithm     44
Least Squares Based Training Algorithm     47
Extended Least Squares Based Algorithm     55
Initialization Consideration     59
Weight Initialization Algorithm I     61
Weight Initialization Algorithm II     64
Weight Initialization Algorithm III     67
Global Learning Algorithms     69
Simulated Annealing Algorithm     70
Alopex Algorithm     71
Reactive Tabu Search     72
The NOVEL Algorithm     73
The Heuristic Hybrid Global Learning Algorithm     74
Concluding Remarks     82
Fast Learning Algorithms     82
Weight Initialization Methods     83
Global Learning Algorithms     84
Appendix 2.1     85
Exercises     87
Generalization and Performance Enhancement     91
Cost Function and Performance Surface     93
Maximum Likelihood Estimation     94
The Least-Square Cost Function     95
Higher-Order Statistic Generalization     98
Definitions and Properties of Higher-Order Statistics     99
The Higher-Order Cumulants based Cost Function     101
Property of the Higher-Order Cumulant Cost Function     105
Learning and Generalization Performance     108
Experiment one: Henon Attractor     109
Experiment Two: Sunspot time-series     116
Regularization for Generalization Enhancement     117
Adaptive Regularization Parameter Selection (ARPS) Method     120
Stalling Identification Method      121
[lambda] Selection Schemes     122
Synthetic Function Mapping     124
Concluding Remarks     126
Objective function selection     128
Regularization selection     129
Confidence Upper Bound of Approximation Error     131
Proof of the Property of the HOC Cost Function     133
The Derivation of the Sufficient Conditions of the Regularization Parameter     136
Exercises     137
Basis Function Networks for Classification     139
Linear Separation and Perceptions     140
Basis Function Model for Parametric Smoothing     142
Radial Basis Function Network     144
RBF Networks Architecture     144
Universal Approximation     146
Initialization and Clustering     149
Learning Algorithms     152
Linear Weights Optimization     152
Gradient Descent Optimization     154
Hybrid of Least Squares and Penalized Optimization     155
Regularization Networks     157
Advanced Radial Basis Function Networks     159
Support Vector Machine     159
Wavelet Network     161
Fuzzy RBF Controllers     164
Probabilistic Neural Networks     167
Concluding Remarks     169
Exercises     170
Self-organizing Maps     173
Introduction     173
Self-Organizing Maps     177
Learning Algorithm     178
Growing SOMs     182
Cell Splitting Grid     182
Growing Hierarchical Self-Organizing Quadtree Map     185
Probabilistic SOMs     188
Cellular Probabilistic SOM     188
Probabilistic Regularized SOM     193
Clustering of SOM     202
Multi-Layer SOM for Tree-Structured data     205
SOM Input Representation     207
MLSOM Training     210
MLSOM visualization and classification     212
Exercises     216
Classification and Feature Selection     219
Introduction     219
Support Vector Machines (SVM)     223
Support Vector Machine Visualization (SVMV)     224
Cost Function     229
MSE and MCE Cost Functions     230
Hybrid MCE-MSE Cost Function     232
Implementing MCE-MSE     236
Feature Selection     239
Information Theory     241
Mutual Information     241
Probability density function (pdf) estimation     243
MI Based Forward Feature Selection     245
MIFS and MIFS-U     247
Using quadratic MI     248
Exercises     253
Engineering Applications     255
Electric Load Forecasting     255
Nonlinear Autoregressive Integrated Neural Network Model     257
Case Studies     261
Content-based Image Retrieval Using SOM     266
GHSOQM Based CBIR Systems     267
Overall Architecture of GHSOQM-Based CBIR System     267
Image Segmentation, Feature Extraction and Region-Based Feature Matrices     268
Image Distance     269
GHSOQM and Relevance Feedback in the CBIR System     270
Performance Evaluation     274
Feature Selection for cDNA Microarray     278
MI Based Forward Feature Selection Scheme     279
The Supervised Grid Based Redundancy Elimination     280
The Forward Gene Selection Process Using MIIO and MISF     281
Results     282
Prostate Cancer Classification Dataset     284
Subtype of ALL Classification Dataset     288
Remarks      294
Bibliography     291
Index     305