Neural and Adaptive Systems: Fundamentals through Simulations / Edition 1

Paperback (Print)
Buy New
Buy New from
Used and New from Other Sellers
Used and New from Other Sellers
from $4.00
Usually ships in 1-2 business days
(Save 96%)
Other sellers (Paperback)
  • All (22) from $4.00   
  • New (7) from $109.48   
  • Used (15) from $4.00   


Develop New Insight into the Behavior of Adaptive Systems

This one-of-a-kind interactive book and CD-ROM will help you develop a better understanding of the behavior of adaptive systems. Developed as part of a project aimed at innovating the teaching of adaptive systems in science and engineering, it unifies the concepts of neural networks and adaptive filters into a common framework. It begins by explaining the fundamentals of adaptive linear regression and builds on these concepts to explore pattern classification, function approximation, feature extraction, and time-series modeling/prediction. The text is integrated with the industry standard neural network/adaptive system simulator NeuroSolutions. This allows the authors to demonstrate and reinforce key concepts using over 200 interactive examples. Each of these examples is 'live,' allowing the user to change parameters and experiment first-hand with real-world adaptive systems. This creates a powerful environment for learning through both visualization and experimentation. Key Features of the Text

  • The text and CD combine to become an interactive learning tool.
  • Emphasis is on understanding the behavior of adaptive systems rather than mathematical derivations.
  • Each key concept is followed by an interactive example.
  • Over 200 fully functional simulations of adaptive systems are included.
  • The text and CD offer a unified view of neural networks, adaptive filters, pattern recognition, and support vector machines.
  • Hyperlinks allow instant access to keyword definitions, bibliographic references, equations, and advanced discussions of concepts.

The CD-ROM Contains:

  • A complete, electronic version of the text in hypertext format
  • NeuroSolutions, an industry standard, icon-based neural network/adaptive system simulator
  • A tutorial on how to use NeuroSolutions
  • Additional data files to use with the simulator

"An innovative approach to describing neurocomputing and adaptive learning systems from a perspective which unifies classical linear adaptive systems approaches with the modern advances in neural networks. It is rich in examples and practical insight."
James Zeidler, University of California, San Diego

Read More Show Less

Editorial Reviews

Mats Rosengren
Mats Rosengren, MSc Engineering Physics Project Manager, Remote Sensing Forest Applications Satellus AB, Swedish Space Corporation Group

I would like to send you my very best congratulations to an extremely comprehensive and pedagogical book on neural systems. After more than 20 years in satellite remote sensing I found your book bringing together most of the different remote sensing analysis tools and methods used in a new and very systematic way. I have been trying to introduce the use of ANN techniques into our projects on forest classification and forest inventory applications combining ground sample measurements with airborne photos and satellite remote sensing in a hierarchical fashion. I found the Neuro Solutions software to be the most easily and yet flexible tool on the market. The problem has been to fully understand the tool and methods. Most courses on Neural Network techniques found in Sweden and all over Europe seem to be very technical. You need to be a C or C++ programmer, which I'm not. Also, most of the books on neural networks are very technical, describing only the different topologies and training methods but not a more general feeling of the practical use and opimal designs for different applications. The strengths of your book is the combination of a comprehensive and pedagicical description. A systematic building of the stastistical aspects and theory combined with the interactive hands on experience from running the examples. I got the book in my hand on friday afternoon (Dec 17, 1999) fresh from the print, and spent most of the weekend reading and running the examples. I like your approach from the "user" point of view. It is the first complete description (as far as I know) giving answers to most of my questions and thoughts on statistical methods, neural networks and classification. I will certainly recommend it to all my collegues in the field.
Read More Show Less

Product Details

  • ISBN-13: 9780471351672
  • Publisher: Wiley, John & Sons, Incorporated
  • Publication date: 12/21/1999
  • Edition number: 1
  • Pages: 672
  • Product dimensions: 9.25 (w) x 7.50 (h) x 1.43 (d)

Meet the Author

Jose C. Principe, University of FloridaNeil R. Euliano and W. Curt Lefebvre, both of NeuroDimension, Inc.

Read More Show Less

Read an Excerpt

Chapter 1: Data Fitting With Linear Models

1.1 Introduction

The study of neural and adaptive systems is a unique and growing interdisciplinary field that considers adaptive, distributed, and mostly nonlinear systems-three of the ingredients found in biology. We believe that neural and adaptive systems should be considered another tool in the scientist's and engineer's toolbox. They will effectively complement present engineering design principles and help build the preprocessors to interface with the real world and ensure the optimality needed in complex systems. When applied correctly, a neural or adaptive system may considerably outperform other methods.

Neural and adaptive systems are used in many important engineering applications, such as signal enhancement, noise cancellation, classification of input patterns, system identification, prediction, and control. They are used in many commercial products, such as modems, image-processing and -recognition systems, speech recognition, front-end signal processors, and biomedical instrumentation. We expect that the list will grow exponentially in the near future.

The leading characteristic of neural and adaptive systems is their adaptivity, which brings a totally new system design style (Figure 1-1). Instead of being built a priori from specification, neural and adaptive systems use external data to automatically set their parameters. This means that neural systems are parametric. It also means that they are made "aware" of their output through a performance feedback loop that includes a cost function. The performance feedback is utilized directly to change the parameters through systematic procedures called learning or training rules, so that the system output improves with respect to the desired goal (i.e., the error decreases through training).

The system designer has to specify just a few crucial steps in the overall process: He or she has to decide the system topology, choose a performance criterion, and design the adaptive algorithms. In neural systems the parameters are often modified in a selected set of data called the training set and are fixed during operation. The designer thus has to know how to specify the input and desired response data and when to stop the training phase. In adaptive systems the system parameters are continuously adapted during operation with the current data. We are at a very exciting stage in neural and adaptive system development because

  • We now know some powerful topologies that are able to create universal input/output mappings.
  • We also know how to design general adaptive algorithms to extract information from data and adapt the parameters of the mappers.
  • We are also starting to understand the prerequisites for generalization, that is, how to guarantee that the performance in the training set extends to the data found during system operation.

Therefore we are in a position to design effective adaptive solutions to moderately difficult real-world problems. Because of the practicality derived from these advances, we believe that the time is right to teach adaptive systems in undergraduate engineering and science curricula.

Throughout this textbook we explain the principles that are necessary to make judicious choices about the design options for neural and adaptive systems. The discussion is slanted toward engineering, both in terminology and in perspective. We are very much interested in the engineering model-based approach and in explaining the mathematical principles at work. We center the explanation on concepts from adaptive signal processing, which are rooted in statistics, pattern recognition, and digital signal processing. Moreover, our study is restricted to model building from data.

1.1.1 Engineering Design and Adaptive Systems

Engineering is a discipline that builds physical systems from human dreams, reinventing the physical world around us. In this respect it transcends physics, which has the passive role of explaining the world, and also mathematics, which stops at the edge of physical reality. Engineering design is like a gigantic Lego construction, where each piece is a subsystem grounded in its physical or mathematical principles. The role of the engineer is to develop the blueprint of the dream through specifications and then to look for the pieces that fit the blueprint. Obviously, the pieces cannot be put together at random, since each has its own principles, so it is mandatory that the scientist or the engineer learn the principles attached to each piece and specify the interface. Normally, this study is done using the scientific method. When the system is physical, we use the principles of physics, and when it is software, we use the principles of mathematics.

Development of the Phone System

A good example of engineering design is the telephone system. Long and meticulous research was conducted at Bell Laboratories on human perception of speech. This created the specification for the required bandwidth and noise level for speech intelligibility. Engineers perfected the microphone that would translate the pressure waves into electrical waves to meet the specification. Then these electrical waves were transmitted through copper wires over long distances to a similar device, still preserving the required specification. For increased functionality the freedom of reaching any other telephone was added to the system, so switching of calls had to be implemented. This created the phone system. Initially, the switching among lines was done by operators. Then a machine was invented that would automatically switch the calls. Operators were still used for special services such as directory assistance, but now that the fundamental engineering aspects are stable, we are asking machines to automatically recognize speech and directly assist callers.

The development of the phone system is an excellent example of engineering design. Once we have a vision, we try to understand the principles at work and create specifications and a system architecture. The fundamental principles at work are found by applying the scientific method. The phenomenon under analysis is first studied with physics or mathematics. The importance of models is that they translate general principles, and through deduction we can apply them to particular cases like the ones we are interested in. These disciplines create approximate models of the external world using the principle of divide and conquer. The problem is divided in manageable pieces, each is studied independently of the others, and protocols among the pieces are drawn such that the system can work as a whole, meeting the specifications drawn a priori. This is what engineering design is today.

The scientific method has been highly successful in engineering, but let us evaluate it in broad terms. First, engineering design requires the availability of a model for each subsystem. Second, when the number of pieces increases, the interactions among the subsystems increase exponentially. Fundamental research will continue to provide a steady flux of new physical and mathematical principles (provided the present trend of reduced federal funding for fundamental science is reversed), but the exponential growth of interactions required for larger and more sophisticated systems is harder to control. In fact, at this point in time, we simply do not have a clear vision of how to handle complexity in the long term. But there are two more factors that present major challenges: the autonomous interaction of systems with the environment and the optimality of the design. We will discuss these now.

Humans have traditionally mediated the interaction of engineering systems with the external world. After all, we use technology to reduce our physical constraints, so we have traditionally maintained control of the machines we build. Since the invention of the digital computer, there has been a trend to create machines that interact directly with the external world without a human in the loop. This brings the complexity of the external world directly into engineering design. We are not yet totally prepared for this, because our mathematical and physical theories about the external world are mere approximations-very good approximations in some cases, but rather poor in others. This disturbs the order of engineering design and creates performance problems (the worst subsystem tends to limit the performance of the full system).

Mars Pathfinder Mission

When machines have to autonomously interact with the environment or operate near the optimal set point, we cannot specify all the functions a priori and in a deterministic way. Take, for instance, the Mars Pathfinder mission. It was totally impossible to specify all the possible conditions that the rover Sojourner would face, even if remotely controlled from Earth, so the problem could not be solved by a sequence of instructions determined a priori in JPLs laboratory. The vehicle was given high-level instructions (way points) and was equipped with cameras and laser sensors that would see the terrain. The information from the sensors was analyzed and catalogued in general classes. For each class a procedure was designed to accomplish the goal of moving from point A to point B. This is the type of engineering system we will build more and more of in the future.

The big difference between the initial machines and Sojourner is that the environment is intrinsically in the loop of the machine function. This brings a very different set of problems, because, as we said earlier the environment is complex and unpredictable. If our physical model does not capture the essentials of the environment, errors accumulate over time, and the solution becomes impractical. We thus no longer have the luxury of dictating the rules of the game, as we did in the early machine-building era. It turns out that animals and humans do Sojourner-type tasks effortlessly.

System optimality is also a rising concern to save resources and augment the performance/price ratio. We might think that optimally designing each subsystem would bring global optimality, but this is not always true. Optimal design of complex systems is thus a difficult problem that must also take into consideration the particular type of system function, meaning that the complexity of the environment is once again present. We can conclude that the current challenges faced in engineering are the complexity of the systems, the need for optimal performance, and the autonomous interaction with the environment that will require some form of intelligence. These are the challenges for engineering in the 21st century and beyond.

Whenever there is a new challenge, we should consider new solutions. Quite often the difficulty of a task is also linked to the particular method we are using to find the solution. Is building machines by specification the only way to proceed?

Let us look at living creatures from an engineering systems perspective. The cell is the optimal factory, building directly from the environment at the fundamental molecular level what it needs to carry out its function. The animals we observe today interact efficiently with the environment (otherwise they would not have survived); they work very close to optimality in terms of resources (otherwise they would have been replaced in their niche by more efficient animals); and they certainly are complex. Biology has, in fact, already conquered some of the challenges we face in building engineering Systems, so it is worthwhile to investigate the principles at work.

Biology has found a set of inductive principles that are particularly well tuned to the interaction with a complex and unpredictable environment. These principles are not known explicitly but are being intensively studied in biology, computational neurosciences, statistics, computer science, and engineering. They involve extraction of information from sensor data (feature extraction), efficient learning from data, creation of invariants and representations, and decision making under uncertainty. In a global sense, autonomous agents have to build and fit models to data through their daily experience; they have to store these models, choose which shall be applied in each circumstance, and assess the likelihood of success for a given task. An implicit optimization principle is at play, since the goal is to do the best with the available information and resources.

From a scientific perspective, biology uses adaptation to build optimal system functionality. The anatomical organization of the animal (the wetware) is specified in the long term by the environment (through evolution), and in the short term it is used as a constraint to extract in real time the information that the animal needs to survive. At the nervous-system level, it is well accepted that the interaction with the environment molds the wetware using a learning-from-examples metaphor.

1.1.2 Experimental Model Building

The problem of data fitting is one of the oldest in experimental science. The real world tends to be very complex and unpredictable, and the exact mechanisms that generate the data are often unknown. Moreover, when we collect physical variables, the sensors are not ideal (of finite precision, noisy, with constrained bandwidth, etc.), so the measurements do not represent the real phenomena exactly. One of the quests in science is to estimate the underlying data model.

The importance of inferring a model from the data is to apply mathematical reasoning to the problem. The major advantage of a mathematical model is the ability to understand, explain, predict, and control outcomes in the natural system [Casti, 1989]. Figure 1-2 illustrates the data-modeling process. The most important advantage of the existence of a formal equivalent model is the ability to predict the natural system's behavior at a future time and to control its outputs by applying appropriate inputs.

In this chapter we address the issues of fitting data with linear models, which is called the linear regression problem [Dunteman, 1989]. Notice that we have not specified what the data is, because it is really immaterial. We are seeking relationships between the values of the external (observable) variables of the natural system in Figure 1-2. This methodology can therefore be applied to meteorological data, biological data, financial data, marketing data, engineering data, and so on.

1.1.3 Data Collection

The data-collection phase must be carefully planned to ensure that

  • Data will be sufficient.
  • Data will capture the fundamental principles at work.
  • Data is as free as possible from observation noise.

Table 1-1 presents a data example with two variables (x, d) in tabular form. the measurement x is assumed error free, and d is contaminated by noise. From table 1-1 very little can be said about the data, except that there is a positive trend between the variables (i.e., when x increases d also increases). Our brain is somehow able to extract much more information from pictures than from numbers, so data should first be plotted before performing data analysis. Plotting the data allows verification, assures the researcher that the data was collected correctly, and provides a "feel" for the relationships that exist in the data (e.g., natural trends)....

Read More Show Less

Table of Contents

Data Fitting with Linear Models.

Pattern Recognition.

Multilayer Perceptrons.

Designing and Training MLPs.

Function Approximation with MLPs, Radial Basis Functions, and Support Vector Machines.

Hebbian Learning and Principal Component Analysis.

Competitive and Kohonen Networks.

Principles of Digital Signal Processing.

Adaptive Filters.

Temporal Processing with Neural Networks.

Training and Using Recurrent Networks.




Read More Show Less

Customer Reviews

Be the first to write a review
( 0 )
Rating Distribution

5 Star


4 Star


3 Star


2 Star


1 Star


Your Rating:

Your Name: Create a Pen Name or

Barnes & Review Rules

Our reader reviews allow you to share your comments on titles you liked, or didn't, with others. By submitting an online review, you are representing to Barnes & that all information contained in your review is original and accurate in all respects, and that the submission of such content by you and the posting of such content by Barnes & does not and will not violate the rights of any third party. Please follow the rules below to help ensure that your review can be posted.

Reviews by Our Customers Under the Age of 13

We highly value and respect everyone's opinion concerning the titles we offer. However, we cannot allow persons under the age of 13 to have accounts at or to post customer reviews. Please see our Terms of Use for more details.

What to exclude from your review:

Please do not write about reviews, commentary, or information posted on the product page. If you see any errors in the information on the product page, please send us an email.

Reviews should not contain any of the following:

  • - HTML tags, profanity, obscenities, vulgarities, or comments that defame anyone
  • - Time-sensitive information such as tour dates, signings, lectures, etc.
  • - Single-word reviews. Other people will read your review to discover why you liked or didn't like the title. Be descriptive.
  • - Comments focusing on the author or that may ruin the ending for others
  • - Phone numbers, addresses, URLs
  • - Pricing and availability information or alternative ordering information
  • - Advertisements or commercial solicitation


  • - By submitting a review, you grant to Barnes & and its sublicensees the royalty-free, perpetual, irrevocable right and license to use the review in accordance with the Barnes & Terms of Use.
  • - Barnes & reserves the right not to post any review -- particularly those that do not follow the terms and conditions of these Rules. Barnes & also reserves the right to remove any review at any time without notice.
  • - See Terms of Use for other conditions and disclaimers.
Search for Products You'd Like to Recommend

Recommend other products that relate to your review. Just search for them below and share!

Create a Pen Name

Your Pen Name is your unique identity on It will appear on the reviews you write and other website activities. Your Pen Name cannot be edited, changed or deleted once submitted.

Your Pen Name can be any combination of alphanumeric characters (plus - and _), and must be at least two characters long.

Continue Anonymously
Sort by: Showing 1 Customer Reviews
  • Anonymous

    Posted January 7, 2000

    For a change - a title that delivers what it promises!

    This has got to be the best textbook I have come across! The amount of thought and care in producing this book is phenomenal. It goes into details very thoroughly, and provides plenty of examples which would easily satisfy the very bright and the very slow. The math in back of all this is ACTUALLY EXPLAINED PROPERLY! Buy the book - it's worth every cent.

    Was this review helpful? Yes  No   Report this review
Sort by: Showing 1 Customer Reviews

If you find inappropriate content, please report it to Barnes & Noble
Why is this product inappropriate?
Comments (optional)