This original work offers the most comprehensive and up-to-date treatment of the important subject of optimal linear estimation, which is encountered in many areas of engineering such as communications, control, and signal processing, and also several other fields, e.g., econometrics and statistics. The book not only highlights the most significant contributions to this field during the 20th century, including the works of Weiner and Kalman, but it does so in an original and novel manner that paves the way for further developments in the new millennium. This book contains a large collection of problems that complement the text and are an important part of it, in addition to numerous sections that offer interesting historical accounts and insights.
Largely focuses on estimation problems for finite-dimensional linear systems with state-space models, covering most aspects of an area now generally known as Wiener and Kalman filtering theory. Distinctive features the treatment are the pervasive use of a geometric point of view; the emphasis on the numerically favored square root/array forms of many algorithms; and the emphasis on equivalence and duality concepts for the solution of several related problems in adaptive filtering, estimation, and control. The authors argue that these features are not as abstract and complicated as other treatments of the topic seem to fear. Annotation c. Book News, Inc., Portland, OR booknews.com
The problem of estimating the values of a random (or stochastic) process given observations of a related random process is encountered in many areas of science and engineering, e.g., communications, control, signal processing, geophysics, econometrics, and statistics. Although the topic has a rich history, and its formative stages can be attributed to illustrious investigators such as Laplace, Gauss, Legendre, and others, the current high interest in such problems began with the work of H. Wold, A. N. Kolmogorov, and N. Wiener in the late 1930s and early 1940s. N. Wiener in particular stressed the importance of modeling not just "noise" but also "signals" as random processes. His thought-provoking originally classified 1942 report, released for open publication in 1949 and now available in paperback form under the title Time Series Analysis, is still very worthwhile background reading.
As with all deep subjects, the extensions of these results have been very far-reaching as well. A particularly important development arose from the incorporation into the theory of multichannel state-space models. Though there were various earlier partial intimations and explorations, especially in the work of R. L. Stratonovich- in the former Soviet Union, the chief credit for the explosion of activity in this direction goes to R. E. Kalman, who also made important related contributions to linear systems, optimal control, passive systems, stability theory, and network synthesis.
In fact, least-squares estimation is one of those happy subjects that is interesting not only in the richness and scope of its results, but also because of its mutually beneficialconnections with a host of other (often apparently very different) subjects. Thus, beyond those already named, we may mention connections with radiative transfer and scattering theory, linear algebra, matrix and operator theory, orthogonal polynomials, moment problems, inverse scattering problems, interpolation theory, decoding of Reed-Solomon and BCH codes, polynomial factorization and root distribution problems, digital filtering, spectral analysis, signal detection, martingale theory, the so-called Hh theories of estimation and control, least-squares and adaptive filtering problems, and many others. We can surely apply to it the lines written by William Shakespeare about another (beautiful) subject:
"Age does not wither her, nor custom stale,
Her infinite variety."
Though we were originally tempted to cover a wider range, many reasons have led us to focus this volume largely on estimation problems for finite-dimensional linear systems with state-space models, covering most aspects of an area now generally known as Wiener and Kalman filtering theory. Three distinctive features of our treatment are the pervasive use of a geometric point of view, the emphasis on the numerically favored square-root/array forms of many algorithms, and the emphasis on equivalence and duality concepts for the solution of several related problems in adaptive filtering, estimation, and control. These features are generally absent in most prior treatments, ostensibly on the grounds that they are too abstract and complicated. It is our hope that these misconceptions will be dispelled by the presentation herein, and that the fundamental simplicity and power of these ideas will be more widely recognized arid exploited.
The material presented in this book can be broadly categorized into the following topics:
Introduction and Foundations
Chapter 1: Overview
Chapter 2: Deterministic Least-Squares Problems
Chapter 3: Stochastic Least-Squares Problems
Chapter 4: The Innovations Process
Chapter 5: State-Space Models
Estimation of Stationary Processes
Chapter 6: Innovations for Stationary Processes
Chapter 7: Wiener Theory for Scalar Processes
Chapter 8: Recursive Wiener Filters
Estimation of Nonstationary Processes
Chapter 9: The Kalman Filter
Chapter 10: Smoothed Estimators
Fast and Array Algorithms
Chapter 11: Fast Algorithms
Chapter 12: Array Algorithms
Chapter 13: Fast Array Algorithms
Advanced Topics
Chapter 14: Asymptotic Behavior
Chapter 15: Duality and Equivalence in Estimation and Control
Chapter 17: A Scattering Theory Approach
Being intended for a graduate-level course, the book assumes familiarity with basic concepts from matrix theory, linear algebra, linear system theory, and random processes. Four appendices at the end of the book provide the reader with background material in all these areas.
There is ample material in this book for the instructor to fashion a course to his or her needs and tastes. The authors have used portions of this book as the basis for one-quarter first-year graduate level courses at Stanford University, the University of California at Los Angeles, and the University of California at Santa Barbara; the students were expected to have had some exposure to discrete-time and state-space theory. A typical course would start with Secs.1.1-1.2 as an overview (perhaps omitting the matrix derivations), with the rest of Ch. 1 left for a quick reading (and re-reading from time to time), most of Chs. 2 and 3 (focusing on the geometric approach) on the basic deterministic and stochastic least-squares problems, Ch. 4 on the innovations process, Secs. 6.4-6.5 and 7.3-7.7 on scalar Wiener filtering, Secs. 9.1-9.3, 9.5, and 9.7 on Kalman filtering, Secs. 10.1-10.2 as an introduction to smoothing, Secs. 12.1-12.5 and 13.1-13.4 on array algorithms, and Secs. 16.1-16.4 and 16.6 on continuous-time problems.
More advanced students and researchers would pursue selections of material from Sec. 2.8, Chs. 8, 11, 14, 15, and 17, and Apps. E and R These cover, among other topics, least-squares problems with uncertain data, the problem of canonical spectral factorization, convergence of the Kalman filter, the algebraic Riccati equation, duality, backwards-time and complementary models, scattering, etc. Those wishing to go on to the more recent H
¥ theory can find a treatment closely related to the philosophy of the current book (cf. Sec. 1.6) in the research monograph of Hassibi, Sayed, and Kailath (1999).
A feature of the book is a collection of nearly 300 problems, several of which complement the text and present additional results and insights. However, there is little discussion of real applications or of the error and sensitivity analyses required for them. The main issue in applications is constructing an appropriate model, or actually a set of models, which are further analyzed and then refined by using the results and algorithms presented in this book. Developing good models and analyzing them effectively requires not only a good appreciation of the actual application, but also a good understanding of the theory, at both an analytical and intuitive level. It is the latter that we have tried to achieve here; examples of successful applications have to be sought in the literature, and some references are provided to this end.
Acknowledgments
The development of this textbook has spanned many years. So the material, as well as its presentation, has benefited greatly from the inputs of the many bright students who have worked with us on these topics: J. Omura, P Frost, T Duncan, R. Geesey, D. Duttweiler, H. Aasnaes, M. Gevers, H. Weinert, A. Segall, M. Mort B. Dickinson, G. Sidhu, B. Friedlander, A. Vieira, S. Y Kung, B. Levy, G. Verghese, D. Lee, J. Delosme, B. Porat, H. Lev-Ari, J. Cioffi, A. Bruckstein, T. Citron, Y Bresler, R. Roy, J. Chun, D. Slock, D. Pal, G. Xu, R. Ackner, Y Cho, P Park, T. Boros, A. Erdogan, U. Forsell, B. Halder, H. Hindi, V Nascimento, T. Pare, R. Merched, and our young friend Amir Ghazanfarian (in memoriam) from whom we had so much more to learn.
We are of course also deeply indebted to the many researchers and authors in this beautiful field. Partial acknowledgment is evident through the citations and references; while the list of the latter is quite long, we apologize for omissions and inadequacies arising from the limitations of our knowledge and our energy. Nevertheless, we would be remiss not to explicitly mention the inspiration and pleasure we have gained in studying the papers and books of N. Wiener, R. E. Kalman, and P Whittle.
Major support for the many years of research that led to this book was provided by the Mathematics Divisions of the Air Force Office of Scientific Research and the Army Research Office, by the Joint Services Electronics Program, by the Defense Advanced Research Projects Agency, and by the National Science Foundation. Finally, we would like to thank Bernard Goodwin and Tom Robbins, as well as the staff of Prentice Hall, for their patience and other contributions to this project.
The problem of estimating the values of a random (or stochastic) process given observations of a related random process is encountered in many areas of science and engineering, e.g., communications, control, signal processing, geophysics, econometrics, and statistics. Although the topic has a rich history, and its formative stages can be attributed to illustrious investigators such as Laplace, Gauss, Legendre, and others, the current high interest in such problems began with the work of H. Wold, A. N. Kolmogorov, and N. Wiener in the late 1930s and early 1940s. N. Wiener in particular stressed the importance of modeling not just "noise" but also "signals" as random processes. His thought-provoking originally classified 1942 report, released for open publication in 1949 and now available in paperback form under the title Time Series Analysis, is still very worthwhile background reading.
As with all deep subjects, the extensions of these results have been very far-reaching as well. A particularly important development arose from the incorporation into the theory of multichannel state-space models. Though there were various earlier partial intimations and explorations, especially in the work of R. L. Stratonovich- in the former Soviet Union, the chief credit for the explosion of activity in this direction goes to R. E. Kalman, who also made important related contributions to linear systems, optimal control, passive systems, stability theory, and network synthesis.
In fact, least-squares estimation is one of those happy subjects that is interesting not only in the richness and scope of its results, but also because of its mutually beneficial connections with a host of other (often apparently very different) subjects. Thus, beyond those already named, we may mention connections with radiative transfer and scattering theory, linear algebra, matrix and operator theory, orthogonal polynomials, moment problems, inverse scattering problems, interpolation theory, decoding of Reed-Solomon and BCH codes, polynomial factorization and root distribution problems, digital filtering, spectral analysis, signal detection, martingale theory, the so-called Hh theories of estimation and control, least-squares and adaptive filtering problems, and many others. We can surely apply to it the lines written by William Shakespeare about another (beautiful) subject:
"Age does not wither her, nor custom stale,
Her infinite variety."
Though we were originally tempted to cover a wider range, many reasons have led us to focus this volume largely on estimation problems for finite-dimensional linear systems with state-space models, covering most aspects of an area now generally known as Wiener and Kalman filtering theory. Three distinctive features of our treatment are the pervasive use of a geometric point of view, the emphasis on the numerically favored square-root/array forms of many algorithms, and the emphasis on equivalence and duality concepts for the solution of several related problems in adaptive filtering, estimation, and control. These features are generally absent in most prior treatments, ostensibly on the grounds that they are too abstract and complicated. It is our hope that these misconceptions will be dispelled by the presentation herein, and that the fundamental simplicity and power of these ideas will be more widely recognized arid exploited.
The material presented in this book can be broadly categorized into the following topics:
Introduction and Foundations
Chapter 1: Overview
Chapter 2: Deterministic Least-Squares Problems
Chapter 3: Stochastic Least-Squares Problems
Chapter 4: The Innovations Process
Chapter 5: State-Space Models
Estimation of Stationary Processes
Chapter 6: Innovations for Stationary Processes
Chapter 7: Wiener Theory for Scalar Processes
Chapter 8: Recursive Wiener Filters
Estimation of Nonstationary Processes
Chapter 9: The Kalman Filter
Chapter 10: Smoothed Estimators
Fast and Array Algorithms
Chapter 11: Fast Algorithms
Chapter 12: Array Algorithms
Chapter 13: Fast Array Algorithms
Advanced Topics
Chapter 14: Asymptotic Behavior
Chapter 15: Duality and Equivalence in Estimation and Control
Chapter 17: A Scattering Theory Approach
Being intended for a graduate-level course, the book assumes familiarity with basic concepts from matrix theory, linear algebra, linear system theory, and random processes. Four appendices at the end of the book provide the reader with background material in all these areas.
There is ample material in this book for the instructor to fashion a course to his or her needs and tastes. The authors have used portions of this book as the basis for one-quarter first-year graduate level courses at Stanford University, the University of California at Los Angeles, and the University of California at Santa Barbara; the students were expected to have had some exposure to discrete-time and state-space theory. A typical course would start with Secs.1.1-1.2 as an overview (perhaps omitting the matrix derivations), with the rest of Ch. 1 left for a quick reading (and re-reading from time to time), most of Chs. 2 and 3 (focusing on the geometric approach) on the basic deterministic and stochastic least-squares problems, Ch. 4 on the innovations process, Secs. 6.4-6.5 and 7.3-7.7 on scalar Wiener filtering, Secs. 9.1-9.3, 9.5, and 9.7 on Kalman filtering, Secs. 10.1-10.2 as an introduction to smoothing, Secs. 12.1-12.5 and 13.1-13.4 on array algorithms, and Secs. 16.1-16.4 and 16.6 on continuous-time problems.
More advanced students and researchers would pursue selections of material from Sec. 2.8, Chs. 8, 11, 14, 15, and 17, and Apps. E and R These cover, among other topics, least-squares problems with uncertain data, the problem of canonical spectral factorization, convergence of the Kalman filter, the algebraic Riccati equation, duality, backwards-time and complementary models, scattering, etc. Those wishing to go on to the more recent H ¥ theory can find a treatment closely related to the philosophy of the current book (cf. Sec. 1.6) in the research monograph of Hassibi, Sayed, and Kailath (1999).
A feature of the book is a collection of nearly 300 problems, several of which complement the text and present additional results and insights. However, there is little discussion of real applications or of the error and sensitivity analyses required for them. The main issue in applications is constructing an appropriate model, or actually a set of models, which are further analyzed and then refined by using the results and algorithms presented in this book. Developing good models and analyzing them effectively requires not only a good appreciation of the actual application, but also a good understanding of the theory, at both an analytical and intuitive level. It is the latter that we have tried to achieve here; examples of successful applications have to be sought in the literature, and some references are provided to this end.
Acknowledgments
The development of this textbook has spanned many years. So the material, as well as its presentation, has benefited greatly from the inputs of the many bright students who have worked with us on these topics: J. Omura, P Frost, T Duncan, R. Geesey, D. Duttweiler, H. Aasnaes, M. Gevers, H. Weinert, A. Segall, M. Mort B. Dickinson, G. Sidhu, B. Friedlander, A. Vieira, S. Y Kung, B. Levy, G. Verghese, D. Lee, J. Delosme, B. Porat, H. Lev-Ari, J. Cioffi, A. Bruckstein, T. Citron, Y Bresler, R. Roy, J. Chun, D. Slock, D. Pal, G. Xu, R. Ackner, Y Cho, P Park, T. Boros, A. Erdogan, U. Forsell, B. Halder, H. Hindi, V Nascimento, T. Pare, R. Merched, and our young friend Amir Ghazanfarian (in memoriam) from whom we had so much more to learn.
We are of course also deeply indebted to the many researchers and authors in this beautiful field. Partial acknowledgment is evident through the citations and references; while the list of the latter is quite long, we apologize for omissions and inadequacies arising from the limitations of our knowledge and our energy. Nevertheless, we would be remiss not to explicitly mention the inspiration and pleasure we have gained in studying the papers and books of N. Wiener, R. E. Kalman, and P Whittle.
Major support for the many years of research that led to this book was provided by the Mathematics Divisions of the Air Force Office of Scientific Research and the Army Research Office, by the Joint Services Electronics Program, by the Defense Advanced Research Projects Agency, and by the National Science Foundation. Finally, we would like to thank Bernard Goodwin and Tom Robbins, as well as the staff of Prentice Hall, for their patience and other contributions to this project.
Our reader reviews allow you to share your comments on titles you liked,
or didn't, with others. By submitting an online review, you are representing to
Barnes & Noble.com that all information contained in your review is original
and accurate in all respects, and that the submission of such content by you
and the posting of such content by Barnes & Noble.com does not and will not
violate the rights of any third party. Please follow the rules below to help
ensure that your review can be posted.
Reviews by Our Customers Under the Age of 13
We highly value and respect everyone's opinion concerning the titles we offer.
However, we cannot allow persons under the age of 13 to have accounts at BN.com or
to post customer reviews. Please see our Terms of Use for more details.
What to exclude from your review:
Please do not write about reviews, commentary, or information posted on the product page. If you see any errors in the
information on the product page, please send us an email.
Reviews should not contain any of the following:
- HTML tags, profanity, obscenities, vulgarities, or comments that defame anyone
- Time-sensitive information such as tour dates, signings, lectures, etc.
- Single-word reviews. Other people will read your review to discover why you liked or didn't like the title. Be descriptive.
- Comments focusing on the author or that may ruin the ending for others
- Phone numbers, addresses, URLs
- Pricing and availability information or alternative ordering information
- Advertisements or commercial solicitation
Reminder:
- By submitting a review, you grant to Barnes & Noble.com and its
sublicensees the royalty-free, perpetual, irrevocable right and license to use the
review in accordance with the Barnes & Noble.com Terms of Use.
- Barnes & Noble.com reserves the right not to post any review -- particularly
those that do not follow the terms and conditions of these Rules. Barnes & Noble.com
also reserves the right to remove any review at any time without notice.
- See Terms of Use for other conditions and disclaimers.
Search for Products You'd Like to Recommend
Create a Pen Name
Welcome, penname
You have successfully created your Pen Name. Start enjoying the benefits of the BN.com Community today.
If you find inappropriate content, please report it to Barnes & Noble
More About This Textbook
Overview
This original work offers the most comprehensive and up-to-date treatment of the important subject of optimal linear estimation, which is encountered in many areas of engineering such as communications, control, and signal processing, and also several other fields, e.g., econometrics and statistics. The book not only highlights the most significant contributions to this field during the 20th century, including the works of Weiner and Kalman, but it does so in an original and novel manner that paves the way for further developments in the new millennium. This book contains a large collection of problems that complement the text and are an important part of it, in addition to numerous sections that offer interesting historical accounts and insights.
Editorial Reviews
Booknews
Largely focuses on estimation problems for finite-dimensional linear systems with state-space models, covering most aspects of an area now generally known as Wiener and Kalman filtering theory. Distinctive features the treatment are the pervasive use of a geometric point of view; the emphasis on the numerically favored square root/array forms of many algorithms; and the emphasis on equivalence and duality concepts for the solution of several related problems in adaptive filtering, estimation, and control. The authors argue that these features are not as abstract and complicated as other treatments of the topic seem to fear. Annotation c. Book News, Inc., Portland, OR booknews.comProduct Details
Related Subjects
Read an Excerpt
Preface
The problem of estimating the values of a random (or stochastic) process given observations of a related random process is encountered in many areas of science and engineering, e.g., communications, control, signal processing, geophysics, econometrics, and statistics. Although the topic has a rich history, and its formative stages can be attributed to illustrious investigators such as Laplace, Gauss, Legendre, and others, the current high interest in such problems began with the work of H. Wold, A. N. Kolmogorov, and N. Wiener in the late 1930s and early 1940s. N. Wiener in particular stressed the importance of modeling not just "noise" but also "signals" as random processes. His thought-provoking originally classified 1942 report, released for open publication in 1949 and now available in paperback form under the title Time Series Analysis, is still very worthwhile background reading.
As with all deep subjects, the extensions of these results have been very far-reaching as well. A particularly important development arose from the incorporation into the theory of multichannel state-space models. Though there were various earlier partial intimations and explorations, especially in the work of R. L. Stratonovich- in the former Soviet Union, the chief credit for the explosion of activity in this direction goes to R. E. Kalman, who also made important related contributions to linear systems, optimal control, passive systems, stability theory, and network synthesis.
In fact, least-squares estimation is one of those happy subjects that is interesting not only in the richness and scope of its results, but also because of its mutually beneficialconnections with a host of other (often apparently very different) subjects. Thus, beyond those already named, we may mention connections with radiative transfer and scattering theory, linear algebra, matrix and operator theory, orthogonal polynomials, moment problems, inverse scattering problems, interpolation theory, decoding of Reed-Solomon and BCH codes, polynomial factorization and root distribution problems, digital filtering, spectral analysis, signal detection, martingale theory, the so-called Hh theories of estimation and control, least-squares and adaptive filtering problems, and many others. We can surely apply to it the lines written by William Shakespeare about another (beautiful) subject:
Her infinite variety."
Though we were originally tempted to cover a wider range, many reasons have led us to focus this volume largely on estimation problems for finite-dimensional linear systems with state-space models, covering most aspects of an area now generally known as Wiener and Kalman filtering theory. Three distinctive features of our treatment are the pervasive use of a geometric point of view, the emphasis on the numerically favored square-root/array forms of many algorithms, and the emphasis on equivalence and duality concepts for the solution of several related problems in adaptive filtering, estimation, and control. These features are generally absent in most prior treatments, ostensibly on the grounds that they are too abstract and complicated. It is our hope that these misconceptions will be dispelled by the presentation herein, and that the fundamental simplicity and power of these ideas will be more widely recognized arid exploited.
The material presented in this book can be broadly categorized into the following topics:
Chapter 1: Overview
Chapter 2: Deterministic Least-Squares Problems
Chapter 3: Stochastic Least-Squares Problems
Chapter 4: The Innovations Process
Chapter 5: State-Space Models
Chapter 6: Innovations for Stationary Processes
Chapter 7: Wiener Theory for Scalar Processes
Chapter 8: Recursive Wiener Filters
Chapter 9: The Kalman Filter
Chapter 10: Smoothed Estimators
Chapter 11: Fast Algorithms
Chapter 12: Array Algorithms
Chapter 13: Fast Array Algorithms
Chapter 16: Continuous-Time State-Space Estimation
Chapter 14: Asymptotic Behavior
Chapter 15: Duality and Equivalence in Estimation and Control
Chapter 17: A Scattering Theory Approach
Being intended for a graduate-level course, the book assumes familiarity with basic concepts from matrix theory, linear algebra, linear system theory, and random processes. Four appendices at the end of the book provide the reader with background material in all these areas.
There is ample material in this book for the instructor to fashion a course to his or her needs and tastes. The authors have used portions of this book as the basis for one-quarter first-year graduate level courses at Stanford University, the University of California at Los Angeles, and the University of California at Santa Barbara; the students were expected to have had some exposure to discrete-time and state-space theory. A typical course would start with Secs.1.1-1.2 as an overview (perhaps omitting the matrix derivations), with the rest of Ch. 1 left for a quick reading (and re-reading from time to time), most of Chs. 2 and 3 (focusing on the geometric approach) on the basic deterministic and stochastic least-squares problems, Ch. 4 on the innovations process, Secs. 6.4-6.5 and 7.3-7.7 on scalar Wiener filtering, Secs. 9.1-9.3, 9.5, and 9.7 on Kalman filtering, Secs. 10.1-10.2 as an introduction to smoothing, Secs. 12.1-12.5 and 13.1-13.4 on array algorithms, and Secs. 16.1-16.4 and 16.6 on continuous-time problems.
More advanced students and researchers would pursue selections of material from Sec. 2.8, Chs. 8, 11, 14, 15, and 17, and Apps. E and R These cover, among other topics, least-squares problems with uncertain data, the problem of canonical spectral factorization, convergence of the Kalman filter, the algebraic Riccati equation, duality, backwards-time and complementary models, scattering, etc. Those wishing to go on to the more recent H
¥ theory can find a treatment closely related to the philosophy of the current book (cf. Sec. 1.6) in the research monograph of Hassibi, Sayed, and Kailath (1999).
A feature of the book is a collection of nearly 300 problems, several of which complement the text and present additional results and insights. However, there is little discussion of real applications or of the error and sensitivity analyses required for them. The main issue in applications is constructing an appropriate model, or actually a set of models, which are further analyzed and then refined by using the results and algorithms presented in this book. Developing good models and analyzing them effectively requires not only a good appreciation of the actual application, but also a good understanding of the theory, at both an analytical and intuitive level. It is the latter that we have tried to achieve here; examples of successful applications have to be sought in the literature, and some references are provided to this end.
Acknowledgments
The development of this textbook has spanned many years. So the material, as well as its presentation, has benefited greatly from the inputs of the many bright students who have worked with us on these topics: J. Omura, P Frost, T Duncan, R. Geesey, D. Duttweiler, H. Aasnaes, M. Gevers, H. Weinert, A. Segall, M. Mort B. Dickinson, G. Sidhu, B. Friedlander, A. Vieira, S. Y Kung, B. Levy, G. Verghese, D. Lee, J. Delosme, B. Porat, H. Lev-Ari, J. Cioffi, A. Bruckstein, T. Citron, Y Bresler, R. Roy, J. Chun, D. Slock, D. Pal, G. Xu, R. Ackner, Y Cho, P Park, T. Boros, A. Erdogan, U. Forsell, B. Halder, H. Hindi, V Nascimento, T. Pare, R. Merched, and our young friend Amir Ghazanfarian (in memoriam) from whom we had so much more to learn.
We are of course also deeply indebted to the many researchers and authors in this beautiful field. Partial acknowledgment is evident through the citations and references; while the list of the latter is quite long, we apologize for omissions and inadequacies arising from the limitations of our knowledge and our energy. Nevertheless, we would be remiss not to explicitly mention the inspiration and pleasure we have gained in studying the papers and books of N. Wiener, R. E. Kalman, and P Whittle.
Major support for the many years of research that led to this book was provided by the Mathematics Divisions of the Air Force Office of Scientific Research and the Army Research Office, by the Joint Services Electronics Program, by the Defense Advanced Research Projects Agency, and by the National Science Foundation. Finally, we would like to thank Bernard Goodwin and Tom Robbins, as well as the staff of Prentice Hall, for their patience and other contributions to this project.
T Kailath
Stanford, CA
A. H. Sayed
Westwood, CA
B. Hassibi
Murray Hill, NJ
Table of Contents
Preface
Preface
The problem of estimating the values of a random (or stochastic) process given observations of a related random process is encountered in many areas of science and engineering, e.g., communications, control, signal processing, geophysics, econometrics, and statistics. Although the topic has a rich history, and its formative stages can be attributed to illustrious investigators such as Laplace, Gauss, Legendre, and others, the current high interest in such problems began with the work of H. Wold, A. N. Kolmogorov, and N. Wiener in the late 1930s and early 1940s. N. Wiener in particular stressed the importance of modeling not just "noise" but also "signals" as random processes. His thought-provoking originally classified 1942 report, released for open publication in 1949 and now available in paperback form under the title Time Series Analysis, is still very worthwhile background reading.
As with all deep subjects, the extensions of these results have been very far-reaching as well. A particularly important development arose from the incorporation into the theory of multichannel state-space models. Though there were various earlier partial intimations and explorations, especially in the work of R. L. Stratonovich- in the former Soviet Union, the chief credit for the explosion of activity in this direction goes to R. E. Kalman, who also made important related contributions to linear systems, optimal control, passive systems, stability theory, and network synthesis.
In fact, least-squares estimation is one of those happy subjects that is interesting not only in the richness and scope of its results, but also because of its mutually beneficial connections with a host of other (often apparently very different) subjects. Thus, beyond those already named, we may mention connections with radiative transfer and scattering theory, linear algebra, matrix and operator theory, orthogonal polynomials, moment problems, inverse scattering problems, interpolation theory, decoding of Reed-Solomon and BCH codes, polynomial factorization and root distribution problems, digital filtering, spectral analysis, signal detection, martingale theory, the so-called Hh theories of estimation and control, least-squares and adaptive filtering problems, and many others. We can surely apply to it the lines written by William Shakespeare about another (beautiful) subject:
Her infinite variety."
Though we were originally tempted to cover a wider range, many reasons have led us to focus this volume largely on estimation problems for finite-dimensional linear systems with state-space models, covering most aspects of an area now generally known as Wiener and Kalman filtering theory. Three distinctive features of our treatment are the pervasive use of a geometric point of view, the emphasis on the numerically favored square-root/array forms of many algorithms, and the emphasis on equivalence and duality concepts for the solution of several related problems in adaptive filtering, estimation, and control. These features are generally absent in most prior treatments, ostensibly on the grounds that they are too abstract and complicated. It is our hope that these misconceptions will be dispelled by the presentation herein, and that the fundamental simplicity and power of these ideas will be more widely recognized arid exploited.
The material presented in this book can be broadly categorized into the following topics:
Chapter 1: Overview
Chapter 2: Deterministic Least-Squares Problems
Chapter 3: Stochastic Least-Squares Problems
Chapter 4: The Innovations Process
Chapter 5: State-Space Models
Chapter 6: Innovations for Stationary Processes
Chapter 7: Wiener Theory for Scalar Processes
Chapter 8: Recursive Wiener Filters
Chapter 9: The Kalman Filter
Chapter 10: Smoothed Estimators
Chapter 11: Fast Algorithms
Chapter 12: Array Algorithms
Chapter 13: Fast Array Algorithms
Chapter 16: Continuous-Time State-Space Estimation
Chapter 14: Asymptotic Behavior
Chapter 15: Duality and Equivalence in Estimation and Control
Chapter 17: A Scattering Theory Approach
Being intended for a graduate-level course, the book assumes familiarity with basic concepts from matrix theory, linear algebra, linear system theory, and random processes. Four appendices at the end of the book provide the reader with background material in all these areas.
There is ample material in this book for the instructor to fashion a course to his or her needs and tastes. The authors have used portions of this book as the basis for one-quarter first-year graduate level courses at Stanford University, the University of California at Los Angeles, and the University of California at Santa Barbara; the students were expected to have had some exposure to discrete-time and state-space theory. A typical course would start with Secs.1.1-1.2 as an overview (perhaps omitting the matrix derivations), with the rest of Ch. 1 left for a quick reading (and re-reading from time to time), most of Chs. 2 and 3 (focusing on the geometric approach) on the basic deterministic and stochastic least-squares problems, Ch. 4 on the innovations process, Secs. 6.4-6.5 and 7.3-7.7 on scalar Wiener filtering, Secs. 9.1-9.3, 9.5, and 9.7 on Kalman filtering, Secs. 10.1-10.2 as an introduction to smoothing, Secs. 12.1-12.5 and 13.1-13.4 on array algorithms, and Secs. 16.1-16.4 and 16.6 on continuous-time problems.
More advanced students and researchers would pursue selections of material from Sec. 2.8, Chs. 8, 11, 14, 15, and 17, and Apps. E and R These cover, among other topics, least-squares problems with uncertain data, the problem of canonical spectral factorization, convergence of the Kalman filter, the algebraic Riccati equation, duality, backwards-time and complementary models, scattering, etc. Those wishing to go on to the more recent H ¥ theory can find a treatment closely related to the philosophy of the current book (cf. Sec. 1.6) in the research monograph of Hassibi, Sayed, and Kailath (1999).
A feature of the book is a collection of nearly 300 problems, several of which complement the text and present additional results and insights. However, there is little discussion of real applications or of the error and sensitivity analyses required for them. The main issue in applications is constructing an appropriate model, or actually a set of models, which are further analyzed and then refined by using the results and algorithms presented in this book. Developing good models and analyzing them effectively requires not only a good appreciation of the actual application, but also a good understanding of the theory, at both an analytical and intuitive level. It is the latter that we have tried to achieve here; examples of successful applications have to be sought in the literature, and some references are provided to this end.
Acknowledgments
The development of this textbook has spanned many years. So the material, as well as its presentation, has benefited greatly from the inputs of the many bright students who have worked with us on these topics: J. Omura, P Frost, T Duncan, R. Geesey, D. Duttweiler, H. Aasnaes, M. Gevers, H. Weinert, A. Segall, M. Mort B. Dickinson, G. Sidhu, B. Friedlander, A. Vieira, S. Y Kung, B. Levy, G. Verghese, D. Lee, J. Delosme, B. Porat, H. Lev-Ari, J. Cioffi, A. Bruckstein, T. Citron, Y Bresler, R. Roy, J. Chun, D. Slock, D. Pal, G. Xu, R. Ackner, Y Cho, P Park, T. Boros, A. Erdogan, U. Forsell, B. Halder, H. Hindi, V Nascimento, T. Pare, R. Merched, and our young friend Amir Ghazanfarian (in memoriam) from whom we had so much more to learn.
We are of course also deeply indebted to the many researchers and authors in this beautiful field. Partial acknowledgment is evident through the citations and references; while the list of the latter is quite long, we apologize for omissions and inadequacies arising from the limitations of our knowledge and our energy. Nevertheless, we would be remiss not to explicitly mention the inspiration and pleasure we have gained in studying the papers and books of N. Wiener, R. E. Kalman, and P Whittle.
Major support for the many years of research that led to this book was provided by the Mathematics Divisions of the Air Force Office of Scientific Research and the Army Research Office, by the Joint Services Electronics Program, by the Defense Advanced Research Projects Agency, and by the National Science Foundation. Finally, we would like to thank Bernard Goodwin and Tom Robbins, as well as the staff of Prentice Hall, for their patience and other contributions to this project.
T Kailath
Stanford, CA
A. H. Sayed
Westwood, CA
B. Hassibi
Murray Hill, NJ