Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems

by Harold Kushner

Overview

The book deals with several closely related topics concerning approximations and perturbations of random processes and their applications to some important and fascinating classes of problems in the analysis and design of shastic control systems and nonlinear filters. The basic mathematical methods which are used and developed are those of the theory of weak con­ vergence. The techniques are quite powerful for getting weak convergence or functional limit theorems for broad classes of problems and many of the techniques are new. The original need for some of the techniques which are developed here arose in connection with our study of the particular applications in this book, and related problems of approximation in control theory, but it will be clear that they have numerous applications elsewhere in weak convergence and process approximation theory. The book is a continuation of the author's long term interest in problems of the approximation of shastic processes and its applications to problems arising in control and communication theory and related areas. In fact, the techniques used here can be fruitfully applied to many other areas. The basic random processes of interest can be described by solutions to either (multiple time scale) Ito differential equations driven by wide band or state dependent wide band noise or which are singularly perturbed. They might be controlled or not, and their state values might be fully observable or not (e. g. , as in the nonlinear filtering problem).

Product Details

ISBN-13:	9781461288381
Publisher:	Birkhäuser Boston
Publication date:	10/04/2011
Series:	Systems & Control: Foundations & Applications
Edition description:	Softcover reprint of the original 1st ed. 1990
Pages:	234
Product dimensions:	5.98(w) x 9.02(h) x 0.02(d)

Table of Contents

1 Weak Convergence.- 0. Outline of the Chapter.- 1. Basic Properties and Definitions.- 2. Examples.- 3. The Skorohod Representation.- 4. The Function Space Ck [0, T].- 5. The Function Space Dk [0, T].- 6. Measure Valued Random Variables and Processes.- 2 Shastic Processes: Background.- 0. Outline of the Chapter.- 1. Martingales.- 2. Shastic Integrals and Itô’s Lemma.- 3. Shastic Differential Equations: Bounds.- 4. Controlled Shastic Differential Equations: Existence of Solutions.- 5. Representing a Martingale as a Shastic Integral.- 6. The Martingale Problem.- 7. Jump-Diffusion Processes.- 8. Jump-Diffusion Processes: The Martingale Problem Formulation.- 3 Controlled Shastic Differential Equations.- 0. Outline of the Chapter.- 1. Controlled S.D.E.’s: Introduction.- 2. Relaxed Controls: Deterministic Case.- 3. Shastic Relaxed Controls.- 4. The Martingale Problem Revisited.- 5. Approximations, Weak Convergence and Optimality.- 4 Controlled Singularly Perturbed Systems.- 0. Outline of the Chapter.- 1. Problem Formulation: Finite Time Interval.- 2. Approximation of the Optimal Controls and Value Functions.- 3. Discounted Cost and Optimal Stopping Problems.- 4. Average Cost Per Unit Time.- 5. Jump-Diffusion Processes.- 6. Other Approaches.- 5 Functional Occupation Measures and Average Cost Per Unit Time Problems.- 0. Outline of the Chapter.- 1. Measure Valued Random Variables.- 2. Limits of Functional Occupation Measures for Diffusions.- 3. The Control Problem.- 4. Singularly Perturbed Control Problems.- 5. Control of the Fast System.- 6. Reflected Diffusions.- 7. Discounted Cost Problem.- 6 The Nonlinear Filtering Problem.- 0. Outline of the Chapter.- 1. A Representation of the Nonlinear Filter.- 2. The Filtering Problem for the SingularlyPerturbed System.- 3. The Almost Optimality of the Averaged Filter.- 4. A Counterexample to the Averaged Filter.- 5. The Near Optimality of the Averaged Filter.- 6. A Repair and Maintainance Example.- 7. Robustness of the Averaged Filters.- 8. A Robust Computational Approximation to the Averaged Filter.- 9. The Averaged Filter on the Infinite Time Interval.- 7 Weak Convergence: The Perturbed Test Function Method.- 0. Outline of the Chapter.- 1. An Example.- 2. The Perturbed Test Function Method: Introduction.- 3. The Perturbed Test Function Method: Tightness and Weak Convergence.- 4. Characterization of the Limits.- 8 Singularly Perturbed Wide-Band Noise Driven Systems.- 0. Outline of the Chapter.- 1. The System and Noise Model.- 2. Weak Convergence of the Fast System.- 3. Convergence to the Averaged System.- 4. The Optimality Theorem.- 5. The Average Cost Per Unit Time Problem.- 9 Stability Theory.- 0. Outline of the Chapter.- 1. Stability Theory for Jump-Diffusion Processes of Itô Type.- 2. Singularly Perturbed Deterministic Systems: Bounds on Paths.- 3. Singularly Perturbed Itô Processes: Tightness.- 4. The Linear Case.- 5. Wide Bandwidth Noise.- 6. Singularly Perturbed Wide Bandwidth Noise Driven Systems.- 10 Parametric Singularities.- 0. Outline of the Chapter.- 1. Singularly Perturbed Itô Processes: Weak Convergence.- 2. Stability.- References.- List of Symbols.