Stochastic Controls: Hamiltonian Systems and HJB Equations / Edition 1

by Jiongmin Yong, Xun Yu Zhou

Overview

As is well known, Pontryagin's maximum principle and Bellman's dynamic programming are the two principal and most commonly used approaches in solving shastic optimal control problems. * An interesting phenomenon one can observe from the literature is that these two approaches have been developed separately and independently. Since both methods are used to investigate the same problems, a natural question one will ask is the fol­ lowing: (Q) What is the relationship betwccn the maximum principlc and dy­ namic programming in shastic optimal controls? There did exist some researches (prior to the 1980s) on the relationship between these two. Nevertheless, the results usually werestated in heuristic terms and proved under rather restrictive assumptions, which were not satisfied in most cases. In the statement of a Pontryagin-type maximum principle there is an adjoint equation, which is an ordinary differential equation (ODE) in the (finite-dimensional) deterministic case and a shastic differential equation (SDE) in the shastic case. The system consisting of the adjoint equation, the original state equation, and the maximum condition is referred to as an (extended) Hamiltonian system. On the other hand, in Bellman's dynamic programming, there is a partial differential equation (PDE), of first order in the (finite-dimensional) deterministic case and of second or­ der in the shastic case. This is known as a Hamilton-Jacobi-Bellman (HJB) equation.

Product Details

ISBN-13:	9780387987231
Publisher:	Springer New York
Publication date:	06/22/1999
Series:	Stochastic Modelling and Applied Probability , #43
Edition description:	1999
Pages:	439
Product dimensions:	6.10(w) x 9.25(h) x 0.04(d)

Table of Contents

1. Basic Shastic Calculus.- 1. Probability.- 2. Shastic Processes.- 3. Stopping Times.- 4. Martingales.- 5. Itô’s Integral.- 6. Shastic Differential Equations.- 2. Shastic Optimal Control Problems.- 1. Introduction.- 2. Deterministic Cases Revisited.- 3. Examples of Shastic Control Problems.- 4. Formulations of Shastic Optimal Control Problems.- 5. Existence of Optimal Controls.- 6. Reachable Sets of Shastic Control Systems.- 7. Other Shastic Control Models.- 8. Historical Remarks.- 3. Maximum Principle and Shastic Hamiltonian Systems.- 1. Introduction.- 2. The Deterministic Case Revisited.- 3. Statement of the Shastic Maximum Principle.- 4. A Proof of the Maximum Principle.- 5. Sufficient Conditions of Optimality.- 6. Problems with State Constraints.- 7. Historical Remarks.- 4. Dynamic Programming and HJB Equations.- 1. Introduction.- 2. The Deterministic Case Revisited.- 3. The Shastic Principle of Optimality and the HJB Equation.- 4. Other Propertiesof the Value Function.- 5. Viscosity Solutions.- 6. Uniqueness of Viscosity Solutions.- 7. Historical Remarks.- 5. The Relationship Between the Maximum Principle and Dynamic Programming.- 1. Introduction.- 2. Classical Hamilton-Jacobi Theory.- 3. Relationship for Deterministic Systems.- 4. Relationship for Shastic Systems.- 5. Shastic Verification Theorems.- 6. Optimal Feedback Controls.- 7. Historical Remarks.- 6. Linear Quadratic Optimal Control Problems.- 1. Introduction.- 2. The Deterministic LQ Problems Revisited.- 3. Formulation of Shastic LQ Problems.- 4. Finiteness and Solvability.- 5. A Necessary Condition and a Hamiltonian System.- 6. Shastic Riccati Equations.- 7. Global Solvability of Shastic Riccati Equations.- 8. A Mean-variance Portfolio Selection Problem.- 9. Historical Remarks.- 7. Backward Shastic Differential Equations.- 1. Introduction.- 2. Linear Backward Shastic Differential Equations.- 3. Nonlinear Backward Shastic Differential Equations.- 4. Feynman—Kac-Type Formulae.- 5. Forward—Backward Shastic Differential Equations.- 6. Option Pricing Problems.- 7. Historical Remarks.- References.

Editorial Reviews

From the reviews:

SIAM REVIEW

"The presentation of this book is systematic and self-contained…Summing up, this book is a very good addition to the control literature, with original features not found in other reference books. Certain parts could be used as basic material for a graduate (or postgraduate) course…This book is highly recommended to anyone who wishes to study the relationship between Pontryagin’s maximum principle and Bellman’s dynamic programming principle applied to diffusion processes."

MATHEMATICS REVIEW

This is an authoratative book which should be of interest to researchers in shastic control, mathematical finance, probability theory, and applied mathematics. Material out of this book could also be used in graduate courses on shastic control and dynamic optimization in mathematics, engineering, and finance curricula. Tamer Basar, Math. Review

From the Publisher