Markov Chains and Stochastic Stability

by Sean P. Meyn, Richard L. Tweedie

Overview

Markov Chains and Shastic Stability is part of the Communications and Control Engineering Series (CCES) edited by Professors B.W. Dickinson, E.D. Sontag, M. Thoma, A. Fettweis, J.L. Massey and J.W. Modestino. The area of Markov chain theory and application has matured over the past 20 years into something more accessible and complete. It is of increasing interest and importance. This publication deals with the action of Markov chains on general state spaces. It discusses the theories and the use to be gained, concentrating on the areas of engineering, operations research and control theory. Throughout, the theme of shastic stability and the search for practical methods of verifying such stability, provide a new and powerful technique. This does not only affect applications but also the development of the theory itself. The impact of the theory on specific models is discussed in detail, in order to provide examples as well as to demonstrate the importance of these models. Markov Chains and Shastic Stability can be used as a textbook on applied Markov chain theory, provided that one concentrates on the main aspects only. It is also of benefit to graduate students with a standard background in countable space shastic models. Finally, the book can serve as a research resource and active tool for practitioners.

Product Details

ISBN-13:	9781447132691
Publisher:	Springer London
Publication date:	12/12/2012
Series:	Communications and Control Engineering
Edition description:	1993
Pages:	550
Product dimensions:	5.51(w) x 8.50(h) x 0.05(d)

Table of Contents

I Communication and Regeneration.- 1 Heuristics.- 2 Markov Models.- 3 Transition Probabilities.- 4 Irreducibility.- 5 Pseudo-atoms.- 6 Topology and Continuity.- 7 The Nonlinear State Space Model.- II Stability Structures.- 8 Transience and Recurrence.- 9 Harris and Topological Recurrence.- 10 The Existence of—.- 11 Drift and Regularity.- 12 Invariance and Tightness.- III Convergence.- 13 Ergodicity.- 14 ƒ-Ergodicity and ƒ-Regularity.- 15 Geometric Ergodicity.- 16 V-Uniform Ergodicity.- 17 Sample Paths and Limit Theorems.- 18 Positivity.- 19 Generalized Classification Criteria.- IV Appendices.- A Mud Maps.- A.l Recurrence versus transience.- A.2 Positivity versus nullity.- A.3 Convergence Properties.- B Testing for Stability.- B.l A Glossary of Drift Conditions.- B.2 The scalar SETAR Model: a complete classification.- C A Glossary of Model Assumptions..- C.l Regenerative Models.- C.2 State Space Models.- D Some Mathematical Background.- D.l Some Measure Theory.- D.2 Some Probability Theory.- D.3 Some Topology.- D.4 Some Real Analysis.- D.5 Some Convergence Concepts for Measures.- D.6 Some Martingale Theory.- D.7 Some Results on Sequences and Numbers.- References.- Symbols Index.