Discrete-Time Markov Chains: Two-Time-Scale Methods and Applications

This book focuses on two-time-scale Markov chains in discrete time. Our motivation stems from existing and emerging applications in optimization and control of complex systems in manufacturing, wireless communication, and financial engineering. Much of our effort in this book is devoted to designing system models arising from various applications, analyzing them via analytic and probabilistic techniques, and developing feasible compu- tionalschemes. Our main concern is to reduce the inheren...