BN.com Gift Guide

Markov Chains: Theory and Applications

Overview

Markov chains are a fundamental class of stochastic processes. They are widely used to solve problems in a large number of domains such as operational research, computer science, communication networks and manufacturing systems. The success of Markov chains is mainly due to their simplicity of use, the large number of available theoretical results and the quality of algorithms developed for the numerical evaluation of many metrics of interest.
The author presents the theory of ...

See more details below
Other sellers (Hardcover)
  • All (6) from $97.50   
  • New (5) from $97.50   
  • Used (1) from $103.36   

Overview

Markov chains are a fundamental class of stochastic processes. They are widely used to solve problems in a large number of domains such as operational research, computer science, communication networks and manufacturing systems. The success of Markov chains is mainly due to their simplicity of use, the large number of available theoretical results and the quality of algorithms developed for the numerical evaluation of many metrics of interest.
The author presents the theory of both discrete-time and continuous-time homogeneous Markov chains. He carefully examines the explosion phenomenon, the Kolmogorov equations, the convergence to equilibrium and the passage time distributions to a state and to a subset of states. These results are applied to birth-and-death processes. He then proposes a detailed study of the uniformization technique by means of Banach algebra. This technique is used for the transient analysis of several queuing systems.

Contents

1. Discrete-Time Markov Chains
2. Continuous-Time Markov Chains
3. Birth-and-Death Processes
4. Uniformization
5. Queues

About the Authors

Bruno Sericola is a Senior Research Scientist at Inria Rennes – Bretagne Atlantique in France. His main research activity is in performance evaluation of computer and communication systems, dependability analysis of fault-tolerant systems and stochastic models.

Read More Show Less

Product Details

  • ISBN-13: 9781848214934
  • Publisher: Wiley
  • Publication date: 5/28/2013
  • Series: ISTE Series
  • Edition number: 1
  • Pages: 416
  • Product dimensions: 6.10 (w) x 9.30 (h) x 1.10 (d)

Meet the Author

Bruno Sericola is a senior Research Scientist at Inria Rennes - Bretagne Atlantique in France. His main research activity is in performance evaluation of computer and communication systems, dependability analysis of fault-tolerant systems and stochastic models.

Read More Show Less

Table of Contents

Preface ix

Chapter 1. Discrete-Time Markov Chains 1

1.1. Definitions and properties 1

1.2. Strong Markov property 5

1.3. Recurrent and transient states 8

1.4. State classification 12

1.5. Visits to a state 14

1.6. State space decomposition 18

1.7. Irreducible and recurrent Markov chains 22

1.8. Aperiodic Markov chains 30

1.9. Convergence to equilibrium 34

1.10. Ergodic theorem 41

1.11. First passage times and number of visits 53

1.12. Finite Markov chains 68

1.13. Absorbing Markov chains 70

1.14. Examples 76

1.15. Bibliographical notes 87

Chapter 2. Continuous-Time Markov Chains 89

2.1. Definitions and properties 92

2.2. Transition functions and infinitesimal generator 93

2.3. Kolmogorov’s backward equation 108

2.4. Kolmogorov’s forward equation 114

2.5. Existence and uniqueness of the solutions 127

2.6. Recurrent and transient states 130

2.7. State classification 137

2.8. Explosion 141

2.9. Irreducible and recurrent Markov chains 148

2.10. Convergence to equilibrium 162

2.11. Ergodic theorem 166

2.12. First passage times 172

2.13. Absorbing Markov chains 184

2.14. Bibliographical notes 190

Chapter 3. Birth-and-Death Processes 191

3.1. Discrete-time birth-and-death processes 191

3.2. Absorbing discrete-time birth-and-death processes 200

3.3. Periodic discrete-time birth-and-death processes 208

3.4. Continuous-time pure birth processes 209

3.5. Continuous-time birth-and-death processes 213

3.6. Absorbing continuous-time birth-and-death processes 228

3.7. Bibliographical notes 233

Chapter 4. Uniformization 235

4.1. Introduction 235

4.2. Banach spaces and algebra 237

4.3. Infinite matrices and vectors 243

4.4. Poisson process 249

4.5. Uniformizable Markov chains 263

4.6. First passage time to a subset of states 273

4.7. Finite Markov chains 275

4.8. Transient regime 276

4.9. Bibliographical notes 286

Chapter 5. Queues 287

5.1. The M/M/1 queue 288

5.2. The M/M/c queue 315

5.3. The M/M/∞ queue 318

5.4. Phase-type distributions 323

5.5. Markovian arrival processes 326

5.6. Batch Markovian arrival process 342

5.7. Block-structured Markov chains 352

5.8. Applications 370

5.9. Bibliographical notes 380

Appendix 1 Basic Results 381

Bibliography 387

Index 395

Read More Show Less

Customer Reviews

Be the first to write a review
( 0 )
Rating Distribution

5 Star

(0)

4 Star

(0)

3 Star

(0)

2 Star

(0)

1 Star

(0)

Your Rating:

Your Name: Create a Pen Name or

Barnes & Noble.com Review Rules

Our reader reviews allow you to share your comments on titles you liked, or didn't, with others. By submitting an online review, you are representing to Barnes & Noble.com that all information contained in your review is original and accurate in all respects, and that the submission of such content by you and the posting of such content by Barnes & Noble.com does not and will not violate the rights of any third party. Please follow the rules below to help ensure that your review can be posted.

Reviews by Our Customers Under the Age of 13

We highly value and respect everyone's opinion concerning the titles we offer. However, we cannot allow persons under the age of 13 to have accounts at BN.com or to post customer reviews. Please see our Terms of Use for more details.

What to exclude from your review:

Please do not write about reviews, commentary, or information posted on the product page. If you see any errors in the information on the product page, please send us an email.

Reviews should not contain any of the following:

  • - HTML tags, profanity, obscenities, vulgarities, or comments that defame anyone
  • - Time-sensitive information such as tour dates, signings, lectures, etc.
  • - Single-word reviews. Other people will read your review to discover why you liked or didn't like the title. Be descriptive.
  • - Comments focusing on the author or that may ruin the ending for others
  • - Phone numbers, addresses, URLs
  • - Pricing and availability information or alternative ordering information
  • - Advertisements or commercial solicitation

Reminder:

  • - By submitting a review, you grant to Barnes & Noble.com and its sublicensees the royalty-free, perpetual, irrevocable right and license to use the review in accordance with the Barnes & Noble.com Terms of Use.
  • - Barnes & Noble.com reserves the right not to post any review -- particularly those that do not follow the terms and conditions of these Rules. Barnes & Noble.com also reserves the right to remove any review at any time without notice.
  • - See Terms of Use for other conditions and disclaimers.
Search for Products You'd Like to Recommend

Recommend other products that relate to your review. Just search for them below and share!

Create a Pen Name

Your Pen Name is your unique identity on BN.com. It will appear on the reviews you write and other website activities. Your Pen Name cannot be edited, changed or deleted once submitted.

 
Your Pen Name can be any combination of alphanumeric characters (plus - and _), and must be at least two characters long.

Continue Anonymously

    If you find inappropriate content, please report it to Barnes & Noble
    Why is this product inappropriate?
    Comments (optional)