Algorithms for Decision Making

Algorithms for Decision Making

Algorithms for Decision Making

Algorithms for Decision Making


    Qualifies for Free Shipping
    Choose Expedited Shipping at checkout for delivery by Monday, December 4
    Check Availability at Nearby Stores

Related collections and offers


A broad introduction to algorithms for decision making under uncertainty, introducing the underlying mathematical problem formulations and the algorithms for solving them.

Automated decision-making systems or decision-support systems—used in applications that range from aircraft collision avoidance to breast cancer screening—must be designed to account for various sources of uncertainty while carefully balancing multiple objectives. This textbook provides a broad introduction to algorithms for decision making under uncertainty, covering the underlying mathematical problem formulations and the algorithms for solving them.

The book first addresses the problem of reasoning about uncertainty and objectives in simple decisions at a single point in time, and then turns to sequential decision problems in stochastic environments where the outcomes of our actions are uncertain. It goes on to address model uncertainty, when we do not start with a known model and must learn how to act through interaction with the environment; state uncertainty, in which we do not know the current state of the environment due to imperfect perceptual information; and decision contexts involving multiple agents. The book focuses primarily on planning and reinforcement learning, although some of the techniques presented draw on elements of supervised learning and optimization. Algorithms are implemented in the Julia programming language. Figures, examples, and exercises convey the intuition behind the various approaches presented.

Product Details

ISBN-13: 9780262047012
Publisher: MIT Press
Publication date: 08/16/2022
Pages: 700
Product dimensions: 8.25(w) x 9.25(h) x 1.49(d)

About the Author

Mykel Kochenderfer is Associate Professor at Stanford University, where he is Director of the Stanford Intelligent Systems Laboratory (SISL). He is the author of Decision Making Under Uncertainty (MIT Press). Tim Wheeler is a software engineer in the Bay Area, working on autonomy, controls, and decision-making systems. Kochenderfer and Wheeler are coauthors of Algorithms for Optimization (MIT Press). Kyle Wray is a researcher who designs and implements the decision-making systems on real-world robots.

Table of Contents

Preface xix
Acknowledgments xxi
1 Introduction 1
Part I Probabilistic Reasoning
2 Representation 19
3 Inference 43
4 Parameter Learning 71
5 Structure Learning 97
6 Simple Decisions 111
Part II Sequential Problems
7 Exact Solution Methods 133
8 Approximate Value Functions 161
9 Online Planning 181
10 Policy Search 213
11 Policy Gradient Estimation 231
12 Policy Gradient Optimization 249
13 Actor-Critic Methods 267
14 Policy Validation 281
Part III Model Uncertainty
15 Exploration and Exploitation 299
16 Model-Based Methods 317
17 Model-Free Methods 335
18 Imitation Learning 335
Part IV State Uncertainty 
19 Beliefs 379
20 Exact Belief State Planning 407
21 Offline Belief State Planning 427
22 Online Belief State Planning 453
23 Controller Abstractions 471
Part V Multiagent Systems
24 Multiagent Reasoning 493
25 Sequential Problems 517
26 State Uncertainty 533
27 Collaborative Agents 545
A Mathematical Concepts 561
B Probability Distributions 573
C Computational Complexity 575
D Neural Representations 581
E Search Algorithms 599
F Problems 609
G Julia 627
References 651
Index 671

What People are Saying About This

From the Publisher

“Its remarkable clarity, range, and depth make this a magnificent book both to learn from and to teach. It opens the door to so many modern techniques while firmly grounding them in the statistical and mathematical theory given us by the founders. t is a wonderful book—truly exceptional.”
—Thomas J. Sargent, Department of Economics, New York University, Senior Fellow, Hoover Institution, Stanford University
“I love the topics covered—a great mix of classical approaches and more recent trends. It'll be my main textbook for teaching reinforcement learning.”
—Michael L. Littman, Professor of Computer Science, Brown University

From the B&N Reads Blog

Customer Reviews