Designing Distributed Systems: Patterns and Paradigms for Scalable, Reliable Services

by Brendan Burns

Overview

Without established design patterns to guide them, developers have had to build distributed systems from scratch, and most of these systems are very unique indeed. Today, the increasing use of containers has paved the way for core distributed system patterns and reusable containerized components. This practical guide presents a collection of repeatable, generic patterns to help make the development of reliable distributed systems far more approachable and efficient.

Author Brendan Burns—Director of Engineering at Microsoft Azure—demonstrates how you can adapt existing software design patterns for designing and building reliable distributed applications. Systems engineers and application developers will learn how these long-established patterns provide a common language and framework for dramatically increasing the quality of your system.

• Understand how patterns and reusable components enable the rapid development of reliable distributed systems
• Use the side-car, adapter, and ambassador patterns to split your application into a group of containers on a single machine
• Explore loosely coupled multi-node distributed patterns for replication, scaling, and communication between the components
• Learn distributed system patterns for large-scale batch data processing covering work-queues, event-based processing, and coordinated workflows

Product Details

ISBN-13:	9781491983645
Publisher:	O'Reilly Media, Incorporated
Publication date:	03/11/2018
Pages:	162
Product dimensions:	7.00(w) x 9.00(h) x 0.40(d)

About the Author

Brendan Burns is a Partner Architect in Microsoft Azure, working on Azure Container Service and Azure Resource Manager. Prior to Microsoft he worked at Google in the Google Cloud Platform, where he co-founded the Kubernetes project and helped build APIs like Deployment Manager and Cloud DNS. Before working on cloud, he worked on Google's web-search infrastructure, with a focus on low-latency indexing. He has a PhD in Computer Science from the University of Massachusetts Amherst with a specialty in Robotics. He lives in Seattle with his wife Robin Sanders, their two children and a cat Mrs. Paws, who rules over their household with an iron paw.

Table of Contents

Preface vii

1 Introduction 1

A Brief History of Systems Develop merit 1

A Brief History of Patterns in Software Development 2

Formalization of Algorithmic Programming 3

Patterns for Object-Oriented Programming 3

The Rise of Open Source Software 3

The Value of Patterns, Practices, and Components 4

Standing on the Shoulders of Giants 4

A Shared Language for Discussing Our Practice 5

Shared Components for Easy Reuse 5

Summary 6

Part I Single-Node Patterns

Motivations 7

Summary 8

2 The Sidecar Pattern 11

An Example Sidecar: Adding HTTPS to a Legacy Service 11

Dynamic Configuration with Sidecars 12

Modular Application Containers 14

Hands On: Deploying the topz Container 14

Building a Simple PaaS with Sidecars 15

Designing Sidecars for Modularity and Reusability 16

Parameterized Containers 17

Define Each Container's API 17

Documenting Your Containers 18

Summary 19

3 Ambassadors 21

Using an Ambassador to Shard a Service 22

Hands On: Implementing a Sharded Redis 23

Using an Ambassador for Service Brokering 25

Using an Ambassador to Do Experimentation or Request Splitting 26

Hands On: Implementing 10% Experiments 27

4 Adapters 31

Monitoring 32

Hands On: Using Prometheus for Monitoring 33

Logging 34

Hands On: Normalizing Different Logging Formats with Fluentd 35

Adding a Health Monitor 36

Hands On: Adding Rich Health Monitoring for MySQL 37

Part II Serving Patterns

Introduction to Microservices 41

5 Replicated Load-Balanced Services 45

Stateless Services 45

Readiness Probes for Load Balancing 46

Hands On: Creating a Replicated Service in Kubernetes 47

Session Tracked Services 48

Application-Layer Replicated Services 49

Introducing a Caching Layer 49

Deploying Your Cache 50

Hands On: Deploying the Caching Layer 51

Expanding the Caching Layer 53

Rate Limiting and Denial-of-Service Defense 54

SSL Termination 54

Hands On: Deploying nginx and SSL Termination 55

Summary 57

6 Sharded Services 59

Sharded Caching 59

Why You Might Need a Sharded Cache 60

The Role of the Cache in System Performance 61

Replicated, Sharded Caches 62

Hands On: Deploying an Ambassador and Memcache for a Sharded Cache 63

An Examination of Sharding Functions 66

Selecting a Key 67

Consistent Hashing Functions 68

Hands On: Building a Consistent HTTP Sharding Proxy 69

Sharded, Replicated Serving 70

Hot Sharding Systems 70

7 Scatter/Gather 73

Scatter/Gather with Root Distribution 74

Hands On: Distributed Document Search 75

Scatter/Gather with Leaf Sharding 76

Hands On: Sharded Document Search 77

Choosing the Right Number of Leaves 78

Scaling Scatter/Gather for Reliability and Scale 79

8 Functions and Event-Driven Processing 81

Determining When FaaS Makes Sense 82

The Benefits of FaaS 82

The Challenges of FaaS 82

The Need for Background Processing 83

The Need to Hold Data in Memory 83

The Costs of Sustained Request-Based Processing 84

Patterns for FaaS 84

The Decorator Pattern: Request or Response Transformation 85

Hands On: Adding Request Defaulting Prior to Request Processing 86

Handling Events 87

Hands On: Implementing Two-Factor Authentication 88

Event-Based Pipelines 89

Hands On: Implementing a Pipeline for New-User Signup 89

9 Ownership Election 93

Determining If You Even Need Master Election 94

The Basics of Master Election 95

Hands On: Deploying etcd 97

Implementing Locks 98

Hands On: Implementing Locks in etcd 101

Implementing Ownership 101

Hands On: Implementing Leases in etcd 102

Handling Concurrent Data Manipulation 103

Part III Batch Computational Patterns

10 Work Queue Systems 109

A Generic Work Queue System 109

The Source Container Interface 110

The Worker Container Interface 112

The Shared Work Queue Infrastructure 113

Hands On: Implementing a Video Thumbnailer 115

Dynamic Scaling of the Workers 117

The Multi-Worker Pattern 118

11 Event-Driven Batch Processing 121

Patterns of Event-Driven Processing 122

Copier 122

Filter 123

Splitter 124

Sharder 125

Merger 127

Hands On: Building an Event-Driven Flow for New User Sign-Up 128

Publisher/Subscriber Infrastructure 129

Hands On: Deploying Kafka 130

12 Coordinated Batch Processing 133

Join (or Barrier Synchronization) 134

Reduce 135

Hands On: Count 136

Sum 137

Histogram 137

Hands On: An Image Tagging and Processing Pipeline 138

13 Conclusion: A New Beginning? 143

Index 145