SRE with Java Microservices: Patterns for Reliable Microservices in the Enterprise

by Jonathan Schneider

Overview

In a microservices architecture, the whole is indeed greater than the sum of its parts. But in practice, individual microservices can inadvertently impact others and alter the end user experience. Effective microservices architectures require standardization on an organizational level with the help of a platform engineering team.

This practical book provides a series of progressive steps that platform engineers can apply technically and organizationally to achieve highly resilient Java applications. Author Jonathan Schneider covers many effective SRE practices from companies leading the way in microservices adoption. You’ll examine several patterns discovered through much trial and error in recent years, complete with Java code examples.

Chapters are organized according to specific patterns, including:

• Application metrics: Monitoring for availability with Micrometer
• Debugging with observability: Logging and distributed tracing; failure injection testing
• Charting and alerting: Building effective charts; KPIs for Java microservices
• Safe multicloud delivery: Spinnaker, deployment strategies, and automated canary analysis
• Source code observability: Dependency management, API utilization, and end-to-end asset inventory
• Traffic management: Concurrency of systems; platform, gateway, and client-side load balancing

Product Details

ISBN-13:	9781492073925
Publisher:	O'Reilly Media, Incorporated
Publication date:	11/03/2020
Pages:	314
Product dimensions:	7.00(w) x 9.19(h) x (d)

About the Author

Jonathan Schneider is CEO and cofounder of Moderne, where he works to modernize applications and infrastructure through automated source code transformation and asset visibility. Previously, he worked for the Spring team on application monitoring and continuous delivery at Netflix on engineering tools related to distributed refactoring, dependency insight, and build tools. He also worked on refactoring at Gradle.

Jon has visited and given workshops on monitoring and delivery at a wide variety of enterprises, developing an understanding of the steps organizations need to take to get closer to Netflix-style resiliency where outages in entire regions don’t disrupt service availability and application teams know about problems before the call center.

Table of Contents

Foreword ix

Preface xiii

1 The Application Platform 1

Platform Engineering Culture 2

Monitoring 7

Monitoring for Availability 7

Monitoring as a Debugging Tool 10

Learning to Expect Failure 12

Effective Monitoring Builds Trust 13

Delivery 13

Traffic Management 15

Capabilities Not Covered 15

Testing Automation 15

Chaos Engineering and Continuous Verification 17

Configuration as Code 17

Encapsulating Capabilities 18

Service Mesh 19

Summary 21

2 Application Metrics 23

Black Box Versus White Box Monitoring 24

Dimensional Metrics 25

Hierarchical Metrics 26

Micrometer Meter Registries 27

Creating Meters 30

Naming Metrics 31

Common Tags 35

Classes of Meters 38

Gauges 39

Counters 42

Timers 45

"Count" Means "Throughput" 46

"Count" and "Sum" Together Mean "Aggregable Average" 46

Maximum Is a Decaying Signal That Isn't Aligned to the Push Interval 50

The Sum of Sum Over an Interval 53

The Base Unit of Time 53

Using Timers 55

Common Features of Latency Distributions 59

Percentiles/Quantiles 60

Histograms 65

Service Level Objective Boundaries 69

Distribution Summaries 73

Long Task Timers 74

Choosing the Right Meter Type 77

Controlling Cost 77

Coordinated Omission 80

Load Testing 82

Meter Filters 87

Deny/Accept Meters 88

Transforming Metrics 89

Configuring Distribution Statistics 91

Separating Platform and Application Metrics 92

Partitioning Metrics by Monitoring System 96

Meter Binders 98

Summary 99

3 Debugging with Observability 101

The Three Pillars of Observability…or Is It Two? 101

Logs 102

Distributed Tracing 103

Metrics 104

Which Telemetry Is Appropriate? 104

Components of a Distributed Trace 107

Types of Distributed Tracing Instrumentation 109

Manual Tracing 109

Agent Tracing 110

Framework Tracing 110

Service Mesh Tracing 111

Blended Tracing 112

Sampling 114

No Sampling 114

Rate-Limiting Samplers 114

Probabilistic Samplers 115

Boundary Sampling 116

Impact of Sampling on Anomaly Detection 116

Distributed Tracing and Monoliths 117

Correlation of Telemetry 118

Metric to Trace Correlation 119

Using Trace Context for Failure Injection and Experimentation 120

Summary 123

4 Charting and Alerting 125

Differences in Monitoring Systems 127

Effective Visualizations of Service Level Indicators 132

Styles for Line Width and Shading 132

Errors Versus Successes 134

"Top k" Visualizations 135

Prometheus Rate Interval Selection 137

Gauges 137

Counters 139

Timers 143

When to Stop Creating Dashboards 147

Service Level Indicators for Every Java Microservice 148

Errors 148

Latency 153

Garbage Collection Pause Times 161

Heap Utilization 164

CPU Utilization 170

File Descriptors 172

Suspicious Traffic 174

Batch Runs or Other Long-Running Tasks 175

Building Alerts Using Forecasting Methods 176

Naive Method 177

Single-Exponential Smoothing 179

Universal Scalability Law 181

Summary 185

5 Safe, Multicloud Continuous Delivery 187

Types of Platforms 188

Resource Types 189

Delivery Pipelines 191

Packaging for the Cloud 194

Packaging for IaaS Platforms 196

Packaging for Container Schedulers 198

The Delete + None Deployment 199

The Highlander 200

Blue/Green Deployment 200

Automated Canary Analysis 205

Spinnaker with Kayenta 209

General-Purpose Canary Metrics for Every Microservice 214

Summary 218

6 Source Code Observability 221

The Stateful Asset Inventory 223

Release Versioning 226

Maven Repositories 227

Build Tools for Release Versioning 230

Capturing Resolved Dependencies in Metadata 234

Capturing Method-Level Utilization of the Source Code 240

Structured Code Search with OpenRewrite 243

Dependency Management 252

Version Misalignments 252

Dynamic Version Constraints 253

Unused Dependencies 254

Undeclared Explicitly Used Dependencies 255

Summary 256

7 Traffic Management 257

Microservices Offer More Potential Failure Points 257

Concurrency of Systems 258

Platform Load Balancing 259

Gateway Load Balancing 259

Join the Shortest Queue 262

Instance-Reported Availability and Utilization 264

Health Checks 267

Choice of Two 269

Instance Probation 269

Knock-On Effects of Smarter Load Balancing 270

Client-Side Load Balancing 270

Hedge Requests 272

Call Resiliency Patterns 273

Retries 274

Rate Limiters 276

Bulkheads 278

Circuit Breakers 280

Adaptive Concurrency Limits 283

Choosing the Right Call Resiliency Pattern 284

Implementation in Service Mesh 285

Implementation in RSocket 287

Summary 288

Index 289