Operations Anti-Patterns, DevOps Solutions

by Jeffery D. Smith

Overview

Operations Anti-Patterns, DevOps Solutions shows how to implement DevOps techniques in the kind of imperfect environments most developers work in. Part technology tutorial, part reference manual, and part psychology handbook, this practical guide shows you realistic ways to bring DevOps to your team when you don’t have the flexibility to make sweeping changes in organizational structure.

Summary
Operations Anti-Patterns, DevOps Solutions shows how to implement DevOps techniques in the kind of imperfect environments most developers work in. Part technology tutorial, part reference manual, and part psychology handbook, this practical guide shows you realistic ways to bring DevOps to your team when you don't have the flexibility to make sweeping changes in organizational structure.

Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.

About the technology
To some extent, all organizations—even yours—suffer from poor development practices, garbled communications, and outdated legacy systems. The good news is DevOps can help you improve your processes. First, however, you'll need to recognize the core issues holding you back. This book empowers you to deliver DevOps with limited resources while navigating the office politics and entrenched mindsets that are all too common in actual workplaces.

About the book
Operations Anti-Patterns, DevOps Solutions offers clear steps for transforming development and communication. Using jargon-free language, this book describes incremental techniques that pay off immediately. Streamline your workflow, manage unplanned time, and build operational metrics. Whatever your issues, this book holds the keys to organizational success.

What's inside

Turn failure into opportunity
Drive change through culture
Break down knowledge silos
Settle middle management turf wars

About the reader
For team leaders and managers.

About the author
Jeffery D. Smith has been in the technology industry for over 15 years. He has managed DevOps transformations at the ad-tech firm Centro and the online ordering platform Grubhub.

Table of Contents

1 The DevOps ingredients

2 The paternalist syndrome

3 Operational blindness

4 Data instead of information

5 Quality as a condiment

6 Alert fatigue

7 The empty toolbox

8 Off-hour deployments

9 Wasting a perfectly good incident

10 Information hoarding: Only Brent knows

11 Culture by decree

12 Too many yardsticks

Product Details

ISBN-13:	9781617296987
Publisher:	Manning
Publication date:	11/24/2020
Pages:	320
Product dimensions:	7.38(w) x 9.25(h) x 0.70(d)

About the Author

Jeff Smith has been in the technology industry for over 15 years, both as management and individual contributor. He has managed DevOps transformations at Centro, an ad-tech firm, and Grubhub, an online ordering platform.

Table of Contents

Preface xi

Acknowledgments xiii

About this book xv

About the author xviii

About the cover illustration xix

1 The DevOps ingredients 1

1.1 What is DevOps? 2

A little DevOps history 2

What DevOps is not 4

1.2 CAMS, the pillars of DevOps 5

1.3 Another DevOps book? 7

2 The paternalist syndrome 8

2.1 Creating barriers instead of safeguards 9

2.2 Introducing the gatekeepers 12

2.3 Examining the gatekeepers 13

2.4 Curing paternalism through automation 17

2.5 Capturing the purpose of the approval 18

2.6 Structuring code for automation 19

Approval process 20

Automating approvals 22

Logging process 25

Notification process 26

Error handling 27

2.7 Ensuring continuous improvement 28

3 Operational blindness 29

3.1 War stories 30

3.2 Changing the scope of development and operations 31

3.3 Understanding the product 32

3.4 Creating operational visibility 33

Creating custom metrics 34

Deciding what to measure 35

Defining healthy metrics 39

Failure mock and effects analysis 40

3.5 Making logging useful 44

Log aggregation 44

What should I be logging? 46

The hurdles of log aggregation 48

4 Data instead of information 52

4.1 Start with the user, not the data 53

4.2 Widgets, the dashboard building blocks 54

The line graph 54

The bar graph 56

The gauge 58

4.3 Giving context to your widgets 58

Giving context through color 58

Giving context through threshold lines 59

Giving context through time comparisons 60

4.4 Organizing your dashboard 61

Working with dashboard rows 61

Leading the reader 62

4.5 Naming your dashboards 63

5 Quality as a condiment 65

5.1 The testing pyramid 66

5.2 Testing structure 69

Unit tests 69

Integration tests 72

End-to-end tests 73

5.3 Confidence in your test suite 76

Restoring confidence in your test suite 77

Avoiding vanity metrics 80

5.4 Continuous deployment vs. continuous delivery 81

5.5 Feature flags 83

5.6 Executing pipelines 84

5.7 Managing the testing infrastructure 88

5.8 DevSecOps 89

6 Alert fatigue 92

6.1 War story 93

6.2 The purpose of on-call rotation 94

6.3 Defining on-call rotations 95

Time to acknowledge 96

Time to begin 97

Time to resolve 97

6.4 Defining alert criteria 97

Thresholds 99

Noisy alerts 101

6.5 Staffing on-call rotations 104

6.6 Compensating for being on call 106

Monetary compensation 107

Time off 108

Increased work-from-home flexibility 109

6.7 Tracking on-call happiness 109

Who is being alerted? 110

What level of urgency is the alert? 110

How is the alert being delivered? 111

When is the team member being alerted? 111

6.8 Providing other on-call tasks 112

On-call support projects 112

Performance reporting 113

7 The empty toolbox 115

7.1 Why internal tools and automation matter 117

Improvements made by automation 117

Business impact to automation 119

7.2 Why organizations don't automate more 121

Setting automation as a cultural priority 121

Staffing for automation and tooling 123

7.3 Fixing your cultural automation problems 126

No manual tasks allowed 126

Supporting "no" as an answer 126

The cost of manual work 128

7.4 Prioritizing automation 131

7.5 Defining your automation goals 132

Automation as a requirement in all your tools 132

Prioritizing automation in your work 133

Reflecting automation as a priority with your staff 135

Providing time for training and learning 135

7.6 Filling the skill-set gap 137

But if I build it, I own it 138

Building the new skill set 140

7.7 Approaching automation 141

Safety in tasks 142

Designing for safety 143

Complexity in tasks 145

How to rank tasks 147

Automating simple tasks 148

Automating complicated tasks 150

Automating complex tasks 152

8 Off-hour deployments 154

8.1 War story 154

8.2 The layers of a deployment 156

8.3 Making deployments routine affairs 159

Accurate preproduction environments 159

Staging will never be exactly like production 162

8.4 Frequency reduces fear 164

8.5 Reducing fear by reducing risk 167

8.6 Handling failure in the layers of the deployment process 168

Feature flags 168

When to toggle off your feature flag 169

Fleet rollbacks 171

Deployment artifact rollbacks 174

Database-level rollbacks 175

8.7 Creating deployment artifacts 179

Leveraging package management 179

Configuration files in packages 184

8.8 Automating your deployment pipeline 188

Safely installing the new application 188

9 Wasting a perfectly good incident 191

9.1 The components of a good postmortem 192

Creating mental models 194

Following the 24-hour rule 195

Setting the rules of the postmortem 196

9.2 The incident 197

9.3 Running the postmortem 198

Choosing whom to invite to the postmortem 198

Running through the timeline 199

Defining action items and following up 205

Documenting your postmortem 207

Sharing the postmortem 210

10 Information hoarding: Only Brent knows 212

10.1 Understanding how information hoarding happens 213

10.2 Recognizing unintentional hoarders 214

Documentation isn't valued 215

Abstraction vs. obfuscation 217

Access restrictions 219

Evaluating gatekeeper behavior 220

10.3 Structuring your communication effectively 221

Defining your topic 221

Defining your audience 222

Outlining your key points 222

Presenting a call to action 222

10.4 Making your knowledge discoverable 223

Structuring your knowledge stores 223

Creating learning rituals 229

10.5 Using chat tools effectively 234

Establishing company etiquette 234

Moving beyond just chat 236

11 Culture by decree 238

11.1 What is culture? 240

Cultural values 240

Cultural rituals 241

Underlying assumptions 242

11.2 How does culture influence behavior? 243

11.3 How do you change a culture? 244

Sharing a culture 244

An individual can change a culture 247

Examining your company's values 249

Creating rituals 251

Using rituals and language to change cultural norms 253

11.4 Talent that matches your culture 255

Old roles, new mindset 255

The obsession with senior engineers 257

Interviewing candidates 260

Evaluating candidates 265

How many candidates to interview? 266

12 Too many yardsticks 269

12.1 Tiers of goals 270

Organizational goals 271

Departmental goals 272

Team goals 272

Getting the goals 273

12.2 Consciousness around what you work on 274

Priority, urgency, and importance 274

The Eisenhower decision matrix 276

How to say no to a commitment 277

12.3 Structuring your team's work 280

Time-slice your work 280

Populating the iteration 281

12.4 Unplanned work 283

Controlling unplanned work 283

Dealing with unplanned work 286

Wrapping it all up 290

Index 291