Effective Software Testing: A developer's guide

by Mauricio Aniche

Overview

Go beyond basic testing! Great software testing makes the entire development process more efficient. This book reveals a systemic and effective approach that will help you customize your testing coverage and catch bugs in tricky corner cases.

In Effective Software Testing you will learn how to:

Engineer tests with a much higher chance of finding bugs
Read code coverage metrics and use them to improve your test suite
Understand when to use unit tests, integration tests, and system tests
Use mocks and stubs to simplify your unit testing
Think of pre-conditions, post-conditions, invariants, and contracts
Implement property-based tests
Utilize coding practices like dependency injection and hexagonal architecture that make your software easier to test
Write good and maintainable test code

Effective Software Testing teaches you a systematic approach to software testing that will ensure the quality of your code. It’s full of techniques drawn from proven research in software engineering, and each chapter puts a new technique into practice. Follow the real-world use cases and detailed code samples, and you’ll soon be engineering tests that find bugs in edge cases and parts of code you’d never think of testing! Along the way, you’ll develop an intuition for testing that can save years of learning by trial and error.

Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.

About the technology
Effective testing ensures that you’ll deliver quality software. For software engineers, testing is a key part of the development process. Mastering specification-based testing, boundary testing, structural testing, and other core strategies is essential to writing good tests and catching bugs before they hit production.

About the book
Effective Software Testing is a hands-on guide to creating bug-free software. Written for developers, it guides you through all the different types of testing, from single units up to entire components. You’ll also learn how to engineer code that facilitates testing and how to write easy-to-maintain test code. Offering a thorough, systematic approach, this book includes annotated source code samples, realistic scenarios, and reasoned explanations.

What's inside

Design rigorous test suites that actually find bugs
When to use unit tests, integration tests, and system tests
Pre-and post-conditions, invariants, contracts, and property-based tests
Design systems that are test-friendly
Test code best practices and test smells

About the reader
The Java-based examples illustrate concepts you can use for any object-oriented language.

About the author
Dr. Maurício Aniche is the Tech Academy Lead at Adyen and an Assistant Professor in Software Engineering at the Delft University of Technology.

Table of Contents
1 Effective and systematic software testing
2 Specification-based testing
3 Structural testing and code coverage
4 Designing contracts
5 Property-based testing
6 Test doubles and mocks
7 Designing for testability
8 Test-driven development
9 Writing larger tests
10 Test code quality
11 Wrapping up the book

Product Details

ISBN-13:	9781633439931
Publisher:	Manning
Publication date:	04/26/2022
Pages:	328
Product dimensions:	7.38(w) x 9.25(h) x 0.80(d)

About the Author

Dr. Maurício Aniche leads the Tech Academy of Adyen, and is also an Assistant Professor in Software Engineering at Delft University of Technology in the Netherlands. He researches on how to make developers more productive during testing and maintenance and his teaching efforts in software testing have awarded him the Teacher of Year 2021 award and the TU Delft Education Fellowship. Maurício holds MSc and PhD degrees in Computer Science from the University of São Paulo, Brazil. He also co-founded Alura, one of the most popular e-learning platforms for software engineers in Brazil.

Table of Contents

Forewords xiii

Preface xvi

Acknowledgments xviii

About this book xxi

About the author xxv

About the cover illustration xxvi

1 Effective and systematic software testing 1

1.1 Developers who test vs. developers who do not 2

1.2 Effective software testing for developers 11

Effective testing in the development process 12

Effective testing as an iterative process 14

Focusing on development and then on testing 14

The myth of "correctness by design" 15

The cost of testing 15

The meaning of effective and systematic 15

The role of test automation 16

1.3 Principles of software testing (or, why testing is so difficult) 16

Exhaustive testing is impossible 16

Knowing when to stop testing 17

Variability is important (the pesticide paradox) 17

Bugs happen in some places more than others 17

No matter what testing you do, it will never be perfect or enough 18

Context is king 18

Verification is not validation 18

1.4 The testing pyramid, and where we should focus 19

Unit testing 19

Integration testing 20

System testing 21

When to use each test level 23

Why do I favor unit tests? 23

What do I test at the different levels? 24

What if you disagree with the testing pyramid? 25

Will this hook help you find all the bugs? 27

2 Specification-based testing 30

2.1 The requirements say it all 31

Step 1: Understanding the requirements, inputs, and outputs 33

Step 2: Explore what the program does for various inputs 34

Step 3: Explore possible inputs and outputs, and identify partitions 35

Step 4: Analyze the boundaries 37

Step 5: Devise test cases 39

Step 6: Automate the test cases 41

Step 7: Augment the test suite with creativity and experience 43

2.2 Specification-based testing in a nutshell 45

2.3 Finding bugs with specification testing 46

2.4 Specification-based testing in the real world 54

The process should be iterative, not sequential 54

How far should specification testing go? 55

Partition or boundary? It does not matter! 55

On and off points are enough, but feel free to add in and outpoints 55

Use variations of the same input to facilitate understanding 55

When the number of combinations explodes, be pragmatic 56

When in doubt, go for the simplest input 56

Pick reasonable values for inputs you do not care about 56

Test for nulls and exceptional cases, but only when it makes sense 56

Go for parameterized tests when tests have the same skeleton 57

Requirements can be of any granularity 57

How does this work with classes and state? 57

The role of experience and creativity 59

3 Structural testing and code coverage 63

3.1 Code coverage, the right way 64

3.2 Structural testing in a nutshell 68

3.3 Code coverage criteria 69

Line coverage 69

Branch coverage 69

Condition + branch coverage 70

Path coverage 71

3.4 Complex conditions and the MC/DC coverage criterion 72

An abstract example 72

Creating a test suite that achieves MC/DC 73

3.5 Handling loops and similar constructs 75

3.6 Criteria subsumption, and choosing a criterion 75

3.7 Specification-based and structural testing: A running example 77

3.8 Boundary testing and structural testing 82

3.9 Structural testing alone often is not enough 82

3.10 Structural testing in the real world 84

Why do some people hate code coverage? 84

What does it mean to achieve 100% coverage? 86

What coverage criterion to use 88

MC/DC when expressions are too complex and cannot be simplified 88

Other coverage criteria 89

What should not be covered? 90

3.11 Mutation testing 90

4 Designing contracts 97

4.1 Pre-conditions and post-conditions 98

The assert keyword 99

Strong and weak pre- and postconditions 100

4.2 Invariants 102

4.3 Changing contracts, and the Liskov substitution principle 105

Inheritance and contracts 107

4.4 How is design-by-contract related to testing? 109

4.5 Design-by-contract in the real world 110

Weak or strong pre-conditions? 110

Input validation, contracts, or both? 110

Asserts and exceptions: When to use one or the other 112

Exception or soft return values? 113

When hot to use design-by-contract 113

Should we write tests for pre-conditions, post-conditions, and invariants? 114

Tooling support 114

5 Property-based testing 117

5.1 Example 1: The passing grade program 118

5.2 Example 2: Testing the unique method 122

5.3 Example 3: Testing the indexOf method 124

5.4 Example 4: Testing the Basket class 129

5.5 Example 5: Creating complex domain objects 136

5.6 Property-based testing in the real world 137

Example-based testing vs. property-based testing 137

Common issues in property-based tests 138

Creativity is key 139

6 Test doubles and mocks 141

6.1 Dummies, fakes, stubs, spies, and mocks 143

Dummy objects 143

Fake objects 144

Stubs 144

Mocks 144

Spies 144

6.2 An introduction to mocking frameworks 145

Stubbing dependencies 145

Mocks and expectations 150

Capturing arguments 153

Simulating exceptions 157

6.3 Mocks in the real world 158

The disadvantages of mocking 159

What to mock and what not to mock 160

Date and time wrappers 164

Mocking types you do not own 166

What do others say about mocking? 168

7 Designing for testability 172

7.1 Separating infrastructure code from domain code 173

7.2 Dependency injection and controllability 181

7.3 Making your classes and methods observable 184

Example 1: Introducing methods to facilitate assertions 184

Example 2: Observing the behavior of void methods 186

7.4 Dependency via class constructor or value via method parameter? 189

7.5 Designing for testability in the real world 191

The cohesion of the class under test 192

The coupling of the class under test 193

Complex conditions and testability 193

Private methods and testability 193

Static methods, singletons, and testability 194

The Hexagonal Architecture and mocks as a design technique 194

Further reading about designing for testability 195

8 Test-driven development 198

8.1 Our first TDD session 199

8.2 Reflecting on our first TDD experience 206

8.3 TDD in the real world 208

To TDD or not to TDD? 208

TDD 100% of the time? 209

Does TDD work for all types of applications and domains? 209

What does the research say about TDD? 209

Other schools of TDD 211

TDD and proper testing 212

9 Writing larger tests 215

9.1 When to use larger tests 216

Testing larger components 216

Testing larger components that go beyond our code base 224

9.2 Database and SQL testing 229

What to test in a SQL query 229

Writing automated tests for SQL queries 231

Setting up infrastructure for SQL tests 236

Best practices 238

9.3 System tests 239

An introduction to Selenium 239

Designing page objects 242

Patterns and best practices 251

9.4 Final notes on larger tests 254

How do all the testing techniques fit? 254

Perform cost/benefit analysis 255

Be careful with methods that are covered but not tested 255

Proper code infrastructure is key 255

DSLs and tools for stakeholders to write tests 256

Testing other types of web systems 256

10 Test code quality 258

10.1 Principles of maintainable test code 259

Tests should be fast 259

Tests should be cohesive, independent, and isolated 259

Tests should have a reason to exist 260

Tests should be repeatable and not flaky 260

Tests should have strong assertions 261

Tests should break if the behavior changes 261

Tests should have a single and clear reason to fail 262

Tests should be easy to write 262

Tests should be easy to read 262

Tests should be easy to change and evolve 266

10.2 Test smells 267

Excessive duplication 267

Unclear assertions 268

Bad handling of complex or external resources 268

Fixtures that are too general 269

Sensitive assertions 270

11 Wrapping up the book 276

11.1 Although the model looks linear, iterations are fundamental 276

11.2 Bug-free software development: Reality or myth? 277

11.3 Involve your final user 278

11.4 Unit testing is hard in practice 278

11.5 Invest in monitoring 279

11.6 What's next? 280

Appendix Answers to exercises 281

References 289

Index 295