Essential Cybersecurity Science: Build, Test, and Evaluate Secure Systems

by Josiah Dykstra

Overview

If you’re involved in cybersecurity as a software developer, forensic investigator, or network administrator, this practical guide shows you how to apply the scientific method when assessing techniques for protecting your information systems. You’ll learn how to conduct scientific experiments on everyday tools and procedures, whether you’re evaluating corporate security systems, testing your own security product, or looking for bugs in a mobile game.

Once author Josiah Dykstra gets you up to speed on the scientific method, he helps you focus on standalone, domain-specific topics, such as cryptography, malware analysis, and system security engineering. The latter chapters include practical case studies that demonstrate how to use available tools to conduct domain-specific scientific experiments.

• Learn the steps necessary to conduct scientific experiments in cybersecurity
• Explore fuzzing to test how your software handles various inputs
• Measure the performance of the Snort intrusion detection system
• Locate malicious “needles in a haystack” in your network and IT environment
• Evaluate cryptography design and application in IoT products
• Conduct an experiment to identify relationships between similar malware binaries
• Understand system-level security requirements for enterprise networks and web services

Product Details

ISBN-13:	9781491920947
Publisher:	O'Reilly Media, Incorporated
Publication date:	01/01/2016
Pages:	188
Product dimensions:	6.90(w) x 9.10(h) x 0.50(d)

About the Author

Josiah Dykstra is a Senior Researcher at the Department of Defense. Dykstra received his PhD in Computer Science from the University of Maryland, Baltimore County, researching the technical and legal challenges of digital forensics for cloud computing. He is known in the DoD and forensics communities for his work on network security, intrusion detection, malware analysis, digital forensics, and cloud computing. He is a member of the ACM, IEEE, American Academy of Forensic Sciences, Cloud Security Alliance, and American Bar Association.

Table of Contents

Preface vii

1 Introduction to Cybersecurity Science 1

What Is Cybersecurity Science? 2

The Importance of Cybersecurity Science 5

The Scientific Method 7

Cybersecurity Theory and Practice 9

Pseudoscience 10

Human Factors 10

Roles Humans Play in Cybersecurity Science 11

Human Cognitive Biases 11

The Role of Metrics 12

Conclusion 13

References 13

2 Conducting Your Own Cybersecurity Experiments 15

Asking Good Questions and Formulating Hypotheses 15

Creating a Hypothesis 15

Security and Testability 18

Designing a Fair Test 19

Analyzing Your Results 21

Putting Results to Work 25

A Checklist for Conducting Experimentation 26

Conclusion 28

References 29

3 Cybersecurity Experimentation and Test Environments 31

Modeling and Simulation 32

Open Datasets for Testing 34

Desktop Testing 35

Cloud Computing 36

Cybersecurity Testbeds 37

A Checklist for Selecting an Experimentation and Test Environment 38

Conclusion 39

References 39

4 Software Assurance 41

An Example Scientific Experiment in Software Assurance 42

Fuzzing for Software Assurance 43

The Scientific Method and the Software Development Life Cycle 45

Adversarial Models 46

Case Study: The Risk of Software Exploitability 47

A New Experiment 48

How to Find More Information 51

Conclusion 51

References 51

5 Intrusion Detection and Incident Response 53

An Example Scientific Experiment in Intrusion Detection 54

False Positives and False Negatives 55

Performance, Scalability, and Stress Testing 58

Case Study: Measuring Snort Detection Performance 60

Building on Previous Work 60

A New Experiment 62

How to Find More Information 64

Conclusion 64

References 64

6 Situational Awareness and Data Analytics 65

An Example Scientific Experiment in Situational Awareness 66

Experimental Results to Assist Human Network Defenders 68

Machine Learning and Data Mining for Network Monitoring 70

Case Study: How Quickly Can You Find the Needle in the Haystack? 73

A New Experiment 74

How to Find More Information 75

Conclusion 75

References 75

7 Cryptography 77

An Example Scientific Experiment in Cryptography 77

Experimental Evaluation of Cryptographic Designs and Implementation 78

Provably Secure Cryptography and Security Assumptions 80

Cryptographic Security and the Internet of Things 83

Case Study: Evaluating Composable Security 85

Background 85

A New Experiment 86

How to Find More Information 87

Conclusion 87

References 88

8 Digital Forensics 89

An Example Scientific Experiment in Digital Forensics 89

Scientific Validity and the Law 90

Scientific Reproducibility and Repeatability 93

Case Study: Scientific Comparison of Forensic Tool Performance 94

How to Find More Information 96

Conclusion 97

References 97

9 Malware Analysis 99

An Example Scientific Experiment in Malware Analysis 100

Scientific Data Collection for Simulators and Sandboxes 100

Game Theory for Malware Analysis 103

Case Study: Identifying Malware Families with Science 106

Building on Previous Work 106

A New Experiment 107

How to Find More Information 108

Conclusion 108

References 108

10 System Security Engineering 111

An Example Scientific Experiment in System Security Engineering 113

Regression Analysis 115

Moving Target Defense 118

Case Study: Defending Against Unintentional Insider Threats 120

How to Find More Information 122

Conclusion 122

References 122

11 Human-Computer Interaction and Usable Security 125

An Example Scientific Experiment in Usable Security 126

Double-Blind Experimentation 128

Usability Measures: Effectiveness, Efficiency, and Satisfaction 129

Methods for Gathering Usability Data 132

Testing Usability during Design 132

Testing Usability during Validation and Verification 134

Case Study: An Interface for User-Friendly Encrypted Email 135

A New Experiment 136

How to Find More Information 138

Conclusion 138

References 139

12 Visualization 141

An Example Scientific Experiment in Cybersecurity Visualization 142

Graphical Representations of Cybersecurity Data 145

Experimental Evaluation of Security Visualization 148

Case Study: Is My Visualization Helping Users Work More Effectively? 152

How to Find More Information 154

Conclusion 154

References 154

A Understanding Bad Science, Scientific Claims, and Marketing Hype 157

Dangers of Manipulative Graphics and Visualizations 158

Recognizing and Understanding Scientific Claims 160

Vendor Marketing 163

Clarifying Questions for Salespeople, Researchers, and Developers 164

References 166

Index 167