Deep Learning with Structured Data

by Mark Ryan

Overview

Deep Learning with Structured Data teaches you powerful data analysis techniques for tabular data and relational databases.

Summary
Deep learning offers the potential to identify complex patterns and relationships hidden in data of all sorts. Deep Learning with Structured Data shows you how to apply powerful deep learning analysis techniques to the kind of structured, tabular data you'll find in the relational databases that real-world businesses depend on. Filled with practical, relevant applications, this book teaches you how deep learning can augment your existing machine learning and business intelligence systems.

Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.

About the technology
Here’s a dirty secret: Half of the time in most data science projects is spent cleaning and preparing data. But there’s a better way: Deep learning techniques optimized for tabular data and relational databases deliver insights and analysis without requiring intense feature engineering. Learn the skills to unlock deep learning performance with much less data filtering, validating, and scrubbing.

About the book
Deep Learning with Structured Data teaches you powerful data analysis techniques for tabular data and relational databases. Get started using a dataset based on the Toronto transit system. As you work through the book, you’ll learn how easy it is to set up tabular data for deep learning, while solving crucial production concerns like deployment and performance monitoring.

What's inside

    When and where to use deep learning
    The architecture of a Keras deep learning model
    Training, deploying, and maintaining models
    Measuring performance

About the reader
For readers with intermediate Python and machine learning skills.

About the author
Mark Ryan is a Data Science Manager at Intact Insurance. He holds a Master's degree in Computer Science from the University of Toronto.

Table of Contents

1 Why deep learning with structured data?

2 Introduction to the example problem and Pandas dataframes

3 Preparing the data, part 1: Exploring and cleansing the data

4 Preparing the data, part 2: Transforming the data

5 Preparing and building the model

6 Training the model and running experiments

7 More experiments with the trained model

8 Deploying the model

9 Recommended next steps

Product Details

ISBN-13:	9781638357179
Publisher:	Manning
Publication date:	12/08/2020
Sold by:	SIMON & SCHUSTER
Format:	eBook
Pages:	264
File size:	25 MB
Note:	This product may take a few minutes to download.

About the Author

Mark Ryan is a Data Science Manager at Intact Insurance. He holds a Master's degree in Computer Science from the University of Toronto.

Table of Contents

Preface xv

Acknowledgments xvii

About this book xviii

About the author xxii

About the cover illustration xxiii

1 Why deep learning with structured data? 1

1.1 Overview of deep learning 2

1.2 Benefits and drawbacks of deep learning 6

1.3 Overview of the deep learning stack 9

1.4 Structured vs. unstructured data 10

1.5 Objections to deep learning with structured data 12

1.6 Why investigate deep learning with a structured data problem? 14

1.7 An overview of the code accompanying this book 14

1.8 What you need to know 15

1.9 Summary 16

2 Introduction to the example problem and Pandas dataframes 18

2.1 Development environment options for deep learning 19

2.2 Code for exploring Pandas 21

2.3 Pandas dataframes in Python 22

2.4 Ingesting CSV files into Pandas dataframes 24

2.5 Using Pandas to do what you would do with SQL 25

2.6 The major example: Predicting streetcar delays 28

2.7 Why is a real-world dataset critical for learning about deep learning? 30

2.8 Format and scope of the input dataset 31

2.9 The destination: An end-to-end solution 33

2.10 More details on the code that makes up the solutions 35

2.11 Development environments: Vanilla vs. deep-learning-enabled 37

2.12 A deeper look at the objections to deep learning 38

2.13 How deep learning has become more accessible 41

2.14 A first taste of training a deep learning model 42

2.15 Summary 44

3 Preparing the data, part 1: Exploring and cleansing the data 45

3.1 Code for exploring and cleansing the data 46

3.2 Using config files with Python 46

3.3 Ingesting XLS files into a Pandas dataframe 48

3.4 Using pickle to save your Pandas dataframe from one session to another 52

3.5 Exploring the data 54

3.6 Categorizing data into continuous, categorical, and text categories 58

3.7 Cleaning up problems in the dataset: missing data, errors, and guesses 60

3.8 Finding out how much data deep learning needs 65

3.9 Summary 66

4 Preparing the data, part 2: Transforming the data 67

4.1 Code for preparing and transforming the data 68

4.2 Dealing with incorrect values: Routes 68

4.3 Why only one substitute for all bad values? 70

4.4 Dealing with incorrect values: Vehicles 71

4.5 Dealing with inconsistent values: Location 72

4.6 Going the distance: Locations 74

4.7 Fixing type mismatches 77

4.8 Dealing with rows that still contain bad data 78

4.9 Creating derived columns 79

4.10 Preparing non-numeric data to train a deep learning model 80

4.11 Overview of the end-to-end solution 83

4.12 Summary 86

5 Preparing and building the model 87

5.1 Data leakage and features that are fair game for training the model 88

5.2 Domain expertise and minimal scoring tests to prevent data leakage 89

5.3 Preventing data leakage in the streetcar delay prediction problem 90

5.4 Code for exploring Keras and building the model 92

5.5 Deriving the dataframe to use to train the model 93

5.6 Transforming the dataframe into the format expected by the Keras model 97

5.7 A brief history of Keras and TensorFlow 98

5.8 Migrating from TensorFlow 1.x to TensorFlow 2 99

5.9 TensorFlow vs. PyTorch 100

5.10 The structure of a deep learning model in Keras 100

5.11 How the data structure defines the Keras model 104

5.12 The power of embeddings 107

5.13 Code to build a Keras model automatically based on the data structure 109

5.14 Exploring your model 111

5.15 Model parameters 117

5.16 Summary 119

6 Training the model and running experiments 120

6.1 Code for training the deep learning model 121

6.2 Reviewing the process of training a deep learning model 121

6.3 Reviewing the overall goal of the streetcar delay prediction model 124

6.4 Selecting the train, validation, and test datasets 127

6.5 Initial training run 127

6.6 Measuring the performance of your model 130

6.7 Keras callbacks: Getting the best out of your training runs 133

6.8 Getting identical results from multiple training runs 140

6.9 Shortcuts to scoring 141

6.10 Explicitly saving trained models 143

6.11 Running a series of training experiments 143

6.12 Summary 148

7 More experiments with the trained model 150

7.1 Code for more experiments with the model 151

7.2 Validating whether removing bad values improves the model 151

7.3 Validating whether embeddings for columns improve the performance of the model 152

7.4 Comparing the deep learning model with XGBoost 153

7.5 Possible next steps for improving the deep learning model 159

7.6 Summary 160

8 Deploying the model 161

8.1 Overview of model deployment 162

8.2 If deployment is so important, why is it so hard? 163

8.3 Review of one-off scoring 164

8.4 The user experience with web deployment 165

8.5 Steps to deploy your model with web deployment 165

8.6 Behind the scenes with web deployment 169

8.7 The user experience with Facebook Messenger deployment 172

8.8 Behind the scenes with Facebook Messenger deployment 174

8.9 More background on Rasa 175

8.10 Steps to deploy your model in Facebook Messenger with Rasa 177

8.11 Introduction to pipelines 180

8.12 Defining pipelines in the model training phase 183

8.13 Applying pipelines in the scoring phase 186

8.14 Maintaining a model after deployment 188

8.15 Summary 190

9 Recommended next steps 192

9.1 Reviewing what we have covered so far 193

9.2 What we could do next with the streetcar delay prediction project 194

9.3 Adding location details to the streetcar delay prediction project 194

9.4 Training our deep learning model with weather data 198

9.5 Adding season or time of day to the streetcar delay prediction project 203

9.6 Imputation: An alternative to removing records with bad values 204

9.7 Making the web deployment of the streetcar delay prediction model generally available 204

9.8 Adapting the streetcar delay prediction model to a new dataset 206

9.9 Preparing the dataset and training the model 209

9.10 Deploying the model with web deployment 211

9.11 Deploying the model with Facebook Messenger 212

9.12 Adapting the approach in this book to a different dataset 215

9.13 Resources for additional learning 219

9.14 Summary 220

Appendix Using Google Colaboratory 223

Index 233