Automated Machine Learning in Action

by Qingquan Song, Haifeng Jin, Xia Hu

Overview

Optimize every stage of your machine learning pipelines with powerful automation components and cutting-edge tools like AutoKeras and KerasTuner.

In Automated Machine Learning in Action you will learn how to:

Improve a machine learning model by automatically tuning its hyperparameters
Pick the optimal components for creating and improving your pipelines
Use AutoML toolkits such as AutoKeras and KerasTuner
Design and implement search algorithms to find the best component for your ML task
Accelerate the AutoML process with data-parallel, model pretraining, and other techniques

Automated Machine Learning in Action reveals how you can automate the burdensome elements of designing and tuning your machine learning systems. It’s written in a math-lite and accessible style, and filled with hands-on examples for applying AutoML techniques to every stage of a pipeline. AutoML can even be implemented by machine learning novices! If you’re new to ML, you’ll appreciate how the book primes you on machine learning basics. Experienced practitioners will love learning how automated tools like AutoKeras and KerasTuner can create pipelines that automatically select the best approach for your task, or tune any customized search space with user-defined hyperparameters, which removes the burden of manual tuning.

Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.

About the technology
Machine learning tasks like data pre-processing, feature selection, and model optimization can be time-consuming and highly technical. Automated machine learning, or AutoML, applies pre-built solutions to these chores, eliminating errors caused by manual processing. By accelerating and standardizing work throughout the ML pipeline, AutoML frees up valuable data scientist time and enables less experienced users to apply machine learning effectively.

About the book
Automated Machine Learning in Action shows you how to save time and get better results using AutoML. As you go, you’ll learn how each component of an ML pipeline can be automated with AutoKeras and KerasTuner. The book is packed with techniques for automating classification, regression, data augmentation, and more. The payoff: Your ML systems will be able to tune themselves with little manual work.

What's inside

Automatically tune model hyperparameters
Pick the optimal pipeline components
Select appropriate models and features
Learn different search algorithms and acceleration strategies

About the reader
For ML novices building their first pipelines and experienced ML engineers looking to automate tasks.

About the author
Drs. Qingquan Song, Haifeng Jin, and Xia “Ben” Hu are the creators of the AutoKeras automated deep learning library.

Table of Contents
PART 1 FUNDAMENTALS OF AUTOML
1 From machine learning to automated machine learning
2 The end-to-end pipeline of an ML project
3 Deep learning in a nutshell
PART 2 AUTOML IN PRACTICE
4 Automated generation of end-to-end ML solutions
5 Customizing the search space by creating AutoML pipelines
6 AutoML with a fully customized search space
PART 3 ADVANCED TOPICS IN AUTOML
7 Customizing the search method of AutoML
8 Scaling up AutoML
9 Wrapping up

Product Details

ISBN-13:	9781617298059
Publisher:	Manning
Publication date:	06/07/2022
Pages:	336
Product dimensions:	7.38(w) x 9.25(h) x 0.90(d)

About the Author

Qingquan Song, Haifeng Jin, and Dr. Xia “Ben” Hu are the creators of the AutoKeras automated deep learning library. Qingquan and Haifeng are PhD students at Texas A&M University, and have both published papers at major data mining conferences and journals. Dr. Hu is an associate professor at Texas A&M University in the Department of Computer Science and Engineering, whose work has been utilized by TensorFlow, Apple, and Bing.

Table of Contents

Preface xiii

Acknowledgments xv

About this book xvii

About the authors xx

About the cover illustration xxii

Part 1 Fundamentals of AutoML 1

1 From machine learning to automated machine learning 3

1.1 A glimpse of automated machine learning 4

1.2 Getting started with machine learning 6

What is machine learning? 6

The machine learning process 7

Hyperparameter tuning 9

The obstacles to applying machine learning 11

1.3 AutoML: The automation of automation 12

Three key components of AutoML 12

Are, we able to achieve full automation? 15

2 The end-to-end pipeline of an ML project 17

2.1 An overview of the end-to-end pipeline 18

2.2 Framing the problem and assembling the dataset 19

2.3 Data preprocessing 22

2.4 Feature engineering 25

2.5 ML algorithm selection 28

Building the linear regression model 29

Building the decision tree model 31

2.6 Fine-tuning the ML model: Introduction to grid search 34

3 Deep learning in a nutshell 41

3.1 What is deep learning? 42

3.2 TensorFlow and Keras 43

3.3 California housing price prediction with a multilayer perceptron 43

Assembling and preparing the data 44

Building up the multilayer perceptron 45

Training and testing the neural network 49

Tuning the number of epochs 52

3.4 Classifying handwritten digits with convolutional neural networks 55

Assembling and preparing the dataset 55

Addressing the problem with an MLP 57

Addressing the problem with a CNN 59

3.5 IMDB review classification with recurrent neural networks 64

Preparing the data 65

Building up the RNN 67

Training and validating the RNN 69

Part 2 AutoML in Practice 71

4 Automated generation of end-to-end ML solutions 73

4.1 Preparing the AutoML toolkit: AutoKeras 73

4.2 Automated image classification 76

Attacking the problem with five lines of code 76

Dealing with different data formats 80

Configuring the tuning process 81

4.3 End-to-end AutoML solutions for four supervised learning problems 83

Text classification with the 20 newsgroups dataset 83

Structured data classification with the Titanic dataset 85

Structured data regression with the California housing dataset 88

Multilabel image classification 89

4.4 Addressing tasks with multiple inputs or outputs 91

Automated image classification with the AutoKeras 10 API 91

Automated multi-input learning 93

Automated multi-output learning 94

5 Customizing the search space by creating AutoML pipelines 99

5.1 Working with sequential AutoML pipelines 100

5.2 Creating a sequential AutoML pipeline for automated hyper parameter tuning 102

TuningMLPs for structured data regression 103

Tuning CNNs for image classification 109

5.3 Automated pipeline search with hyperblocks 111

Automated model selection for image classification 112

A utomaled selection of image preprocessing methods 117

5.4 Designing a graph-structured AutoML pipeline 121

5.5 Designing custom AutoML blocks 125

Tuning MLPs with a custom MLP block 125

Designing a hyperblock for model selection 132

6 AutoML with a fully customized search space 138

6.1 Customizing the search space in a layerwise fashion 139

Tuning an MLP for regression with KerasTuner 139

Tuning an autoencoder model for unsupervised learning 147

6.2 Tuning the autoencoder model 151

6.3 Tuning shallow models with different search methods 154

Selecting and tuning shallow models 154

Tuning a shallow model pipeline 157

Trying out different search methods 158

Automated feature engineering 159

6.4 Controlling the AutoML process by customizing tuners 169

Creating a tuner for tuning scikit-learn models 170

Creating a tuner for tuning Keras models 174

Jointly tuning and selection among deep learning and shallow models 176

Hyperparameter tuning beyond Keras and scikit-learn models 179

Part 3 Advanced Topics in AutoML 185

7 Customizing the search method of AutoML 187

7.1 Sequential search methods 188

7.2 Getting started with a random search method 189

7.3 Customizing a Bayesian optimization search method 193

Vectorizing the hyperparameters 194

Updating the surrogate junction based on historical model evaluations 198

Designing the acquisition function 201

Sampling the new hypeiparameters via the acquisition junction 204

Tuning the GBDT model with the Bayesian optimization method 206

Resuming the search process and recovering the search method 208

7.4 Customizing an evolutionary search method 210

Selection strategies in the evolutionary search method 210

The aging evolutionary search method 212

Implementing a simple mutation operation 215

Evaluating the aging evolutionary search method 219

8 Scaling up AutoML 223

8.1 Handling large-scale datasets 224

Loading an image-classification dataset 225

Splitting the loaded dataset 226

Loading a text-classification dataset 229

Handling large datasets in general 231

8.2 Parallelization on multiple GPUs 234

Data parallelism 236

Model parallelism 237

Parallel tuning 238

8.3 Search speedup strategies 240

Model scheduling with Hyperband 241

Faster convergence with pretrained weights in the search space 244

Warm-starting the search space 247

9 Wrapping up 249

9.1 Key concepts in review 250

The AutoML process and its key components 250

The machine learning pipeline 251

The taxonomy of AutoML 252

Applications of AutoML 252

Automated deep learning with AutoKeras 253

Fully personalized AutoML with KerasTuner 255

Implementing search techniques 257

Scaling up the AutoML process 258

9.2 AutoML tools and platforms 259

Open source AutoML tools 259

Commercial AutoML platforms 261

9.3 The challenges and future of AutoML 262

Measuring the performance of AutoML 262

Resource complexity 263

Interpretability and transparency 263

Reproducibility and robustness 263

Generalizability and transferability 264

Democratization and productionizalion 264

9.4 Staying up-to-date in a fast-moving field 264

Appendix A Setting up an environment for running code 266

Appendix B Three examples: Classification of image, text, and tabular data 278

Index 305