Table of Contents
Preface xi
1 Introduction to Al and Machine Learning 1
What Is Artificial Intelligence? 1
What Is Machine Learning? 3
Moving from Traditional Programming to Machine Learning 3
How Can a Machine Learn? 6
Comparing Machine Learning with Traditional Programming 11
Building and Using Models on Mobile 13
Summary 13
2 Introduction to Computer Vision 15
Using Neurons for Vision 16
Your First Classifier: Recognizing Clothing Items 21
The Data: Fashion MNIST 22
A Model Architecture to Parse Fashion MNIST 23
Coding the Fashion MNIST Model 25
Transfer Learning for Computer Vision 29
Summary 32
3 Introduction to ML Kit 33
Building a Face Detection App on Android 34
Step 1 Create the App with Android Studio 35
Step 2 Add and Configure ML Kit 37
Step 3 Define the User Interface 38
Step 4 Add the Images as Assets 39
Step 5 Load the UI with a Default Picture 40
Step 6 Call the Face Detector 42
Step 7 Add the Bounding Rectangles 43
Building a Face Detector App for iOS 45
Step 1 Create the Project in Xcode 45
Step 2 Using CocoaPods and Podfiles 47
Step 3 Create the User Interface 48
Step 4 Add the Application Logic 52
Summary 55
4 Computer Vision Apps with ML Kit on Android 57
Image Labeling and Classification 57
Step 1 Create the App and Configure ML Kit 58
Step 2 Create the User Interface 59
Step 3 Add the Images as Assets 60
Step 4 Load an Image to the ImageView 60
Step 5 Write the Button Handler Code 61
Next Steps 63
Object Detection 63
Step 1 Create the App and Import ML Kit 65
Step 2 Create the Activity Layout XML 65
Step 3 Load an Image into the ImageView 65
Step 4 Set Up the Object Detector Options 66
Step 5 Handling the Button Interaction 67
Step 6 Draw the Bounding Boxes 67
Step 7 Label the Objects 69
Detecting and Tracking Objects in Video 70
Exploring the Layout 71
The GraphicOverlay Class 72
Capturing the Camera 73
The ObjectAnalyzer Class 74
The ObjectGraphic Class 75
Putting It All Together 75
Summary 76
5 Text Processing Apps with ML Kit on Android 77
Entity Extraction 77
Start Creating the App 79
Create the Layout for the Activity 79
Write the Entity Extraction Code 80
Putting It All Together 82
Handwriting and Other Recognition 84
Start the App 85
Creating a Drawing Surface 85
Parsing the Ink with ML Kit 87
Smart Reply to Conversations 90
Start the App 91
Mock a Conversation 91
Generating a Smart Reply 92
Summary 92
6 Computer Vision Apps with ML Kit on iOS 95
Image Labeling and Classification 95
Step 1 Create the App in Xcode 96
Step 2 Create the Podfile 96
Step 3 Set Up the Storyboard 98
Step 4 Edit the View Controller Code to Use ML Kit 100
Object Detection in iOS with ML Kit 103
Step 1 Get Started 104
Step 2 Create Your UI on the Storyboard 105
Step 3 Create a Subview for Annotation 106
Step 4 Perform the Object Detection 107
Step 5 Handle the Callback 108
Combining Object Detection with Image Classification 111
Object Detection and Tracking in Video 112
Summary 115
7 Text Processing Apps with ML Kit on iOS 117
Entity Extraction 117
Step 1 Create the App and Add the ML Kit Pods 118
Step 2 Create the Storyboard with Actions and Outlets 119
Step 3 Allow Your View Controller to be Used for Text Entry 120
Step 4 Initialize the Model 120
Step 5 Extract Entities from Text 121
Handwriting Recognition 123
Step 1 Create the App and Add the ML Kit Pods 123
Step 2 Create the Storyboard, Actions, and Outlets 124
Step 3 Strokes, Points, and Ink 125
Step 4 Capture User Input 125
Step 5 Initialize the Model 127
Step 6 Do the Ink Recognition 128
Smart Reply to Conversations 129
Step 1 Create an App and Integrate ML Kit 130
Step 2 Create Storyboard, Outlets, and Actions 130
Step 3 Create a Simulated Conversation 131
Step 4 Get Smart Reply 133
Summary 134
8 Going Deeper: Understanding TensorFlow Lite 135
What Is TensorFlow Lite? 135
Getting Started with TensorFlow Lite 137
Save the Model 138
Convert the Model 139
Testing the Model with a Standalone Interpreter 140
Create an Android App to Host TFLite 142
Import the TFLite File 144
Write Kotlin Code to Interface with the Model 144
Going Beyond the Basics 148
Create an iOS App to Host TFLite 151
Step 1 Create a Basic iOS App 151
Step 2 Add TensorFlow Lite to Your Project 152
Step 3 Create the User Interface 153
Step 4 Add and Initialize the Model Inference Class 156
Step 5 Perform the Inference 158
Step 6 Add the Model to Your App 160
Step 7 Add the UI Logic 162
Moving Beyond "Hello World": Processing Images 164
Exploring Model Optimization 167
Quantization 167
Using Representative Data 170
Summary 171
9 Creating Custom Models 173
Creating a Model with TensorFlow Lite Model Maker 174
Creating a Model with Cloud Auto ML 179
Using AutoML Vision Edge 179
Creating a Model with TensorFlow and Transfer Learning 189
Creating Language Models 191
Create a Language Model with Model Maker 193
Summary 195
10 Using Custom Models in Android 197
Bridging Models to Android 197
Building an Image Classification App from a Model Maker Output 199
Using a Model Maker Output with ML Kit 203
Using Language Models 206
Creating an Android App for Language Classification 206
Summary 210
11 Using Custom Models in iOS 211
Bridging Models to iOS 211
A Custom Model Image Classifier 213
Step 1 Create the App and Add the TensorFlow Lite Pod 213
Step 2 Create the UI and Image Assets 214
Step 3 Load and Navigate Through the Image Assets 216
Step 4 Load the Model 217
Step 5 Convert an Image to an Input Tensor 218
Step 6 Get Inference for the Tensor 221
Use a Custom Model in ML Kit 222
Building an App for Natural Language Processing in Swift 224
Step 1 Load the Vocab 226
Step 2 Convert the Sentence to a Sequence 228
Step 3 Extend Array to Handle Unsafe Data 228
Step 4 Copy the Array to a Data Buffer 229
Step 5 Run Inference on the Data and Process the Results 230
Summary 230
12 Productizing Your App Using Firebase 233
Why Use Firebase Custom Model Hosting? 233
Create Multiple Model Versions 234
Using Firebase Model Hosting 235
Step 1 Create a Firebase Project 236
Step 2 Use Custom Model Hosting 240
Step 3 Create a Basic Android App 243
Step 4 Add Firebase to the App 244
Step 5 Get the Model from Firebase Model Hosting 246
Step 6 Use Remote Configuration 248
Step 7 Read Remote Configuration in Your App 251
Next Steps 252
Summary 252
13 Create ML and Core ML for Simple iOS Apps 255
A Core ML Image Classifier Built Using Create ML 255
Making a Core ML App That Uses a Create ML Model 262
Add the MLModel File 263
Run the Inference 263
Using Create ML to Build a Text Classifier 268
Use the Model in an App 269
Summary 272
14 Accessing Cloud-Based Models from Mobile Apps 273
Installing TensorFlow Serving 274
Installing Using Docker 274
Installing Directly on Linux 276
Building and Serving a Model 276
Accessing a Server Model from Android 279
Accessing a Server Model from iOS 283
Summary 285
15 Ethics, Fairness, and Privacy for Mobile Apps 287
Ethics, Fairness, and Privacy with Responsible AI 288
Responsibly Defining Your Problem 288
Avoiding Bias in Your Data 289
Building and Training Your Model 295
Evaluating Your Model 297
Google's AI Principles 299
Summary 300
Index 301
