Google Cloud Cookbook: Practical Solutions for Building and Deploying Cloud Services

by Rui Costa, Drew Hodun

Overview

Get quick hands-on experience with Google Cloud. This cookbook provides a variety of self-contained recipes that show you how to use Google Cloud services for your enterprise application. Whether you're looking for practical ways to apply microservices, AI, analytics, security, or networking solutions, these recipes take you step-by-step through the process and provide discussions that explain how and why the recipes work.

Ideal for system engineers and administrators, developers, network and database administrators, and data analysts, this cookbook helps you get started with Google Cloud regardless of your level of experience. Google veterans Rui Costa and Drew Hodun also cover advanced-level Google Cloud services for those who have appreciable experience with the platform.

• Learn how to get started with Google Cloud
• Understand the depth of services Google Cloud provides
• Gain hands-on experience using practical examples and labs
• Explore topics that include BigQuery, Cloud Run, and Kubernetes
• Build and run mobile and web applications on Google Cloud
• Examine ways to build your cloud applications for scale
• Build a minimum viable product (MVP) app to use in production
• Learn data platform and pipeline skills

Product Details

ISBN-13:	9781492092896
Publisher:	O'Reilly Media, Incorporated
Publication date:	11/16/2021
Pages:	282
Product dimensions:	6.90(w) x 9.10(h) x 0.70(d)

About the Author

Rui Costa has worked at Google in various roles, most recently as a Learning Consultant working with strategic customers and partners to create and execute on their Google Cloud learning plans. He has also served as an AI Coach for the Google AI Impact Challenge, where he had the opportunity to speak at multiple conferences including the NetHope Global Summit 2019. He is also the founder of the Speech Analysis Framework, which has successfully graduated to become a Product within Google.

Drew Hodun has had a number of roles at Google, all customer facing and focused on Machine Learning applications running on Google Cloud. He’s worked primarily in the Autonomous Vehicle, Financial Services, and Media+Entertainment industries. Most recently Drew is a Learning Consultant, working with large strategic customers and building engineer to engineer workshops to meet their toughest training needs. Drew has spoken at a variety of conferences, including Predictive Analytics World 2020, Google Cloud Next ‘18-’19, O’Reilly AI Conference ‘18, and others.

Table of Contents

Preface vii

1 Introduction 1

2 Cloud Functions 25

2.1 Creating a Public HTTP Google Cloud Function 26

2.2 Authenticating an HTTP Google Cloud Function 27

2.3 Accessing Environment Variables at Runtime 29

2.4 Sending Emails from Cloud Functions with SendGrid 30

2.5 Deploying Cloud Functions with a GitLab CI/CD Pipeline 33

2.6 Responding to SMS Messages with Twilio and Cloud Functions 37

2.7 Unit Testing with GitLab and Cloud Functions 39

2.8 Building an API Gateway to Gather Telemetry Data 42

3 Google Cloud Run 47

3.1 Deploying a Prebuilt Hello World Container 48

3.2 Building Your Own Hello World Container 51

3.3 Using Cloud Run with a Custom Domain 54

3.4 Triggering a Cloud Run from Cloud Pub/Sub 56

3.5 Deploying a Web Application to Cloud Run 61

3.6 Rolling Back a Cloud Run Service Deployment 63

3.7 Deploying Cloud Run Services in a Gradual Rollout 64

3.8 Cloud Run Configuration Parameters 65

4 Google App Engine 69

4.1 Deploying a Hello World to App Engine (Standard) 70

4.2 Deploying a Hello World to App Engine (Flexible) 71

4.3 Securing Your Application with Identity-Aware Proxy 73

4.4 Mapping Custom Domains with App Engine 76

4.5 Using the Google Cloud Translation Machine Learning APIs with App Engine 78

4.6 Building User Interfaces for Viewing Charts and Graphs 80

4.7 Debugging an Instance 88

4.8 Using CI/CD 89

5 Google Cloud Compute Engine 91

5.1 Creating a Windows Virtual Machine 91

5.2 Creating a Linux Virtual Machine and Installing NGINX 93

5.3 Connecting to Your Windows Virtual Machines with Identity-Aware Proxy TCP Forwarding 96

5.4 Securing Your Virtual Machine Logins with Two-Step Verification 100

5.5 Running Startup Scripts 102

5.6 Creating a Group of NGINX Web Servers with a Managed Instance Group 105

5.7 Deploying Containers to Managed Instance Groups 107

5.8 Transferring Files to Your Virtual Machine 109

5.9 Using VM Manager for Patch Management 111

5.10 Backing Up Your Virtual Machine 113

6 Google Cloud Kubernetes Engine 115

6.1 Creating a Zonal Cluster 115

6.2 Creating a Regional Cluster 117

6.3 Resizing a Cluster 118

6.4 Automatically Routing Traffic to the Nearest Cluster with Multi-Cluster Ingress 121

6.5 Deploying a Spring Boot Java Application 126

6.6 Deploying a Java Application to Kubernetes, Using Skaffold 129

6.7 Using GKE Autopilot for Running an Application You Don't Have to Manage 130

7 Working with Data 133

7.1 Speeding Up Cloud Storage Bulk Transfers by Multiprocessing 133

7.2 Speeding Up GCS Transfers for Large Files with Parallel Composite Uploads 135

7.3 Mounting GCS as a Filesystem 136

7.4 Automatically Archiving and Deleting GCS Objects 138

7.5 Creating and Restoring from Persistent Disk Snapshots in GCE 140

7.6 Using Interleaved Tables in Your Cloud Spanner Database 142

7.7 Locking Down Firestore Database So a User Can Edit Only Their Data 147

8 BigQuery and Data Warehousing 149

8.1 Using Cloud Console to Run a BigQuery Query 149

8.2 Loading Data to BigQuery from CSV 152

8.3 Building a Pivot Table in BigQuery 156

8.4 Adding Partitioned and Clustered Columns to an Existing Table 158

8.5 Adding Clustering to a Table That Can't or Shouldn't Be Partitioned 162

8.6 Selecting the Top-1 Result 163

8.7 Merging Tables in BigQuery Without Duplicates 165

8.8 Deduplicating Rows in BigQuery with Timestamps 167

8.9 Undeleting a Table in BigQuery 169

8.10 Streaming Json or Avro Data into BigQuery with a Dataflow Template 170

9 Data Processing Tools 175

9.1 Cleaning Data Using the Data Fusion GUI 175

9.2 Running a Simple Python Dataflow Pipeline 180

9.3 Building a Streaming Pipeline in Dataflow SQL 184

9.4 Querying BigQuery from a Dataproc Job 188

9.5 Adding Event Timestamps to Pub/Sub 191

9.6 Inferring and Using Schemas in Dataflow 192

9.7 Mini-batching and Streaming Dataflow Data to BigQuery Using Filters 195

9.8 Triggering a Dataflow Job Automatically from a GCS Upload 197

10 AI/ML 201

10.1 Creating a Vertex AI Notebook 201

10.2 Training a Python ML Model Serverlessly 204

10.3 Making Serverless Predictions with a Python Model 206

10.4 Creating a Custom Notebook Environment 208

10.5 Extracting Data from BigQuery to Pandas for Model Training 210

10.6 Training a Model in SQL with BQML 213

11 Google Cloud Security and Access 217

11.1 Creating a Service Account 217

11.2 Creating Custom Roles to Access a Cloud Storage Bucket 219

11.3 Authenticating an Application Running on Kubernetes Engine 221

11.4 Retrieving the Authenticated User's Identity 227

11.5 Authenticating a Java Application Using a Service Account 228

11.6 Building Reports Using the Cloud Asset API 230

11.7 Allowing a List of IP Addresses to Access Your Application 234

12 Google Cloud Networking 237

12.1 Creating a Custom Mode VPC Network 237

12.2 Creating a Static External IP Address 239

12.3 Create a Firewall Rule 239

12.4 Serving Content for Users in a Specific Region 241

12.5 Configuring VPC Network Peering 244

12.6 Creating VPN Gateways with Cloud Routers 247

12.7 Deployments of Networks Using Terraform 252

12.8 Limiting Access to Only Authorized Networks with VPC Service Controls 253

Index 257