Google Compute Engine: Managing Secure and Scalable Cloud Computing

by Marc Cohen, Kathryn Hurley, Paul Newson

Overview

Learn how to run large-scale, data-intensive workloads with Compute Engine, Google’s cloud platform. Written by Google engineers, this tutorial walks you through the details of this Infrastructure as a Service by showing you how to develop a project with it from beginning to end. You’ll learn best practices for using Compute Engine, with a focus on solving practical problems.

With programming examples written in Python and JavaScript, you’ll also learn how to use Compute Engine with Docker containers and other platforms, frameworks, tools, and services. Discover how this IaaS helps you gain unparalleled performance and scalability with Google’s advanced storage and computing technologies.

• Access and manage Compute Engine resources with a web UI, command-line interface, or RESTful interface
• Configure, customize, and work with Linux VM instances
• Explore storage options: persistent disk, Cloud Storage, Cloud SQL (MySQL in the cloud), or Cloud Datastore NoSQL service
• Use multiple private networks, and multiple instances on each network
• Build, deploy, and test a simple but comprehensive cloud computing application step-by-step
• Use Compute Engine with Docker, Node.js, ZeroMQ, Web Starter Kit, AngularJS, WebSocket, and D3.js

Product Details

ISBN-13:	9781449360887
Publisher:	O'Reilly Media, Incorporated
Publication date:	01/04/2015
Pages:	244
Product dimensions:	6.90(w) x 9.00(h) x 0.50(d)

About the Author

Marc manages Google's Developer Relations Engineering team in London, which helps software developers get the most out of the Google APIs and services in the EMEA region. In a previous life,Marc helped design and build communication systems at Bell Labs and Lucent Technologies. When he's not working, Marc enjoys indie music and films, writing, teaching, and chess.

Kathryn Hurley is a Developer Programs Engineer at Google for Compute Engine. In this role, she helps developers learn how to use the Compute Engine API by developing sample applications. She received an MS in Web Science from the University of San Francisco and a BS in Genetics from the University of California, Davis. Prior work experience includes research in mobile and peer-to-peer computing.

Paul Newson is a Software Engineer at Google. Currently, he is focusing on helping developers harness the power of the Google Could Platform to solve their Big Data problems. Prior to his current role in Developer Relations, Paul was helping build Google's Cloud Platform as an engineer on Google Cloud Storage. Before joining Google, Paul cofounded a tiny game technology startup, sold it to Microsoft, where he then worked on DirectX, Xbox, Xbox Live, and Forza Motorsport, before spending some time working on interesting machine learning problems in Microsoft Research. Outside of work Paul enjoys rock climbing, motorcycling, and other activities that demand complete focus.

Table of Contents

Preface vii

1 Getting Started 1

Creating a Compute Engine Project 2

Enabling Billing 3

Adding Team Members 3

Compute Engine Resources 4

Manage Compute Engine Resources 5

Summary 24

Up Next 24

2 Instances 25

Creating an Instance Using the Developers Console 26

Accessing an Instance Using the Developers Console 30

Deleting an Instance Using the Developers Console 31

Creating an Instance Using gcloud 32

Instance Attributes 34

Accessing an Instance Using gcloud 40

Deleting an Instance Using gcloud 44

Creating an Instance Programmatically 44

Creating an Instance Using a Service Account 49

Selecting an Access Mode 50

Cleaning Up 50

Summary 51

Up Next 51

3 Storage: Persistent Disk 53

Compute Engine Storage Options at a Glance 53

Persistent Disk 55

Persistent Disk Performance 56

Create a Persistent Disk Using Developers Console 56

Create a Persistent Disk Using gcloud 60

Attaching/Detaching a PD to/from a Running VM 64

Create a Persistent Disk Programmatically 67

Persistent Disk Snapshots 73

Summary 76

Up Next 76

4 Storage: Cloud Storage 77

Understanding BLOB Storage 78

Getting Started 79

Introducing gsutil 80

Using Cloud Storage from Your Code 82

Configuring Access Control 85

Understanding ACLs 88

Using Default Object ACLs 90

Understanding Object Immutability 93

Understanding Strong Consistency 94

Summary 94

Up Next 95

5 Storage: Cloud SQL and Cloud Datastore 97

Cloud SQL 97

Getting Started 98

Creating Databases and Tables 103

Running SQL Queries 106

Cloud Datastore 108

Getting Started 109

Creating and Viewing Entities via the Developers Console 110

Creating and Retrieving Entities Programmatically from a VM 112

Bring Your Own Database 115

Summary 116

Up Next 116

6 Networking 117

A Short Networking Primer 117

Network Addresses and Routing 118

Transport Control Protocol (TCP) 119

The Domain Name System (DNS) 120

Hypertext Transfer Protocol (HTTP) 122

Load Balancing 123

Firewalls 124

Default Networking 124

Configuring Firewall Rules 128

Configuring Load Balancing 131

Reserving External IP Addresses 136

Configuring Networks 137

Understanding Networking Costs 140

Understanding Routing 141

Selecting an Access Mode 142

Summary 142

Up Next 143

7 Advanced Topics 145

Startup Scripts 145

gcloud compute 145

Literal-Value Approach 146

Local-File Approach 147

Cloud-Storage Approach 148

Publicly Available Approach 149

API Approach 149

Custom Images 153

Creating a Custom Image 153

Using a Custom Image 156

Metadata 159

Metadata Server 159

Metadata Entries 160

Project Metadata 160

Instance Metadata 161

Data Formats 164

Default Versus Custom 165

Project-Level Custom Metadata 165

Instance-Level Custom Metadata 174

wait_for_change URL parameter 182

Summary 182

Up Next 182

8 A Complete Application 183

Application Concept 183

Requirements 183

Application Information Flow 186

Building Our App Step by Step 187

Step 0 Create Your Development VM and Clone the Sample App 187

Step 1 Create a Docker image for your app 188

Step 2 Build the UI framework 192

Step 3 Build the user interface 195

Step 4 Implement the master 200

Step 5 Implement the slave 207

Step 6 Add our visualization 211

Step 7 Deploy a cluster into production 216

Conclusion 220

Index 221