ZooKeeper: Distributed Process Coordination

by Flavio Junqueira, Benjamin Reed

Overview

Building distributed applications is difficult enough without having to coordinate the actions that make them work. This practical guide shows how Apache ZooKeeper helps you manage distributed systems, so you can focus mainly on application logic. Even with ZooKeeper, implementing coordination tasks is not trivial, but this book provides good practices to give you a head start, and points out caveats that developers and administrators alike need to watch for along the way.

In three separate sections, ZooKeeper contributors Flavio Junqueira and Benjamin Reed introduce the principles of distributed systems, provide ZooKeeper programming techniques, and include the information you need to administer this service.

• Learn how ZooKeeper solves common coordination tasks
• Explore the ZooKeeper API’s Java and C implementations and how they differ
• Use methods to track and react to ZooKeeper state changes
• Handle failures of the network, application processes, and ZooKeeper itself
• Learn about ZooKeeper’s trickier aspects dealing with concurrency, ordering, and configuration
• Use the Curator high-level interface for connection management
• Become familiar with ZooKeeper internals and administration tools

Product Details

ISBN-13:	9781449361303
Publisher:	O'Reilly Media, Incorporated
Publication date:	12/13/2013
Pages:	243
Product dimensions:	7.00(w) x 9.00(h) x 0.60(d)

About the Author

Flavio Junqueira is a member of the research staff of Microsoft Research in Cambridge, UK. He holds a PhD degree in Computer Science from the University of California, San Diego. He is interested in various aspects of distributed systems, including distributed algorithms, concurrency, and scalability. He is an active contributor of Apache projects, such as Apache ZooKeeper (PMC chair and committer) and Apache BookKeeper (committer). When he is idle, he sleeps.

Benjamin Reed is a Software Engineer at Facebook working on all things small. His previous positions include Principal Research Scientist at Yahoo! Research (working on all things big) and Research Staff Member (working on the big and the small) at IBM Almaden Research. The University of California, Santa Cruz granted him a PhD in computer science. He has worked in the areas of distributed computing, big data processing, distributed storage, systems management, and embedded frameworks. He participated in various open source projects such as Hadoop and Linux. He helped start the Pig, Zookeeper, and BookKeeper projects hosted by the Apache Software Foundation.

Table of Contents

Preface; Audience; Contents of This Book; Conventions Used in this Book; Using Code Examples; Safari® Books Online; How to Contact Us; Acknowledgments; ZooKeeper Concepts and Basics; Chapter 1: Introduction; 1.1 The ZooKeeper Mission; 1.2 Example: Master-Worker Application; 1.3 Why Is Distributed Coordination Hard?; 1.4 ZooKeeper Is a Success, with Caveats; Chapter 2: Getting to Grips with ZooKeeper; 2.1 ZooKeeper Basics; 2.2 ZooKeeper Architecture; 2.3 Getting Started with ZooKeeper; 2.4 Implementation of a Master-Worker Example; 2.5 Takeaway Messages; Programming with ZooKeeper; Chapter 3: Getting Started with the ZooKeeper API; 3.1 Setting the ZooKeeper CLASSPATH; 3.2 Creating a ZooKeeper Session; 3.3 Getting Mastership; 3.4 Registering Workers; 3.5 Queuing Tasks; 3.6 The Admin Client; 3.7 Takeaway Messages; Chapter 4: Dealing with State Change; 4.1 One-Time Triggers; 4.2 Getting More Concrete: How to Set Watches; 4.3 A Common Pattern; 4.4 The Master-Worker Example; 4.5 An Alternative Way: Multiop; 4.6 Watches as a Replacement for Explicit Cache Management; 4.7 Ordering Guarantees; 4.8 The Herd Effect and the Scalability of Watches; 4.9 Takeaway Messages; Chapter 5: Dealing with Failure; 5.1 Recoverable Failures; 5.2 Unrecoverable Failures; 5.3 Leader Election and External Resources; 5.4 Takeaway Messages; Chapter 6: ZooKeeper Caveat Emptor; 6.1 Using ACLs; 6.2 Session Recovery; 6.3 Version Is Reset When Znode Is Re-Created; 6.4 The sync Call; 6.5 Ordering Guarantees; 6.6 Data and Child Limits; 6.7 Embedding the ZooKeeper Server; 6.8 Takeaway Messages; Chapter 7: The C Client; 7.1 Setting Up the Development Environment; 7.2 Starting a Session; 7.3 Bootstrapping the Master; 7.4 Taking Leadership; 7.5 Assigning Tasks; 7.6 Single-Threaded versus Multithreaded Clients; 7.7 Takeaway Messages; Chapter 8: Curator: A High-Level API for ZooKeeper; 8.1 The Curator Client; 8.2 Fluent API; 8.3 Listeners; 8.4 State Changes in Curator; 8.5 A Couple of Edge Cases; 8.6 Recipes; 8.7 Takeaway Messages; Administering ZooKeeper; Chapter 9: ZooKeeper Internals; 9.1 Requests, Transactions, and Identifiers; 9.2 Leader Elections; 9.3 Zab: Broadcasting State Updates; 9.4 Observers; 9.5 The Skeleton of a Server; 9.6 Local Storage; 9.7 Servers and Sessions; 9.8 Servers and Watches; 9.9 Clients; 9.10 Serialization; 9.11 Takeaway Messages; Chapter 10: Running ZooKeeper; 10.1 Configuring a ZooKeeper Server; 10.2 Configuring a ZooKeeper Ensemble; 10.3 Reconfiguration; 10.4 Quotas; 10.5 Multitenancy; 10.6 File System Layout and Formats; 10.7 Four-Letter Words; 10.8 Monitoring with JMX; 10.9 Tools; 10.10 Takeaway Messages; Index; Colophon;