Hadoop: The Definitive Guide: Storage and Analysis at Internet Scale

Hadoop: The Definitive Guide: Storage and Analysis at Internet Scale

by Tom White
Hadoop: The Definitive Guide: Storage and Analysis at Internet Scale

Hadoop: The Definitive Guide: Storage and Analysis at Internet Scale

by Tom White

eBook

$41.99  $55.99 Save 25% Current price is $41.99, Original price is $55.99. You Save 25%.

Available on Compatible NOOK Devices and the free NOOK Apps.
WANT A NOOK?  Explore Now

Related collections and offers


Overview

Get ready to unlock the power of your data. With the fourth edition of this comprehensive guide, youâ??ll learn how to build and maintain reliable, scalable, distributed systems with Apache Hadoop. This book is ideal for programmers looking to analyze datasets of any size, and for administrators who want to set up and run Hadoop clusters.

Using Hadoop 2 exclusively, author Tom White presents new chapters on YARN and several Hadoop-related projects such as Parquet, Flume, Crunch, and Spark. Youâ??ll learn about recent changes to Hadoop, and explore new case studies on Hadoopâ??s role in healthcare systems and genomics data processing.

  • Learn fundamental components such as MapReduce, HDFS, and YARN
  • Explore MapReduce in depth, including steps for developing applications with it
  • Set up and maintain a Hadoop cluster running HDFS and MapReduce on YARN
  • Learn two data formats: Avro for data serialization and Parquet for nested data
  • Use data ingestion tools such as Flume (for streaming data) and Sqoop (for bulk data transfer)
  • Understand how high-level data processing tools like Pig, Hive, Crunch, and Spark work with Hadoop
  • Learn the HBase distributed database and the ZooKeeper distributed configuration service

Product Details

ISBN-13: 9781491901700
Publisher: O'Reilly Media, Incorporated
Publication date: 03/25/2015
Sold by: Barnes & Noble
Format: eBook
Pages: 756
File size: 10 MB

About the Author

Tom White has been an Apache Hadoop committer since February 2007, and is a member of the Apache Software Foundation. He works for Cloudera, a company set up to offer Hadoop support and training. Previously he was as an independent Hadoop consultant, working with companies to set up, use, and extend Hadoop. He has written numerous articles for O'Reilly, java.net and IBM's developerWorks, and has spoken at several conferences, including at ApacheCon 2008 on Hadoop. Tom has a Bachelor's degree in Mathematics from the University of Cambridge and a Master's in Philosophy of Science from the University of Leeds, UK.

Table of Contents

Foreword;
Preface;
Administrative Notes;
What’s in This Book?;
What’s New in the Second Edition?;
What’s New in the Third Edition?;
Conventions Used in This Book;
Using Code Examples;
Safari® Books Online;
How to Contact Us;
Acknowledgments;
Chapter 1: Meet Hadoop;
1.1 Data!;
1.2 Data Storage and Analysis;
1.3 Comparison with Other Systems;
1.4 A Brief History of Hadoop;
1.5 Apache Hadoop and the Hadoop Ecosystem;
1.6 Hadoop Releases;
Chapter 2: MapReduce;
2.1 A Weather Dataset;
2.2 Analyzing the Data with Unix Tools;
2.3 Analyzing the Data with Hadoop;
2.4 Scaling Out;
2.5 Hadoop Streaming;
2.6 Hadoop Pipes;
Chapter 3: The Hadoop Distributed Filesystem;
3.1 The Design of HDFS;
3.2 HDFS Concepts;
3.3 The Command-Line Interface;
3.4 Hadoop Filesystems;
3.5 The Java Interface;
3.6 Data Flow;
3.7 Data Ingest with Flume and Sqoop;
3.8 Parallel Copying with distcp;
3.9 Hadoop Archives;
Chapter 4: Hadoop I/O;
4.1 Data Integrity;
4.2 Compression;
4.3 Serialization;
4.4 Avro;
4.5 File-Based Data Structures;
Chapter 5: Developing a MapReduce Application;
5.1 The Configuration API;
5.2 Setting Up the Development Environment;
5.3 Writing a Unit Test with MRUnit;
5.4 Running Locally on Test Data;
5.5 Running on a Cluster;
5.6 Tuning a Job;
5.7 MapReduce Workflows;
Chapter 6: How MapReduce Works;
6.1 Anatomy of a MapReduce Job Run;
6.2 Failures;
6.3 Job Scheduling;
6.4 Shuffle and Sort;
6.5 Task Execution;
Chapter 7: MapReduce Types and Formats;
7.1 MapReduce Types;
7.2 Input Formats;
7.3 Output Formats;
Chapter 8: MapReduce Features;
8.1 Counters;
8.2 Sorting;
8.3 Joins;
8.4 Side Data Distribution;
8.5 MapReduce Library Classes;
Chapter 9: Setting Up a Hadoop Cluster;
9.1 Cluster Specification;
9.2 Cluster Setup and Installation;
9.3 SSH Configuration;
9.4 Hadoop Configuration;
9.5 YARN Configuration;
9.6 Security;
9.7 Benchmarking a Hadoop Cluster;
9.8 Hadoop in the Cloud;
Chapter 10: Administering Hadoop;
10.1 HDFS;
10.2 Monitoring;
10.3 Maintenance;
Chapter 11: Pig;
11.1 Installing and Running Pig;
11.2 An Example;
11.3 Comparison with Databases;
11.4 Pig Latin;
11.5 User-Defined Functions;
11.6 Data Processing Operators;
11.7 Pig in Practice;
Chapter 12: Hive;
12.1 Installing Hive;
12.2 An Example;
12.3 Running Hive;
12.4 Comparison with Traditional Databases;
12.5 HiveQL;
12.6 Tables;
12.7 Querying Data;
12.8 User-Defined Functions;
Chapter 13: HBase;
13.1 HBasics;
13.2 Concepts;
13.3 Installation;
13.4 Clients;
13.5 Example;
13.6 HBase Versus RDBMS;
13.7 Praxis;
Chapter 14: ZooKeeper;
14.1 Installing and Running ZooKeeper;
14.2 An Example;
14.3 The ZooKeeper Service;
14.4 Building Applications with ZooKeeper;
14.5 ZooKeeper in Production;
Chapter 15: Sqoop;
15.1 Getting Sqoop;
15.2 Sqoop Connectors;
15.3 A Sample Import;
15.4 Generated Code;
15.5 Imports: A Deeper Look;
15.6 Working with Imported Data;
15.7 Importing Large Objects;
15.8 Performing an Export;
15.9 Exports: A Deeper Look;
Chapter 16: Case Studies;
16.1 Hadoop Usage at Last.fm;
16.2 Hadoop and Hive at Facebook;
16.3 Nutch Search Engine;
16.4 Log Processing at Rackspace;
16.5 Cascading;
16.6 TeraByte Sort on Apache Hadoop;
16.7 Using Pig and Wukong to Explore Billion-edge Network Graphs;
Installing Apache Hadoop;
Prerequisites;
Installation;
Configuration;
Cloudera’s Distribution Including Apache Hadoop;
Preparing the NCDC Weather Data;
Colophon;
From the B&N Reads Blog

Customer Reviews