Delta Lake: The Definitive Guide: Modern Data Lakehouse Architectures with Data Lakes

by Denny Lee, Tristen Wentling, Scott Haines, Prashanth Babu

Overview

Ready to simplify the process of building data lakehouses and data pipelines at scale? In this practical guide, learn how Delta Lake is helping data engineers, data scientists, and data analysts overcome key data reliability challenges with modern data engineering and management techniques.

Authors Denny Lee, Tristen Wentling, Scott Haines, and Prashanth Babu (with contributions from Delta Lake maintainer R. Tyler Croy) share expert insights on all things Delta Lake—including how to run batch and streaming jobs concurrently and accelerate the usability of your data. You'll also uncover how ACID transactions bring reliability to data lakehouses at scale.

This book helps you:

• Understand key data reliability challenges and how Delta Lake solves them
• Explain the critical role of Delta transaction logs as a single source of truth
• Learn the Delta Lake ecosystem with technologies like Apache Flink, Kafka, and Trino
• Architect data lakehouses with the medallion architecture
• Optimize Delta Lake performance with features like deletion vectors and liquid clustering

Product Details

ISBN-13:	9781098151942
Publisher:	O'Reilly Media, Incorporated
Publication date:	12/10/2024
Pages:	380
Product dimensions:	7.00(w) x 9.19(h) x 0.79(d)

About the Author

Denny Lee is a Staff Developer Advocate at Databricks. He is a hands-on distributed systems and data sciences engineer with extensive experience developing internet-scale infrastructure, data platforms, and predictive analytics systems for both on-premise and cloud environments. He also has a Masters of Biomedical Informatics from Oregon Health and Sciences Universityand has architected and implemented powerful data solutions for enterprise Healthcare customers. His current technical focuses include Distributed Systems, Apache Spark, Deep Learning, Machine Learning, and Genomics.

Tristen Wentling works in machine learning, data engineering, and statistical analysis using Python, Apache Spark, and Scala. He is a machine learning advocate loves the flexibility of neural networks. Tristen holds an M.S. in Mathematics and B.S. in Applied Mathematics.

Scott Haines is a Databricks Beacon and has been working with data systems and distributed systems and architectures for over 15 years. He recently wrote a book encapsulating his journey called Modern Data Engineering with Apache Spark: A Hands-on guide for building mission-critical streaming applications. He enjoys teaching people how to simplify data systems and data-intensive services and takes to the snow in the winter to pursue his love of snowboarding.

Prashanth Babu is a Databricks Certified Developer who helps guide design and implementation of customer use cases by building out reference architectures, best practices, frameworks, MVP, and prototypes, which enables customers to succeed in turning their data into value.