Mac OS X Internals: A Systems Approach / Edition 1

Hardcover (Print)
Buy New
Buy New from BN.com
$76.78
Used and New from Other Sellers
Used and New from Other Sellers
from $21.49
Usually ships in 1-2 business days
(Save 77%)
Other sellers (Hardcover)
  • All (14) from $21.49   
  • New (7) from $64.65   
  • Used (7) from $21.49   

Overview

Mac OS X was released in March 2001, but many components, such as Mach and BSD, are considerably older. Understanding the design, implementation, and workings of Mac OS X requires examination of several technologies that differ in their age, origins, philosophies, and roles.

Mac OS X Internals: A Systems Approach is the first book that dissects the internals of the system, presenting a detailed picture that grows incrementally as you read. For example, you will learn the roles of the firmware, the bootloader, the Mach and BSD kernel components (including the process, virtual memory, IPC, and file system layers), the object-oriented I/O Kit driver framework, user libraries, and other core pieces of software. You will learn how these pieces connect and work internally, where they originated, and how they evolved. The book also covers several key areas of the Intel-based Macintosh computers.

A solid understanding of system internals is immensely useful in design, development, and debugging for programmers of various skill levels. System programmers can use the book as a reference and to construct a better picture of how the core system works. Application programmers can gain a deeper understanding of how their applications interact with the system. System administrators and power users can use the book to harness the power of the rich environment offered by Mac OS X. Finally, members of the Windows, Linux, BSD, and other Unix communities will find the book valuable in comparing and contrasting Mac OS X with their respective systems.

Mac OS X Internals focuses on the technical aspects of OS X and is so full of extremely useful information and programming examples that it will definitely become a mandatory tool for every Mac OS X programmer.

Read More Show Less

Product Details

  • ISBN-13: 9780321278548
  • Publisher: Addison-Wesley
  • Publication date: 7/3/2006
  • Edition description: New Edition
  • Edition number: 1
  • Pages: 1641
  • Sales rank: 987,469
  • Product dimensions: 7.04 (w) x 9.88 (h) x 2.27 (d)

Meet the Author

Amit Singh is an operating systems researcher, programmer, and author. He manages the Macintosh engineering team at Google. Previously, Amit has worked on operating systems at IBM Research, Bell Laboratories, and a Silicon Valley startup doing cutting-edge work in the area of virtualization. He also created and maintains osxbook.com and kernelthread.com. Amit often writes and releases open source software, such as MacFUSE, a Mac OS X implementation of the FUSE (File System in USEr Space) mechanism.
Read More Show Less

Table of Contents

Preface xxv

Acknowledgments xxxi

About the Author xxxiii

Chapter 1 Origins of Mac OS X 1

1.1 Apple's Quest for the Operating System 2

1.2 The NeXT Chapter 9

1.3 The Mach Factor 15

1.4 Strategies 24

1.5 Toward Mac OS X 31

Chapter 2 An Overview of Mac OS X 43

2.1 Firmware 46

2.2 Bootloader 46

2.3 Darwin 46

2.4 The xnu Kernel 48

2.5 A User-Space View of the File System 57

2.6 The Runtime Architecture 61

2.7 The C Library 74

2.8 Bundles and Frameworks 76

2.9 Core Services 89

2.10 Application Services 90

2.11 Application Environments 101

2.12 User Interface 117

2.13 Programming 121

2.14 Security 131

2.15 Mac OS X Server 145

2.16 Networking 153

Chapter 3 Inside an Apple 155

3.1 The Power Mac G5 156

3.2 The G5: Lineage and Roadmap 166

3.3 The PowerPC 970FX 174

3.4 Software Conventions 224

3.5 Examples 240

Chapter 4 The Firmware and the Bootloader 263

4.1 Introduction 263

4.2 A Whole New World 266

4.3 Power-On Reset 271

4.4 Open Firmware 272

4.5 Forth 279

4.6 The Device Tree 289

4.7 Open Firmware Interfaces 298

4.8 Programming Examples 300

4.9 Firmware Boot Sequence 324

4.10 BootX 328

4.11 Alternate Booting Scenarios 340

4.12 Firmware Security 349

4.13 Launching the Kernel 352

4.14 The BootCache Optimization 353

4.15 The Boot-Time Kernel Arguments 355

4.16 The Extensible Firmware Interface 362

Chapter 5 Kernel and User-Level Startup 381

5.1 Arranging for the Kernel to Execute 382

5.2 Low-Level Processor Initialization 388

5.3 High-Level Processor Initialization 405

5.4 Mach Subsystem Initialization 421

5.5 The First Thread 432

5.6 I/O Kit Initialization 435

5.7 BSD Initialization 443

5.8 Launching the First User-Space Program 469

5.9 Slave Processors 470

5.10 User-Level Startup 472

Chapter 6 The xnu Kernel 501

6.1 xnu Source 501

6.2 Mach 510

6.3 A Flavor of the Mach APIs 519

6.4 Entering the Kernel 529

6.5 Exception Processing 543

6.6 System Call Processing 553

6.7 System Call Categories 557

6.8 Kernel Support for Debugging, Diagnostics, and Tracing 601

6.9 Virtual Machine Monitor 659

6.10 Compiling the Kernel 676

Chapter 7 Processes 683

7.1 Processes: From Early UNIC to Mac OS X 684

7.2 Mach Abstractions, Data Structures, and APIs 687

7.3 Many Threads of a New System 726

7.4 Scheduling 774

7.5 The execve() System Call 812

7.6 Launching Applications 828

Chapter 8 Memory 835

8.1 Looking Back 835

8.2 An Overview of Mac OS X Memory Management 838

8.3 Mac VM 846

8.4 Resident Memory 868

8.5 Virtual Memory Initialization during Bootstrap 877

8.6 The Mach VM User-Space Interface 878

8.7 Using the Mach VM Interfaces 893

8.8 Kernel and User Address Space Layouts 907

8.9 Universal Page Lists (UPLs) 912

8.10 Unified Buffer Cache (UBC) 913

8.11 The Dynamic Pager Program 918

8.12 The Update Daemon 921

8.13 System Shared Memory 922

8.14 Task Working Set Detection and Maintenance 942

8.15 Memory Allocation in User Space 948

8.16 Memory Allocation in the Kernel 980

8.17 Memory-Mapped Files 1001

8.18 64-bit Computing 1005

Chapter 9 Interprocess Communication 1021

9.1 Introduction 1021

9.2 Mach IPC: An Overview 1025

9.3 Mach IPC: The Mac OS X Implementation 1041

9.4 Name and Bootstrap Servers 1060

9.5 Using Mach IPC 1080

9.6 MIG 1094

9.7 Mach Exceptions 1112

9.8 Signals 1129

9.9 Pipes 1145

9.10 Named Pipes (Fifos) 1147

9.11 File Descriptor Passing 1148

9.12 XSI IPC 1155

9.13 POSIX IPC 1156

9.14 Distributed Objects 1164

9.15 Apple Events 1172

9.16 Notifications 1181

9.17 Core Foundation IPC 1197

9.18 Synchronization 1210

Chapter 10 Extending the Kernel 1233

10.1 A Driver down the Memory Lane 1233

10.2 The I/O Kit 1235

10.3 DART 1257

10.4 Dynamically Extending the Kernel 1259

10.5 Communicating with the Kernel 1269

10.6 Creating Kernel Extensions 1271

10.7 A Programming Tour of the I/O Kit's Functionality 1288

10.8 Debugging 1321

Chapter 11 File Systems 1345

11.1 Disks and Partitions 1345

11.2 Disk Arbitration 1353

11.3 The Implementation of Disk Devices 1362

11.4 Disk Images 1366

11.5 Files and File Descriptors 1374

11.6 The VFS Layer 1376

11.7 File System Types 1386

11.8 Spotlight 1409

11.9 Access Control Lists 1441

11.10 The Kauth Authorization Subsystem 1445

Chapter 12 The HFS Plus File System 1471

12.1 Analysis Tools 1474

12.2 Fundamental Concepts 1477

12.3 The Structure of an HFS+ Volume 1491

12.4 Reserved Areas 1493

12.5 The Volume Header 1493

12.6 The HFS Wrapper 1501

12.7 Special Files 1505

12.8 Examining HFS+ Features 1531

12.9 Optimizations 1558

12.10 Miscellaneous Features 1570

12.11 Comparing Mac OS X File Systems

12.12 Comparing HFS+ and NTFS 1582

Appendix A Mac OS X on x86-Based Macintosh Computers 1587

A.1 Hardware Differences 1587

A.2 Firmware and Booting 1589

A.3 Partitioning 1590

A.4 Universal Binaries 1591

A.5 Rosetta 1592

A.6 Byte Ordering 1594

A.7 Miscellaneous Changes 1594

Index 1599

Read More Show Less

Preface

Although Mac OS X is a relatively new operating system, its lineage is rather colorful, and the history of most of its components is much older. Mac OS X, and Apple in general, has attracted a lot of attention in recent years. Given Apple's "cult-like" status, and the unique cultural and technical composition of Mac OS X, it is not surprising that people with vastly different backgrounds and interests—both users and non-users of Mac OS X—are curious about the system.

After working on, programming on, and extending several operating systems over the years, I was introduced to Mac OS X on April 1, 2003. I was soon curious about the structure of the system. Despite the availability of several good books on Mac OS X, I was dismayed to find that I could not learn the details of Mac OS X internals from a book—no such book existed. There are books describing how to perform various tasks on Mac OS X; how to configure, customize, and tweak the system; and how certain user-visible aspects differ between Mac OS X and Windows. There are also books covering specific Mac OS X programming topics, such as the Cocoa and Carbon APIs. Other books make life easier for a Unix user migrating to Mac OS X—such books typically discuss the command-line interface to the operating system. Although these books play important roles in making Mac OS X accessible, the core architecture and implementation of Mac OS X and its components have remained mysteries. To make matters worse, besides the lack of information, it is common to find misinformation on the composition of Mac OS X. Consequently, the system is often misunderstood because of long-standing myths and stereotypes, or is perceived as a black box.

The purpose of this book is to deconstruct and demystify Mac OS X by describing it from a system-design perspective. It takes an implementation-oriented approach to understanding the system. Consider the example of interprocess communication (IPC). Mac OS X has numerous user-level and kernel-level IPC mechanisms, some of which are well known and documented. Rather than just showing how to use these mechanisms, the book explains the design and implementation of the most fundamental mechanism (Mach IPC), and then discusses how the other mechanisms are layered on top of one another. My goal is not to teach you how to do something specific—my goal is to provide you with enough knowledge and examples so that after reading the book, depending on your interests and background, you can build upon your newly gained knowledge and do what you choose.

Along with text, the book uses detailed illustrations, function call graphs, annotated code excerpts, and programming examples to present a detailed examination of Mac OS X. To keep the subject matter interesting and accessible—even to the casual reader—the book contains relevant trivia, digressions, and other tidbits.

Who This Book Is For

I hope the book will be useful to anyone curious about the composition and working of Mac OS X.

Application programmers can gain a deeper understanding of how their applications interact with the system. System programmers can use the book as a reference and to construct a better picture of how the core system works. In my experience as a programmer, a solid understanding of system internals is immensely useful in design, development, and debugging. For example, you know what the system is capable of, what is feasible and what is not, what the "best" option is in a given situation, and what the plausible reasons are for certain program behavior. This book's primary goal is to build a strong foundation for anyone who programs on Mac OS X.

Mac OS X users can read the book to better understand how the system is designed and how it comes together. System administrators and technical support staff will also find value in the book.

Besides those who use Mac OS X, the intended audience includes members of other technical communities, such as the BSD, Linux, and Windows communities. Given that many internal aspects of Mac OS X are radically different from these systems (for example, how the Mach kernel is used), the book will help such readers broaden their knowledge, and will assist them in comparing and contrasting Mac OS X with other operating systems.

The book will also be useful in an advanced operating systems course, particularly if you wish to do a case study on Mac OS X. The book is not suitable, however, as an introductory text. Most of the material is not presented at an introductory level, although I introduce many of the advanced topics with at least some background information.

The Book's Structure

Modern operating systems have become so large and complex that it is impossible to reasonably describe an entire system in a book. This book is somewhat ambitious in that it attempts to cover Mac OS X in substantial breadth and depth. The most important contributors to the book's depth are the carefully selected programming examples. The book is organized into twelve chapters. Although much of the book's content is rather technical, each chapter has sections that should be accessible to non-programmers.

Chapter 1, "Origins of Mac OS X," describes the technical history of Mac OS X and the systems it derives from. An unabridged version of Chapter 1, which covers all of Apple's past and present operating systems, is available on this book's accompanying web site.

Chapter 2, "An Overview of Mac OS X," is an eclectic tour of Mac OS X and its important features. It contains brief overviews of various layers that constitute the system.

Chapter 3, "Inside an Apple," describes the PowerPC architecture, using the PowerPC 970 ("G5") processor line as a specific example. It also discusses the PowerPC assembly language and calling conventions.

Chapter 4, "The Firmware and the Bootloader," describes both Open Firmware and the Extensible Firmware Interface (EFI), along with their respective boot-loaders. It discusses the roles the firmware and the bootloader play in the system's operation, usage scenarios, and events that occur during early bootstrapping.

Chapter 5, "Kernel and User-Level Startup," describes the sequence of events—including initializations of kernel subsystems—from where the kernel starts executing to the point where the first user-space program (launchd) is run by the kernel. The discussion includes launchd's function and implementation.

Chapter 6, "The xnu Kernel," describes the core kernel architecture of Mac OS X. The discussion includes system call families and their implementation, low-level tracing and debugging mechanisms, and special features such as the virtual machine monitor in the PowerPC version of the kernel.

Chapter 7, "Processes," describes abstractions such as tasks, threads, and processes, the various forms in which they exist in Mac OS X subsystems, and processor scheduling. The discussion includes using various kernel-level and user-level interfaces for manipulating the aforementioned abstractions.

Chapter 8, "Memory," describes the Mac OS X memory subsystem's architecture, including discussions of the Mach virtual memory architecture, paging, the unified buffer cache, the working-set detection mechanism, kernel-level and user-level memory allocators, and support for 64-bit addressing.

Chapter 9, "Interprocess Communication," describes various IPC and synchronization mechanisms available in Mac OS X. In particular, it discusses the implementation and usage of Mach IPC.

Chapter 10, "Extending the Kernel," describes the I/O Kit, the object-oriented driver subsystem in Mac OS X.

Chapter 11, "File Systems," describes the overall file system layer in Mac OS X, including brief discussions of each file system type. The discussion also covers partitioning schemes, disk management, and the Spotlight search technology.

Chapter 12, "The HFS Plus File System," describes the internals of the HFS Plus file system. The discussion is aided by the use of a custom file system debugger written for this chapter.

Appendix A, "Mac OS X on x86-Based Macintosh Computers," highlights the key differences between the x86-based and PowerPC-based versions of Mac OS X. Besides this appendix, the book covers the details of several key x86-specific topics, such as EFI, GUID-based partitioning, and Universal Binaries. Most of Mac OS X is architecture-independent, and consequently, the majority of the book is architecture-independent.

Given the book's length, I chose to exclude several topics that are well covered in other texts. The TCP/IP stack is an example—there is no "networking" chapter in the book since the Mac OS X TCP/IP stack is largely a derivative of the FreeBSD stack, which is already well documented. In general, information that is generic across Unix variants and can be found in standard texts is not included in this book.

How to Read This Book

Since the first two chapters provide the background and overall picture of Mac OS X, respectively, I recommend that you read these chapters first. The subsequent chapters are best read sequentially, although, depending on your interests and familiarity with the topics, you can skip certain sections (and perhaps even chapters) and still gain value from the book.

It will help if you have some familiarity with operating system concepts and have used a Unix operating system.

Given that the book has a large number of C programs and program excerpts, you should have some programming experience, and in particular, knowledge of the C programming language. I sometimes use code not only to demonstrate the working of a concept, but also to describe the concept. I realize that it is usually considered "difficult" to "read" code, and authors often expect that many readers will simply skip the code. My belief is that reading the code (as opposed to only running it) in this book will be particularly helpful to programmers.

Despite the book's technical nature, several parts of the book can be read casually by both programmers and non-programmers. I hope that as a reference on Mac OS X internals, the book and its examples will be useful to its readers for a long time to come.

How to Use the Examples

I have included a number of self-contained examples in the book. Many of these examples are non-trivial in that they do something that is both useful and interesting. I hope that these examples will serve as food for thought and building blocks for other projects. Almost all of the examples in the book are shown along with the command lines used to compile and run them.

The examples were tested on both PowerPC-based and x86-based Macintosh computers where applicable. It is interesting to note that in the cases where the code is PowerPC-only, say, in a PowerPC assembly language example, it can usually be both compiled and run on an x86-based Macintosh—such code will run under the Rosetta binary translation software. However, a small number of examples in the book require a PowerPC Macintosh—they will not run under Rosetta.

Related Material

Technology moves so fast these days that it is almost impossible to publish a fully up-to-date book. Thankfully, Internet access allows the author and publisher to make various materials available to readers after the book is published. The most useful resource for this book is its accompanying web site, http://www.osxbook.com, which includes the following:

  • Errata and updates
  • Source code from the book
  • The book's blog, with news and announcements about the availability of new material
  • A set of discussion forums where topics related to the book (and to Mac OS X in general) can be discussed
  • A bonus content section, which contains additional articles, presentations, binaries, and source code relevant to the book
  • Sample content from the book, including a detailed table of contents
Read More Show Less

Customer Reviews

Be the first to write a review
( 0 )
Rating Distribution

5 Star

(0)

4 Star

(0)

3 Star

(0)

2 Star

(0)

1 Star

(0)

Your Rating:

Your Name: Create a Pen Name or

Barnes & Noble.com Review Rules

Our reader reviews allow you to share your comments on titles you liked, or didn't, with others. By submitting an online review, you are representing to Barnes & Noble.com that all information contained in your review is original and accurate in all respects, and that the submission of such content by you and the posting of such content by Barnes & Noble.com does not and will not violate the rights of any third party. Please follow the rules below to help ensure that your review can be posted.

Reviews by Our Customers Under the Age of 13

We highly value and respect everyone's opinion concerning the titles we offer. However, we cannot allow persons under the age of 13 to have accounts at BN.com or to post customer reviews. Please see our Terms of Use for more details.

What to exclude from your review:

Please do not write about reviews, commentary, or information posted on the product page. If you see any errors in the information on the product page, please send us an email.

Reviews should not contain any of the following:

  • - HTML tags, profanity, obscenities, vulgarities, or comments that defame anyone
  • - Time-sensitive information such as tour dates, signings, lectures, etc.
  • - Single-word reviews. Other people will read your review to discover why you liked or didn't like the title. Be descriptive.
  • - Comments focusing on the author or that may ruin the ending for others
  • - Phone numbers, addresses, URLs
  • - Pricing and availability information or alternative ordering information
  • - Advertisements or commercial solicitation

Reminder:

  • - By submitting a review, you grant to Barnes & Noble.com and its sublicensees the royalty-free, perpetual, irrevocable right and license to use the review in accordance with the Barnes & Noble.com Terms of Use.
  • - Barnes & Noble.com reserves the right not to post any review -- particularly those that do not follow the terms and conditions of these Rules. Barnes & Noble.com also reserves the right to remove any review at any time without notice.
  • - See Terms of Use for other conditions and disclaimers.
Search for Products You'd Like to Recommend

Recommend other products that relate to your review. Just search for them below and share!

Create a Pen Name

Your Pen Name is your unique identity on BN.com. It will appear on the reviews you write and other website activities. Your Pen Name cannot be edited, changed or deleted once submitted.

 
Your Pen Name can be any combination of alphanumeric characters (plus - and _), and must be at least two characters long.

Continue Anonymously

    If you find inappropriate content, please report it to Barnes & Noble
    Why is this product inappropriate?
    Comments (optional)