ISBN-10:
0321968972
ISBN-13:
9780321968975
Pub. Date:
09/19/2014
Publisher:
Addison-Wesley
The Design and Implementation of the FreeBSD Operating System / Edition 2

The Design and Implementation of the FreeBSD Operating System / Edition 2

Current price is , Original price is $69.99. You

Temporarily Out of Stock Online

Please check back later for updated availability.

Product Details

ISBN-13: 9780321968975
Publisher: Addison-Wesley
Publication date: 09/19/2014
Edition description: New Edition
Pages: 928
Sales rank: 582,090
Product dimensions: 6.20(w) x 9.40(h) x 1.40(d)

About the Author

Marshall Kirk McKusick writes books and articles, consults, and teaches classes on UNIX- and BSD-related subjects. While at the University of California at Berkeley, he implemented the 4.2BSD fast filesystem and was the Research Computer Scientist at the Berkeley Computer Systems Research Group (CSRG), overseeing the development and release of 4.3BSD and 4.4BSD. His particular areas of interest are the virtual-memory system and the filesystem. He earned his undergraduate degree in electrical engineering from Cornell University and did his graduate work at the University of California at Berkeley, where he received master’s degrees in computer science and business administration, and a doctoral degree in computer science. He has twice been president of the board of the Usenix Association, is currently a member of the FreeBSD Foundation Board of Directors, a member of the editorial board of ACM’s Queue magazine, a senior member of the IEEE, and a member of the Usenix Association, ACM, and AAAS. In his spare time, he enjoys swimming, scuba diving, and wine collecting. The wine is stored in a specially constructed wine cellar (accessible from the Web at http://www.McKusick.com/cgi-bin/readhouse) in the basement of the house that he shares with Eric Allman, his partner of 35-and-some-odd years and husband since 2013.

George V. Neville-Neil hacks, writes, teaches, and consults in the areas of Security, Networking, and Operating Systems. Other areas of interest include embedded and real-time systems, network time protocols, and code spelunking. In 2007, he helped start the AsiaBSDCon series of conferences in Tokyo, Japan, and has served on the program committee every year since then. He is a member of the FreeBSD Foundation Board of Directors, and was a member of the FreeBSD Core Team for 4 years. Contributing broadly to open source, he is the lead developer on the Precision Time Protocol project (http://ptpd.sf.net) and the developer of the Packet Construction Set (http://pcs.sf.net). Since 2004, he has written a monthly column, ‘‘Kode Vicious,’’ that appears both in ACM’s Queue and Communications of the ACM. He serves on the editorial board of ACM’s Queue magazine, is vice-chair of ACM’s Practitioner Board, and is a member of the Usenix Association, ACM, IEEE, and AAAS. He earned his bachelor’s degree in computer science at Northeastern University in Boston, Massachusetts. He is an avid bicyclist, hiker, and traveler who has lived in Amsterdam, The Netherlands, and Tokyo, Japan. He is currently based in Brooklyn, New York, where he lives with his husband, Kaz Senju.

Robert N.M. Watson is a University Lecturer in Systems, Security, and Architecture in the Security Research Group at the University of Cambridge Computer Laboratory. He supervises doctoral students and postdoctoral researchers in cross-layer research projects spanning computer architecture, compilers, program analysis, program transformation, operating systems, networking, and security. Dr. Watson is a member of the FreeBSD Foundation Board of Directors, was a member of the FreeBSD Core Team for 10 years, and has been a FreeBSD committer for 15 years. His open-source contributions include work on FreeBSD networking, security, and multiprocessing. Having grown up in Washington, D. C., he earned his undergraduate degree in Logic and Computation, with a double major in Computer Science, at Carnegie Mellon University in Pittsburgh, Pennsylvania, and then worked at a series of industrial research labs investigating computer security. He earned his doctoral degree at the University of Cambridge, where his graduate research was in extensible operating system access control. Dr. Watson and his wife Dr. Leigh Denault have lived in Cambridge, England, for 10 years.

Read an Excerpt

This book follows the earlier authoritative and full-length descriptions of the design and implementation of the 4.3BSD and 4.4BSD versions of the UNIX system developed at the University of California at Berkeley. Since the final Berkeley release in 1994, several groups have continued development of BSD. This book details FreeBSD, the system with the largest set of developers and the most widely distributed releases. Although the FreeBSD distribution includes nearly 1000 utility programs in its base system and nearly 10,000 optional utilities in its ports collection, this book concentrates almost exclusively on the kernel. UNIX-like Systems

UNIX-like systems include the traditional vendor systems such as Solaris and HP-UX; the Linux-based distributions such as Red Hat, Debian, Suse, and Slackware; and the BSD-based distributions such as FreeBSD, NetBSD, OpenBSD, and Darwin. They run on computers ranging from laptops to the largest supercomputers. They are the operating system of choice for most multiprocessor, graphics, and vector-processing systems, and are widely used for the original purpose of timesharing. The most common platform for providing network services (from FTP to WWW) on the Internet, they are collectively the most portable operating system ever dev eloped. This portability is due partly to their implementation language, C Kernighan & Ritchie, 1989 (which is itself a widely ported language), and partly to the elegant design of the system.

Since its inception in 1969 Ritchie & Thompson, 1978, the UNIX system has developed in several divergent and rejoining streams. The original developers continued to advance the state of the art with their Ninthand Tenth Edition UNIX inside AT&T Bell Laboratories, and then their Plan 9 successor to UNIX. Meanwhile, AT&T licensed UNIX System V as a product before selling it to Novell. Novell passed the UNIX trademark to X/OPEN and sold the source code and distribution rights to Santa Cruz Operation (SCO). Both System V and Ninth Edition UNIX were strongly influenced by the Berkeley Software Distributions produced by the Computer Systems Research Group (CSRG) of the University of California at Berkeley. The Linux operating system, although developed independently of the other UNIX variants, implements the UNIX interface. Thus, applications developed to run on other UNIX-based platforms can be easily ported to run on Linux. Berkeley Software Distributions

The distributions from Berkeley were the first UNIX-based systems to introduce many important features including the following:

  • Demand-paged virtual-memory support
  • Automatic configuration of the hardware and I/O system
  • A fast and recoverable filesystem
  • The socket-based interprocess-communication (IPC) primitives
  • The reference implementation of TCP/IP

The Berkeley releases found their way into the UNIX systems of many vendors and were used internally by the development groups of many other vendors. The implementation of the TCP/IP networking protocol suite in 4.2BSD and 4.3BSD, and the availability of those systems, played a key role in making the TCP/IP networking protocol suite a world standard. Even the non-UNIX vendors such as Microsoft have adopted the Berkeley socket design in their Winsock IPC interface.

The BSD releases have also been a strong influence on the POSIX (IEEE Std 1003.1) operating-system interface standard, and on related standards. Several features—such as reliable signals, job control, multiple access groups per process, and the routines for directory operations—have been adapted from BSD for POSIX.

Early BSD releases contained licensed UNIX code, thus requiring recipients to have an AT&T source license to be able to obtain and use BSD. In 1988, Berkeley separated its distribution into AT&T licensed and freely redistributable code. The freely redistributable code was licensed separately and could be obtained, used, and redistributed by anyone. The final freely redistributable 4.4BSD-Lite2 release from Berkeley in 1994 contained nearly the entire kernel and all the important libraries and utilities.

Two groups, NetBSD and FreeBSD, sprang up in 1993 to begin supporting and distributing systems built from the freely redistributable releases being done by Berkeley. The NetBSD group emphasized portability and the minimalist approach, porting the systems to nearly forty platforms and pushing to keep the system lean to aid embedded applications. The FreeBSD group emphasized maximal support for the PC architecture and pushed to ease installation for, and market their system to, as wide an audience as possible. In 1995, the OpenBSD group split from the NetBSD group to develop a distribution that emphasized security. Over the years there has been a healthy competition among the BSD distributions, with many ideas and much code flowing between them. Material Covered in this Book

This book is about the internal structure of the FreeBSD 5.2 kernel and about the concepts, data structures, and algorithms used in implementing FreeBSD’s system facilities. Its level of detail is similar to that of Bach’s book about UNIX System V Bach, 1986; however, this text focuses on the facilities, data structures, and algorithms used in the FreeBSD variant of the UNIX operating system. The book covers FreeBSD from the system-call level down—from the interface to the kernel to the hardware itself. The kernel includes system facilities, such as process management, virtual memory, the I/O system, filesystems, the socket IPC mechanism, and network protocol implementations. Material above the system-call level—such as libraries, shells, commands, programming languages, and other user interfaces—is excluded, except for some material related to the terminal interface and to system startup. Following the organization first established by Organick’s book about Multics Organick, 1975, this book is an in-depth study of a contemporary operating system.

Where particular hardware is relevant, the book refers to the Intel Personal Computer (PC) architecture. Because FreeBSD has emphasized development on the PC, that is the architecture with the most complete support, so it provides a convenient point of reference. Use by Computer Professionals

FreeBSD is widely used to support the core infrastructure of many companies worldwide. Because it can be built with a small footprint, it is also seeing increased use in embedded applications. The licensing terms of FreeBSD do not require the distribution of changes and enhancements to the system. The licensing terms of Linux require that all changes and enhancements to the kernel be made available in source form at minimal cost. Thus, companies that need to control the distribution of their intellectual property build their products using FreeBSD.

This book is of direct use to the professionals who work with FreeBSD systems. Individuals involved in technical and sales support can learn the capabilities and limitations of the system; applications developers can learn how to effectively and efficiently interface to the system; system administrators without direct experience with the FreeBSD kernel can learn how to maintain, tune, and configure the system; and systems programmers can learn how to extend, enhance, and interface to the system.

Readers who will benefit from this book include operating-system implementors, system programmers, UNIX application developers, administrators, and curious users. The book can be read as a companion to the source code of the system, falling as it does between the manual pages and the code in detail of treatment. But this book is neither exclusively a UNIX programming manual nor a user tutorial (for a tutorial, see Libes & Ressler 1988). Familiarity with the use of some version of the UNIX system (see, for example, Stevens 1992) and with the C programming language (see, for example, Kernighan & Ritchie 1989) would be extremely useful. Use in Courses on Operating Systems

This book is suitable for use as a reference text to provide background for a primary textbook in a first-level course on operating systems. It is not intended for use as an introductory operating-system textbook; the reader should have already encountered terminology such as memory management, process scheduling, and I/O systems Silberschatz et al., 2002. Familiarity with the concepts of network protocols Comer, 2000; Stallings, 2000; Tanenbaum, 2003 will be useful for understanding some of the later chapters.

This book can be used in combination with a copy of the FreeBSD system for more advanced operating systems courses. Students’ assignments can include changes to, or replacements of, key system components such as the scheduler, the paging daemon, the filesystems, thread signalling, various networking layers, and I/O management. The ability to load, replace, and unload modules from a running kernel allows students to experiment without the need to compile and reboot the system. By working with a real operating system, students can directly measure and experience the effects of their changes. Because of the intense peer review and insistence on well-defined coding standards throughout its 25-year lifetime, the FreeBSD kernel is considerably cleaner, more modular, and thus easier to understand and modify than most software projects of its size and age.

Exercises are provided at the end of each chapter. The exercises are graded into three categories indicated by zero, one, or two asterisks. The answers to exercises that carry no asterisks can be found in the text. Exercises with a single asterisk require a step of reasoning or intuition beyond a concept presented in the text. Exercises with two asterisks present major design projects or open research questions. Organization

This text discusses both philosophical and design issues, as well as details of the actual implementation. Often, the discussion starts at the system-call level and descends into the kernel. Tables and figures are used to clarify data structures and control flow. Pseudocode similar to the C language displays algorithms. Boldface font identifies program names and filesystem pathnames. Italics font introduces terms that appear in the glossary and identifies the names of system calls, variables, routines, and structure names. Routine names (other than system calls) are further identified by the name followed by a pair of parentheses (e.g., malloc() is the name of a routine, whereas argv is the name of a variable).

The book is divided into five parts, organized as follows:

  • Part I, Overview      Three introductory chapters provide the context for the complete operating system and for the rest of the book. Chapter 1, History and Goals, sketches the historical development of the system, emphasizing the system’s research orientation. Chapter 2, Design Overview of FreeBSD, describes the services offered by the system and outlines the internal organization of the kernel. It also discusses the design decisions that were made as the system was developed. Sections 2.3 through 2.14 in Chapter 2 give an overview of their corresponding chapter. Chapter 3, Kernel Services, explains how system calls are done and describes in detail several of the basic services of the kernel.
  • Part II, Processes      The first chapter in this part—Chapter 4, Process Management— lays the foundation for later chapters by describing the structure of a process, the algorithms used for scheduling the execution of the threads that make up a process, and the synchronization mechanisms used by the system to ensure consistent access to kernel-resident data structures. In Chapter 5, Memory Management, the virtual-memory-management system is discussed in detail.
  • Part III, I/O System      First, Chapter 6, I/O System Overview, explains the system interface to I/O and describes the structure of the facilities that support this interface. Following this introduction are four chapters that give the details of the main parts of the I/O system. Chapter 7, Devices, giv es a description of the I/O architecture of the PC and describes how the I/O subsystem is managed and how the kernel initially maps out and later manages the arrival and departure of connected devices. Chapter 8, Local Filesystems, details the data structures and algorithms that implement filesystems as seen by application programs as well as how local filesystems are interfaced with the device interface described in Chapter 7. Chapter 9, The Network Filesystem, explains the network filesystem from both the server and client perspectives. Chapter 10, Terminal Handling, discusses support for character terminals and provides a description of the pseudo-terminal device driver.
  • Part IV, Interprocess Communication      Chapter 11, Interprocess Communication, describes the mechanism for providing communication between related or unrelated processes. Chapters 12 and 13, Network Communication and Network Protocols, are closely related because the facilities explained in the former are implemented by specific protocols, such as the TCP/IP protocol suite, explained in the latter.
  • Part V, System Operation      Chapter 14, Startup and Shutdown, discusses system startup and shutdown and explains system initialization at the process level from kernel initialization to user login.

The book is intended to be read in the order that the chapters are presented, but the parts other than Part I are independent of one another and can be read separately. Chapter 14 should be read after all the others, but knowledgeable readers may find it useful independently.

At the end of the book are a Glossary with brief definitions of major terms and an Index. Each chapter contains a Reference section with citations of related material. Getting BSD

All the BSD distributions are available either for downloading from the net or on removable media such as CD-ROM or DVD. Information on obtaining source and binaries for FreeBSD can be obtained from http://www.FreeBSD.org. The NetBSD distribution is compiled and ready to run on most workstation architectures. For more information, contact the NetBSD Project at http://www.NetBSD.org/. The OpenBSD distribution is compiled and ready to run on a wide variety of workstation architectures and has been extensively vetted for security and reliability. For more information, visit the OpenBSD project’sWeb site at http://www.OpenBSD.org/.

For you diehards who actually read to the end of the preface, your reward is finding out that you can get T-shirts that are a reproduction of the original artwork drawn by John Lasseter for the cover of this book (yes, he is the John Lasseter of Pixar fame who masterminded the production of Toy Story). For further information on purchasing a shirt, visit the “History of BSD T-shirts” Web page at http://www.McKusick.com/beastie/. Other items available for sale on the site include a 32-hour introductory video course based on this book, a 40-hour advanced video course based on the FreeBSD 5.2 source code, a 2.5-hour video lecture on the history of BSD, and a 4-CD set containing all the releases and the source-control history of BSD from Berkeley. These items are described in the advertisements that follow the Index.

Table of Contents

Preface xxi

About the Authors xxix

Part I: Over view 1

Chapter 1: History and Goals 3

1.1 History of the UNIX System 3

1.2 BSD and Other Systems 7

1.3 The Transition of BSD to Open Source 9

1.4 The FreeBSD Development Model 14

References 17

Chapter 2: Design Overview of FreeBSD 21

2.1 FreeBSD Facilities and the Kernel 21

2.2 Kernel Organization 23

2.3 Kernel Services 26

2.4 Process Management 26

2.5 Security 29

2.6 Memory Management 36

2.7 I/O System Overview 39

2.8 Devices 44

2.9 The Fast Filesystem 45

2.10 The Zettabyte Filesystem 49

2.11 The Network Filesystem 50

2.12 Interprocess Communication 50

2.13 Network-Layer Protocols 51

2.14 Transport-Layer Protocols 52

2.15 System Startup and Shutdown 52

Exercises 54

References 54

Chapter 3: Kernel Services 57

3.1 Kernel Organization 57

3.2 System Calls 62

3.3 Traps and Interrupts 64

3.4 Clock Interrupts 65

3.5 Memory-Management Services 69

3.6 Timing Services 73

3.7 Resource Services 75

3.8 Kernel Tracing Facilities 77

Exercises 84

References 85

Part II: Processes 87

Chapter 4: Process Management 89

4.1 Introduction to Process Management 89

4.2 Process State 92

4.3 Context Switching 99

4.4 Thread Scheduling 114

4.5 Process Creation 126

4.6 Process Termination 128

4.7 Signals 129

4.8 Process Groups and Sessions 136

4.9 Process Debugging 142

Exercises 144

References 146

Chapter 5: Security 147

5.1 Operating-System Security 148

5.2 Security Model 149

5.3 Process Credentials 151

5.4 Users and Groups 154

5.5 Privilege Model 157

5.6 Interprocess Access Control 159

5.7 Discretionary Access Control 161

5.8 Capsicum Capability Model 174

5.9 Jails 180

5.10 Mandatory Access-Control Framework 184

5.11 Security Event Auditing 200

5.12 Cryptographic Services 206

5.13 GELI Full-Disk Encryption 212

Exercises 217

References 217

Chapter 6: Memory Management 221

6.1 Terminology 221

6.2 Overview of the FreeBSD Virtual-Memory System 227

6.3 Kernel Memory Management 230

6.4 Per-Process Resources 244

6.5 Shared Memory 250

6.6 Creation of a New Process 258

6.7 Execution of a File 262

6.8 Process Manipulation of Its Address Space 263

6.9 Termination of a Process 266

6.10 The Pager Interface 267

6.11 Paging 276

6.12 Page Replacement 289

6.13 Portability 298

Exercises 308

References 310

Part III: I/OSystem 313

Chapter 7: I/O System Overview 315

7.1 Descriptor Management and Services 316

7.2 Local Interprocess Communication 333

7.3 The Virtual-Filesystem Interface 339

7.4 Filesystem-Independent Services 344

7.5 Stackable Filesystems 352

Exercises 358

References 359

Chapter 8: Devices 361

8.1 Device Overview 361

8.2 I/O Mapping from User to Device 367

8.3 Character Devices 370

8.4 Disk Devices 374

8.5 Network Devices 378

8.6 Terminal Handling 382

8.7 The GEOM Layer 391

8.8 The CAM Layer 399

8.9 Device Configuration 402

8.10 Device Virtualization 414

Exercises 428

References 429

Chapter 9: The Fast Filesystem 431

9.1 Hierarchical Filesystem Management 431

9.2 Structure of an Inode 433

9.3 Naming 443

9.4 Quotas 451

9.5 File Locking 454

9.6 Soft Updates 459

9.7 Filesystem Snapshots 480

9.8 Journaled Soft Updates 487

9.9 The Local Filestore 496

9.10 The Berkeley Fast Filesystem 501

Exercises 517

References 519

Chapter 10: The Zettabyte Filesystem 523

10.1 Introduction 523

10.2 ZFS Organization 527

10.3 ZFS Structure 532

10.4 ZFS Operation 535

10.5 ZFS Design Tradeoffs 547

Exercises 549

References 549

Chapter 11: The Network Filesystem 551

11.1 Overview 551

11.2 Structure and Operation 553

11.3 NFS Evolution 567

Exercises 586

References 587

Part IV: Interprocess Communication 591

Chapter 12: Interprocess Communication 593

12.1 Interprocess-Communication Model 593

12.2 Implementation Structure and Overview 599

12.3 Memory Management 601

12.4 IPC Data Structures 606

12.5 Connection Setup 612

12.6 Data Transfer 615

12.7 Socket Shutdown 620

12.8 Network-Communication Protocol Internal Structure 621

12.9 Socket-to-Protocol Interface 626

12.10 Protocol-to-Protocol Interface 631

12.11 Protocol-to-Network Interface 634

12.12 Buffering and Flow Control 643

12.13 Network Virtualization 644

Exercises 646

References 648

Chapter 13: Network-Layer Protocols 649

13.1 Internet Protocol Version 4 650

13.2 Internet Control Message Protocols (ICMP) 657

13.3 Internet Protocol Version 6 659

13.4 Internet Protocols Code Structure 670

13.5 Routing 675

13.6 Raw Sockets 686

13.7 Security 688

13.8 Packet-Processing Frameworks 700

Exercises 715

References 717

Chapter 14: Transport-Layer Protocols 721

14.1 Internet Ports and Associations 721

14.2 User Datagram Protocol (UDP) 723

14.3 Transmission Control Protocol (TCP) 725

14.4 TCP Algorithms 732

14.5 TCP Input Processing 741

14.6 TCP Output Processing 745

14.7 Stream Control Transmission Protocol (SCTP) 761

Exercises 768

References 770

Part V: System Operation 773

Chapter 15: System Startup and Shutdown 775

15.1 Firmware and BIOSes 776

15.2 Boot Loaders 777

15.3 Kernel Boot 782

15.4 User-Level Initialization 798

15.5 System Operation 800

Exercises 805

References 806

Glossary 807

Index 847

Preface

This book follows the earlier authoritative and full-length descriptions of the design and implementation of the 4.3BSD and 4.4BSD versions of the UNIX system developed at the University of California at Berkeley. Since the final Berkeley release in 1994, several groups have continued development of BSD. This book details FreeBSD, the system with the largest set of developers and the most widely distributed releases. Although the FreeBSD distribution includes nearly 1000 utility programs in its base system and nearly 10,000 optional utilities in its ports collection, this book concentrates almost exclusively on the kernel.

UNIX-like Systems

UNIX-like systems include the traditional vendor systems such as Solaris and HP-UX; the Linux-based distributions such as Red Hat, Debian, Suse, and Slackware; and the BSD-based distributions such as FreeBSD, NetBSD, OpenBSD, and Darwin. They run on computers ranging from laptops to the largest supercomputers. They are the operating system of choice for most multiprocessor, graphics, and vector-processing systems, and are widely used for the original purpose of timesharing. The most common platform for providing network services (from FTP to WWW) on the Internet, they are collectively the most portable operating system ever dev eloped. This portability is due partly to their implementation language, C Kernighan & Ritchie, 1989 (which is itself a widely ported language), and partly to the elegant design of the system.

Since its inception in 1969 Ritchie & Thompson, 1978, the UNIX system has developed in several divergent and rejoining streams. The original developers continued to advance the state of the art with their Ninth and Tenth Edition UNIX inside AT&T Bell Laboratories, and then their Plan 9 successor to UNIX. Meanwhile, AT&T licensed UNIX System V as a product before selling it to Novell. Novell passed the UNIX trademark to X/OPEN and sold the source code and distribution rights to Santa Cruz Operation (SCO). Both System V and Ninth Edition UNIX were strongly influenced by the Berkeley Software Distributions produced by the Computer Systems Research Group (CSRG) of the University of California at Berkeley. The Linux operating system, although developed independently of the other UNIX variants, implements the UNIX interface. Thus, applications developed to run on other UNIX-based platforms can be easily ported to run on Linux.

Berkeley Software Distributions

The distributions from Berkeley were the first UNIX-based systems to introduce many important features including the following:

  • Demand-paged virtual-memory support
  • Automatic configuration of the hardware and I/O system
  • A fast and recoverable filesystem
  • The socket-based interprocess-communication (IPC) primitives
  • The reference implementation of TCP/IP

The Berkeley releases found their way into the UNIX systems of many vendors and were used internally by the development groups of many other vendors. The implementation of the TCP/IP networking protocol suite in 4.2BSD and 4.3BSD, and the availability of those systems, played a key role in making the TCP/IP networking protocol suite a world standard. Even the non-UNIX vendors such as Microsoft have adopted the Berkeley socket design in their Winsock IPC interface.

The BSD releases have also been a strong influence on the POSIX (IEEE Std 1003.1) operating-system interface standard, and on related standards. Several features—such as reliable signals, job control, multiple access groups per process, and the routines for directory operations—have been adapted from BSD for POSIX.

Early BSD releases contained licensed UNIX code, thus requiring recipients to have an AT&T source license to be able to obtain and use BSD. In 1988, Berkeley separated its distribution into AT&T licensed and freely redistributable code. The freely redistributable code was licensed separately and could be obtained, used, and redistributed by anyone. The final freely redistributable 4.4BSD-Lite2 release from Berkeley in 1994 contained nearly the entire kernel and all the important libraries and utilities.

Two groups, NetBSD and FreeBSD, sprang up in 1993 to begin supporting and distributing systems built from the freely redistributable releases being done by Berkeley. The NetBSD group emphasized portability and the minimalist approach, porting the systems to nearly forty platforms and pushing to keep the system lean to aid embedded applications. The FreeBSD group emphasized maximal support for the PC architecture and pushed to ease installation for, and market their system to, as wide an audience as possible. In 1995, the OpenBSD group split from the NetBSD group to develop a distribution that emphasized security. Over the years there has been a healthy competition among the BSD distributions, with many ideas and much code flowing between them.

Material Covered in this Book

This book is about the internal structure of the FreeBSD 5.2 kernel and about the concepts, data structures, and algorithms used in implementing FreeBSD’s system facilities. Its level of detail is similar to that of Bach’s book about UNIX System V Bach, 1986; however, this text focuses on the facilities, data structures, and algorithms used in the FreeBSD variant of the UNIX operating system. The book covers FreeBSD from the system-call level down—from the interface to the kernel to the hardware itself. The kernel includes system facilities, such as process management, virtual memory, the I/O system, filesystems, the socket IPC mechanism, and network protocol implementations. Material above the system-call level—such as libraries, shells, commands, programming languages, and other user interfaces—is excluded, except for some material related to the terminal interface and to system startup. Following the organization first established by Organick’s book about Multics Organick, 1975, this book is an in-depth study of a contemporary operating system.

Where particular hardware is relevant, the book refers to the Intel Personal Computer (PC) architecture. Because FreeBSD has emphasized development on the PC, that is the architecture with the most complete support, so it provides a convenient point of reference.

Use by Computer Professionals

FreeBSD is widely used to support the core infrastructure of many companies worldwide. Because it can be built with a small footprint, it is also seeing increased use in embedded applications. The licensing terms of FreeBSD do not require the distribution of changes and enhancements to the system. The licensing terms of Linux require that all changes and enhancements to the kernel be made available in source form at minimal cost. Thus, companies that need to control the distribution of their intellectual property build their products using FreeBSD.

This book is of direct use to the professionals who work with FreeBSD systems. Individuals involved in technical and sales support can learn the capabilities and limitations of the system; applications developers can learn how to effectively and efficiently interface to the system; system administrators without direct experience with the FreeBSD kernel can learn how to maintain, tune, and configure the system; and systems programmers can learn how to extend, enhance, and interface to the system.

Readers who will benefit from this book include operating-system implementors, system programmers, UNIX application developers, administrators, and curious users. The book can be read as a companion to the source code of the system, falling as it does between the manual pages and the code in detail of treatment. But this book is neither exclusively a UNIX programming manual nor a user tutorial (for a tutorial, see Libes & Ressler 1988). Familiarity with the use of some version of the UNIX system (see, for example, Stevens 1992) and with the C programming language (see, for example, Kernighan & Ritchie 1989) would be extremely useful.

Use in Courses on Operating Systems

This book is suitable for use as a reference text to provide background for a primary textbook in a first-level course on operating systems. It is not intended for use as an introductory operating-system textbook; the reader should have already encountered terminology such as memory management, process scheduling, and I/O systems Silberschatz et al., 2002. Familiarity with the concepts of network protocols Comer, 2000; Stallings, 2000; Tanenbaum, 2003 will be useful for understanding some of the later chapters.

This book can be used in combination with a copy of the FreeBSD system for more advanced operating systems courses. Students’ assignments can include changes to, or replacements of, key system components such as the scheduler, the paging daemon, the filesystems, thread signalling, various networking layers, and I/O management. The ability to load, replace, and unload modules from a running kernel allows students to experiment without the need to compile and reboot the system. By working with a real operating system, students can directly measure and experience the effects of their changes. Because of the intense peer review and insistence on well-defined coding standards throughout its 25-year lifetime, the FreeBSD kernel is considerably cleaner, more modular, and thus easier to understand and modify than most software projects of its size and age.

Exercises are provided at the end of each chapter. The exercises are graded into three categories indicated by zero, one, or two asterisks. The answers to exercises that carry no asterisks can be found in the text. Exercises with a single asterisk require a step of reasoning or intuition beyond a concept presented in the text. Exercises with two asterisks present major design projects or open research questions.

Organization

This text discusses both philosophical and design issues, as well as details of the actual implementation. Often, the discussion starts at the system-call level and descends into the kernel. Tables and figures are used to clarify data structures and control flow. Pseudocode similar to the C language displays algorithms. Boldface font identifies program names and filesystem pathnames. Italics font introduces terms that appear in the glossary and identifies the names of system calls, variables, routines, and structure names. Routine names (other than system calls) are further identified by the name followed by a pair of parentheses (e.g., malloc() is the name of a routine, whereas argv is the name of a variable).

The book is divided into five parts, organized as follows:

  • Part I, Overview      Three introductory chapters provide the context for the complete operating system and for the rest of the book. Chapter 1, History and Goals, sketches the historical development of the system, emphasizing the system’s research orientation. Chapter 2, Design Overview of FreeBSD, describes the services offered by the system and outlines the internal organization of the kernel. It also discusses the design decisions that were made as the system was developed. Sections 2.3 through 2.14 in Chapter 2 give an overview of their corresponding chapter. Chapter 3, Kernel Services, explains how system calls are done and describes in detail several of the basic services of the kernel.
  • Part II, Processes      The first chapter in this part—Chapter 4, Process Management— lays the foundation for later chapters by describing the structure of a process, the algorithms used for scheduling the execution of the threads that make up a process, and the synchronization mechanisms used by the system to ensure consistent access to kernel-resident data structures. In Chapter 5, Memory Management, the virtual-memory-management system is discussed in detail.
  • Part III, I/O System      First, Chapter 6, I/O System Overview, explains the system interface to I/O and describes the structure of the facilities that support this interface. Following this introduction are four chapters that give the details of the main parts of the I/O system. Chapter 7, Devices, giv es a description of the I/O architecture of the PC and describes how the I/O subsystem is managed and how the kernel initially maps out and later manages the arrival and departure of connected devices. Chapter 8, Local Filesystems, details the data structures and algorithms that implement filesystems as seen by application programs as well as how local filesystems are interfaced with the device interface described in Chapter 7. Chapter 9, The Network Filesystem, explains the network filesystem from both the server and client perspectives. Chapter 10, Terminal Handling, discusses support for character terminals and provides a description of the pseudo-terminal device driver.
  • Part IV, Interprocess Communication      Chapter 11, Interprocess Communication, describes the mechanism for providing communication between related or unrelated processes. Chapters 12 and 13, Network Communication and Network Protocols, are closely related because the facilities explained in the former are implemented by specific protocols, such as the TCP/IP protocol suite, explained in the latter.
  • Part V, System Operation      Chapter 14, Startup and Shutdown, discusses system startup and shutdown and explains system initialization at the process level from kernel initialization to user login.

The book is intended to be read in the order that the chapters are presented, but the parts other than Part I are independent of one another and can be read separately. Chapter 14 should be read after all the others, but knowledgeable readers may find it useful independently.

At the end of the book are a Glossary with brief definitions of major terms and an Index. Each chapter contains a Reference section with citations of related material.

Getting BSD

All the BSD distributions are available either for downloading from the net or on removable media such as CD-ROM or DVD. Information on obtaining source and binaries for FreeBSD can be obtained from http://www.FreeBSD.org. The NetBSD distribution is compiled and ready to run on most workstation architectures. For more information, contact the NetBSD Project at http://www.NetBSD.org/. The OpenBSD distribution is compiled and ready to run on a wide variety of workstation architectures and has been extensively vetted for security and reliability. For more information, visit the OpenBSD project’sWeb site at http://www.OpenBSD.org/.

For you diehards who actually read to the end of the preface, your reward is finding out that you can get T-shirts that are a reproduction of the original artwork drawn by John Lasseter for the cover of this book (yes, he is the John Lasseter of Pixar fame who masterminded the production of Toy Story). For further information on purchasing a shirt, visit the “History of BSD T-shirts” Web page at http://www.McKusick.com/beastie/. Other items available for sale on the site include a 32-hour introductory video course based on this book, a 40-hour advanced video course based on the FreeBSD 5.2 source code, a 2.5-hour video lecture on the history of BSD, and a 4-CD set containing all the releases and the source-control history of BSD from Berkeley. These items are described in the advertisements that follow the Index.

Customer Reviews

Most Helpful Customer Reviews

See All Customer Reviews