UNIX and Linux System Administration Handbook / Edition 5

by Evi Nemeth, Garth Snyder, Trent Hein, Ben Whaley, Dan Mackin

Overview

“As an author, editor, and publisher, I never paid much attention to the competition—except in a few cases. This is one of those cases. The UNIX System Administration Handbook is one of the few books we ever measured ourselves against.”

—Tim O’Reilly, founder of O’Reilly Media

“This edition is for those whose systems live in the cloud or in virtualized data centers; those whose administrative work largely takes the form of automation and configuration source code; those who collaborate closely with developers, network engineers, compliance officers, and all the other worker bees who inhabit the modern hive.”

—Paul Vixie, Internet Hall of Fame-recognized innovator and founder of ISC and Farsight Security

“This book is fun and functional as a desktop reference. If you use UNIX and Linux systems, you need this book in your short-reach library. It covers a bit of the systems’ history but doesn’t bloviate. It’s just straight-forward information delivered in a colorful and memorable fashion.”

—Jason A. Nunnelley

UNIX^® and Linux^® System Administration Handbook, Fifth Edition, is today’s definitive guide to installing, configuring, and maintaining any UNIX or Linux system, including systems that supply core Internet and cloud infrastructure.

Updated for new distributions and cloud environments, this comprehensive guide covers best practices for every facet of system administration, including storage management, network design and administration, security, web hosting, automation, configuration management, performance analysis, virtualization, DNS, security, and the management of IT service organizations. The authors—world-class, hands-on technologists—offer indispensable new coverage of cloud platforms, the DevOps philosophy, continuous deployment, containerization, monitoring, and many other essential topics.

Whatever your role in running systems and networks built on UNIX or Linux, this conversational, well-written guide will improve your efficiency and help solve your knottiest problems.

Product Details

ISBN-13:	9780134277554
Publisher:	Pearson Education
Publication date:	08/08/2017
Edition description:	New Edition
Pages:	1232
Product dimensions:	6.90(w) x 9.00(h) x 1.90(d)

About the Author

Evi Nemeth pioneered the discipline of UNIX system administration. She taught and mentored computer science students at the University of Colorado Boulder, was visiting faculty member at Dartmouth College and UC San Diego, and helped bring Internet technology to the developing world through her work with the Internet Society and the United Nations.

Garth Snyder has worked at NeXT and Sun and holds a BS in Engineering from Swarthmore College and an MD and an MBA from the University of Rochester.

Trent R. Hein (@trenthein) is a serial entrepreneur who is passionate about practical cybersecurity and automation. Outside of technology, he loves hiking, skiing, fly fishing, camping, bluegrass, dogs, and the Oxford comma. Trent holds a BS in Computer Science from the University of Colorado.

Ben Whaley is the founder of WhaleTech, an independent consultancy. He was honored by Amazon as one of the first AWS Community Heroes. He obtained a B.S. in Computer Science from the University of Colorado at Boulder.

Dan Mackin’s (@dan_mackin) long-standing passion for technology inspired him to get a BS in Electrical and Computer Engineering from the University of Colorado at Boulder. He applies Linux and other open source technologies not only in his day job, but also to automation, monitoring, and weather metrics collection projects at home. Dan loves spending time with his wife and dog, skiing, movies, sailing, and backcountry touring.

Read an Excerpt

Preface

When we were writing the first edition of this book in the mid-1980s, we were eager to compare our manuscript with other books about unix system administration. To our delight, we could find only three.

These days, you have your choice of at least fifty. Here are the features that distinguish our book:

We take a practical approach. Our purpose is not to restate the contents of your manuals, but rather to give you the benefit of our collective experience in system administration. This book contains plenty of war stories and a wealth of pragmatic advice.

We cover unix networking in detail. It is the most difficult aspect of unix system administration, and the area in which we can most likely be of help to you.

We do not oversimplify the material. Our examples reflect true-life situations, with all their warts and unsightly complications. In most cases, the examples have been taken directly from production systems.

We emphasize the use of software tools. Every piece of software mentioned in the text is either a standard unix tool, or is included on the cd-rom at the back of this book sometimes both, since many vendors don't do a perfect job of keeping up with new releases.

We cover all the major variants of unix. Our Six Example Systems There are two main flavors of unix: one from AT&T (original) and one from the University of California, Berkeley (extra crispy). Neither AT&T nor Berkeley is still active in the unix marketplace, but the terms at&t unix and Berkeley unix survive for historical reasons.

This book covers six different operating systems:

Solaris 2.4 SunOS 4.1.3
hp-ux 9.0 dec'sosf/1 2.0
irix 5.2 bsd/os 1.1

We chose these systems because they are among the most popular, and because they illustrate a broad range of approaches to unix administration. The systems in the left column are predominantly derived from AT&T unix, while those on the right are more like Berkeley unix. We provide detailed information about each of these example systems for every topic that we discuss. Comments specific to a particular operating system are marked with the manufacturer's logo. There are many other versions of unix. Most fall within the range of variation defined by these six systems, but a few (such as aix and sco) are so beautifully strange that they must be taken on their own terms.

The Organization of this Book.

This book is divided into three large chunks: Basic Administration, Networking, and Bunch o Stuff. Basic Administration provides a broad overview of unix from a system administrator's perspective. The chapters in this section cover most of the facts and techniques needed to run a stand-alone unix system.

The Networking section describes the protocols used on unix systems and the techniques used to set up, extend, and maintain networks. High-level network software is also covered here. Among the featured topics are the Domain Name System, the Network File System, network routing, and sendmail.

Bunch o Stuff includes a variety of supplemental information. Some chapters discuss optional software packages such as uucp and the unix printing system. Others give sage advice on topics ranging from hardware maintenance to disk space management to the politics of running a unix installation.

Aout the CD-ROM.

The cd-rom contains software and reference information that we recommend for system administrators. Most of the items on the cd-rom are available over the Internet, but you may find the cd-rom faster and more convenient to use.

The cd-rom uses the iso-9660 format, which is supported by most computers (including our six example systems). This format does not allow filenames longer than eight characters, so we have packaged up the tools using the standard unix tar command. Decoding instructions are included at the back of this book.

The cd-rom will be updated on an approximately yearly basis. To determine the age of your copy, check the date printed on the cd-rom itself.

Table of Contents

Tribute to Evi xl

Preface xlii

Foreword xliv

Acknowledgments xlvi

Section One: Basic Administration 1

Chapter 1: Where to Start 3

Essential duties of a system administrator 4

Suggested background 7

Linux distributions 8

Example systems used in this book 9

Notation and typographical conventions 12

Units 13

Man pages and other on-line documentation 14

Other authoritative documentation 16

Other sources of information 18

Ways to find and install software 19

Where to host 25

Specialization and adjacent disciplines 26

Recommended reading28

Chapter 2: Booting and System Management Daemons 30

Boot process overview 30

System firmware 32

Boot loaders 35

GRUB: the GRand Unified Boot loader 35

The FreeBSD boot process 39

System management daemons .41

systemd in detail 44

FreeBSD init and startup scripts 57

Reboot and shutdown procedures 59

Stratagems for a nonbooting system 60

Chapter 3: Access Control and Rootly Powers 65

Standard UNIX access control 66

Management of the root account69

Extensions to the standard access control model 79

Modern access control 83

Recommended reading89

Chapter 4: Process Control 90

Components of a process 90

The life cycle of a process 93

ps: monitor processes 98

Interactive monitoring with top101

nice and renice: influence scheduling priority102

The /proc filesystem 104

strace and truss: trace signals and system calls 105

Runaway processes 107

Periodic processes109

Chapter 5: The Filesystem 120

Pathnames 122

Filesystem mounting and unmounting 122

Organization of the file tree125

File types 126

File attributes132

Access control lists 140

Chapter 6: Software Installation and Management 153

Operating system installation 154

Managing packages 162

Linux package management systems 164

High-level Linux package management systems 166

FreeBSD software management175

Software localization and configuration 178

Recommended reading 181

Chapter 7: Scripting and the Shell 182

Scripting philosophy 183

Shell basics 189

sh scripting 198

Regular expressions 209

Python programming 215

Ruby programming 223

Library and environment management for Python and Ruby 229

Revision control with Git 235

Recommended reading 241

Chapter 8: User Management 243

Account mechanics 244

The /etc/passwd file 245

The Linux /etc/shadow file250

FreeBSD's /etc/master.passwd and /etc/login.conf files 252

The /etc/group file 254

Manual steps for adding users 255

Scripts for adding users: useradd, adduser, and newusers 260

Safe removal of a user’s account and files264

User login lockout265

Risk reduction with PAM 266

Centralized account management 266

Chapter 9: Cloud Computing 270

The cloud in context 271

Cloud platform choices 273

Cloud service fundamentals 276

Clouds: VPS quick start by platform283

Cost control 291

Recommended Reading 293

Chapter 10: Logging 294

Log locations296

The systemd journal 299

Syslog 302

Kernel and boot-time logging 318

Management and rotation of log files 319

Management of logs at scale 321

Logging policies 323

Chapter 11: Drivers and the Kernel 325

Kernel chores for system administrators 326

Kernel version numbering 327

Devices and their drivers 328

Linux kernel configuration339

FreeBSD kernel configuration 344

Loadable kernel modules 346

Booting 348

Booting alternate kernels in the cloud 355

Kernel errors356

Recommended reading 359

Chapter 12: Printing 360

CUPS printing 361

CUPS server administration 365

Troubleshooting tips 369

Recommended reading 371

Section Two: Networking 373

Chapter 13: TCP/IP Networking 375

TCP/IP and its relationship to the Internet 375

Networking basics 378

Packet addressing384

IP addresses: the gory details 387

Routing 398

IPv4 ARP and IPv6 neighbor discovery 401

DHCP: the Dynamic Host Configuration Protocol402

Security issues 406

Basic network configuration 410

Linux networking417

FreeBSD networking 425

Network troubleshooting 428

Network monitoring 437

Firewalls and NAT 440

Cloud networking448

Recommended reading 457

Chapter 14: Physical Networking 459

Ethernet: the Swiss Army knife of networking460

Wireless: Ethernet for nomads 469

SDN: software-defined networking 473

Network testing and debugging474

Building wiring 475

Network design issues476

Management issues 478

Recommended vendors 479

Recommended reading 480

Chapter 15: IP Routing 481

Packet forwarding: a closer look482

Routing daemons and routing protocols 485

Protocols on parade 488

Routing protocol multicast coordination490

Routing strategy selection criteria 490

Routing daemons492

Cisco routers494

Recommended reading 496

Chapter 16: DNS: The Domain Name System 498

DNS architecture 499

DNS for lookups 500

The DNS namespace 502

How DNS works 503

The DNS database512

The BIND software 525

Split DNS and the view statement 541

BIND configuration examples 543

Zone file updating547

DNS security issues 551

BIND debugging 568

Recommended reading 576

Chapter 17: Single Sign-On 578

Core SSO elements 579

LDAP: “lightweight” directory services 580

Using directory services for login 586

Alternative approaches594

Recommended reading 595

Chapter 18: Electronic Mail 596

Mail system architecture 597

Anatomy of a mail message600

The SMTP protocol 603

Spam and malware 605

Message privacy and encryption 607

Mail aliases 608

Email configuration 612

sendmail 613

Exim 640

Postfix 658

Recommended reading 672

Chapter 19: Web Hosting 674

HTTP: the Hypertext Transfer Protocol 674

Web software basics 682

Web hosting in the cloud 694

Apache httpd696

NGINX 704

HAProxy 710

Recommended reading 714

Section Three: Storage 715

Chapter 20: Storage 717

I just want to add a disk! 718

Storage hardware 721

Storage hardware interfaces 730

Attachment and low-level management of drives 733

The software side of storage: peeling the onion 739

Disk partitioning 742

Logical volume management 747

RAID: redundant arrays of inexpensive disks 753

Filesystems 762

Traditional filesystems: UFS, ext4, and XFS 763

Next-generation filesystems: ZFS and Btrfs 772

ZFS: all your storage problems solved 773

Btrfs: “ZFS lite” for Linux 783

Data backup strategy 788

Recommended reading 790

Chapter 21: The Network File System 791

Meet network file services 791

The NFS approach794

Server-side NFS 801

Client-side NFS 807

Identity mapping for NFS version 4 810

nfsstat: dump NFS statistics 811

Dedicated NFS file servers 812

Automatic mounting 812

Recommended reading 818

Chapter 22: SMB 819

Samba: SMB server for UNIX 820

Installing and configuring Samba 821

Mounting SMB file shares 825

Browsing SMB file shares 826

Ensuring Samba security 826

Debugging Samba827

Recommended reading 829

Section Four: Operations 831

Chapter 23: Configuration Management 833

Configuration management in a nutshell834

Dangers of configuration management 834

Elements of configuration management 835

Popular CM systems compared 841

Introduction to Ansible 852

Introduction to Salt 871

Ansible and Salt compared 893

Best practices895

Recommended reading 899

Chapter 24: Virtualization 900

Virtual vernacular901

Virtualization with Linux 905

FreeBSD bhyve 910

VMware910

VirtualBox 911

Packer 911

Vagrant 913

Recommended reading 914

Chapter 25: Containers 915

Background and core concepts 916

Docker: the open source container engine 919

Containers in practice937

Container clustering and management 942

Recommended reading 948

Chapter 26: Continuous Integration and Delivery 949

CI/CD essentials 951

Pipelines 955

Jenkins: the open source automation server 961

CI/CD in practice964

Containers and CI/CD978

Recommended reading 980

Chapter 27: Security 981

Elements of security 983

How security is compromised 983

Basic security measures 987

Passwords and user accounts 992

Security power tools 996

Cryptography primer1005

SSH, the Secure SHell1016

Firewalls 1027

Virtual private networks (VPNs) 1030

Certifications and standards 1031

Sources of security information 1034

When your site has been attacked 1037

Recommended reading 1038

Chapter 28: Monitoring 1040

An overview of monitoring 1041

The monitoring culture 1044

The monitoring platforms1045

Data collection 1051

Network monitoring 1055

Systems monitoring 1056

Application monitoring 1059

Security monitoring 1061

SNMP: the Simple Network Management Protocol 1063

Tips and tricks for monitoring1068

Recommended reading 1069

Chapter 29: Performance Analysis 1070

Performance tuning philosophy 1071

Ways to improve performance 1073

Factors that affect performance 1074

Stolen CPU cycles 1075

Analysis of performance problems 1076

System performance checkup 1077

Help! My server just got really slow! 1088

Recommended reading 1090

Chapter 30: Data Center Basics 1091

Racks1092

Power 1092

Cooling and environment1096

Data center reliability tiers 1101

Data center security 1102

Tools 1103

Recommended reading 1104

Chapter 31: Methodology, Policy, and Politics 1105

The grand unified theory: DevOps 1106

Ticketing and task management systems 1111

Local documentation maintenance1115

Environment separation 1118

Disaster management1119

IT policies and procedures 1122

Service level agreements 1125

Compliance: regulations and standards 1127

Legal issues 1131

Organizations, conferences, and other resources 1133

Recommended reading 1135

Index 1136

A Brief History of System Administration 1166

Colophon 1176

About the Contributors 1178

About the Authors 1179

Preface

Preface

When we were writing the first edition of this book in the mid-1980s, we were eager to compare our manuscript with other books about UNIX system administration. To our delight, we could find only three. These days, you have your choice of at least fifty. Here are the features that distinguish our book:

• We take a practical approach. Our purpose is not to restate the contents of your manuals but rather to give you the benefit of our collective experience in system administration. This book contains numerous war stories and a wealth of pragmatic advice.
• We cover UNIX networking in detail. It is the most difficult aspect of UNIX system administration, and the area in which we think we can most likely be of help to you.
• We do not oversimplify the material. Our examples reflect true-life situations, with all their warts and unsightly complications. In most cases, the examples have been taken directly from production systems.
• We emphasize the use of software tools. Every piece of software mentioned in the text is either a standard UNIX tool or is freely available from the Internet—sometimes both, since many vendors don't do a perfect job of keeping up with new releases.
• We cover all the major variants of UNIX.

Our four example systems

There have historically been two main flavors of UNIX: one from AT&T (known generically as System V) and one from the University of California, Berkeley (known as BSD). Neither AT&T nor Berkeley is still active in the UNIX marketplace, but the terms "AT&T UNIX" and "Berkeley UNIX" live on.This book covers four differentsystems:

• Solaris 2.7
• HP-UX 11.00
• Red Hat Linux 6.2
• FreeBSD 3.4 (and bits of 4.0)

We chose these systems because they are among the most popular and because they illustrate a broad range of approaches to UNIX administration. The first two systems are similar to AT&T UNIX, FreeBSD is a direct descendant of Berkeley UNIX, and Red Hat Linux is something of a mix.

We provide detailed information about each of these example systems for every topic that we discuss. Comments specific to a particular operating system are marked with the manufacturer's logo.

There are many other versions of UNIX. Most fall within the range of variation defined by these four systems, but a few (such as AIX and SCO) are so beautifully strange that they must be taken on their own terms.

The organization of this book

This book is divided into three large chunks: Basic Administration, Networking, and Bunch o' Stuff.

Basic Administration provides a broad overview of UNIX from a system administrator's perspective. The chapters in this section cover most of the facts and techniques needed to run a stand-alone UNIX system.

The Networking section describes the protocols used on UNIX systems and the techniques used to set up, extend, and maintain networks. High-level network software is also covered here. Among the featured topics are the Domain Name System, the Network File System, routing, sendmail, and network management.

Bunch o' Stuff includes a variety of supplemental information. Some chapters discuss optional software packages such as the UNIX printing system (or more accurately, system ranging from hardware maintenance to the politics of running a UNIX installation.

Contact information

In this edition, we're pleased to welcome Adam Boggs, Rob Braun, Dan Crawl, Ned McClain, Lynda McGinley, and Todd Miller as contributing authors. We've turned to them for their deep knowledge in a variety of areas (and also for their ability to function amid the shifting sands of this book and its temperamental parents). Their contributions have greatly enriched the overall content of the book and the collective experience that we're able to share with you.

Please send suggestions, comments, typos, and bug reports to [e-mail]. We answer all mail, but please be patient; it is sometimes a few days before one of us is able to respond. To get a copy of our current bug list and other late-breaking information, visit our web site at www.admin.com.

We hope you enjoy this book, and we wish you the best of luck with your adventures in system administration!

Evi Nemeth
Garth Snyder
Scott Seebass
Trent R. Hein

June, 2000

Editorial Reviews

Fatbrain Review

The authors assume that you have a certain amount of UNIX experience; in particular, a general concept of how UNIX looks and feels from a user's perspective. From their own practical experiences the authors give you an overview of the major administrative systems, introduce general administrative techniques, help you choose solutions that will continue to work well as your site grows in size and complexity, summarize common procedures saving you the time of digging through the manuals to accomplish simple tasks. This book is designed to be a compliment to, not a replacement for your system's documentation. Encompassing everything from adding new users, troubleshooting, adding hardware, configuring the Kernel, communications/networking, the Internet, policy and politics. A most complete UNIX system administration book.

First covers the techniques needed to run a stand-alone UNIX system, then describes the protocols used on UNIX systems and the techniques to set up, extend, and maintain networks. The third edition covers Red Hat Linux. Annotation c. Book News, Inc., Portland, OR (booknews.com)

Booknews