sed and awk

All Available Formats + Editions

$0.26

$23.23

Paperback

$4.78

$23.23

sed and awk (Second Edition) Pub. Date: 3/28/1997 Publisher: O'Reilly Media, Incorporated

$0.26

——

sed and awk Pub. Date: 11/8/1990 Publisher: O'Reilly Media, Incorporated

Want to read this on your NOOK? Request as NOOK Book from the publisher

Thank you for requesting this book as a NOOK book from the publisher.

Customers Who Bought This Also Bought

Advertising

Overview

sed & awk describes two text processing programs that are mainstays of the UNIX programmer's toolbox.

sed is a "stream editor" for editing streams of text that might be too large to edit as a single file, or that might be generated on the fly as part of a larger data processing step. The most common operation done with sed is substitution, replacing one block of text with another.

awk is a complete programming language. Unlike many conventional languages, awk is "data driven" — you specify what kind of data you are interested in and the operations to be performed when that data is found. awk does many things for you, including automatically opening and closing data files, reading records, breaking the records up into fields, and counting the records. While awk provides the features of most conventional programming languages, it also includes some unconventional features, such as extended regular expression matching and associative arrays. sed & awk describes both programs in detail and includes a chapter of example sed and awk scripts.

This edition covers features of sed and awk that are mandated by the POSIX standard. This most notably affects awk, where POSIX standardized a new variable, CONVFMT, and new functions, toupper() and tolower(). The CONVFMT variable specifies the conversion format to use when converting numbers to strings (awk used to use OFMT for this purpose). The toupper() and tolower() functions each take a (presumably mixed case) string argument and return a new version of the string with all letters translated to the corresponding case.

In addition, this edition covers GNU sed, newly available since the first edition. It also updates the first edition coverage of Bell Labs nawk and GNU awk (gawk), covers mawk, an additional freely available implementation of awk, and briefly discusses three commercial versions of awk, MKS awk, Thompson Automation awk (tawk), and Videosoft (VSAwk).

Serious UNIX programmers and administrators will enjoy the second edition of this best-selling book on a set of the most popular UNIX utilities, sed and awk. Why? Because it covers awk as described by the POSIX standard as well as NetBSD, FreeBSD, and the Linux versions of awk.

Editorial Reviews

From Barnes & Noble

Fatbrain Review

The journey begins with an overview of the basic operations of sed and awk, showing a progression in functionality from grep to sed to awk. The next stop is writing sed scripts. You'll learn the syntax of sed commands, and advanced features, including multiple pattern space and hold space commands.

The book then moves to writing scripts for awk. Discussions include pattern matching, expressions, relational and Boolean operators, and informal retrieval. The text also explains awk's built-in functions and user-defined functions. The authors keep you learn by outlining the development of an index processing application, and they offer the readers contact information on how to obtain various versions of awk. This tutorial includes a miscellany of sed and awk scripting styles and techniques.

Product Details

ISBN-13: 9781565922259
Publisher: O'Reilly Media, Incorporated
Publication date: 3/28/1997
Edition description: Second Edition
Edition number: 2
Pages: 434
Sales rank: 265,285
Series: Nutshell Handbooks Series
Product dimensions: 7.00 (w) x 9.19 (h) x 0.88 (d)

Meet the Author

Dale Dougherty is the publisher of the O'Reilly Network and Director of O'Reilly Research. Dale has been instrumental in many of O'Reilly's most important efforts, including founding O'Reilly & Associates with Tim O'Reilly. He was the developer and publisher of Global Network Navigator (GNN), the first commercial Web site. Dale was developer and publisher of Web Review, the online magazine for Web designers, and he was O'Reilly & Associates' first editor. Dale has written and edited numerous books at O'Reilly & Associates. Dougherty is a Lecturer in the School of Information Management and Systems (SIMS) at the University of California at Berkeley.

Arnold Robbins, an Atlanta native, is a professional programmer and technical author. He has worked with Unix systems since 1980, when he was introduced to a PDP-11 running a version of Sixth Edition Unix. He has been a heavy AWK user since 1987, when he became involved with gawk, the GNU project's version of AWK. As a member of the POSIX 1003.2 balloting group, he helped shape the POSIX standard for AWK. He is currently the maintainer of gawk and its documentation. He is also coauthor of the sixth edition of O'Reilly's Learning the vi Editor. Since late 1997, he and his family have been living happily in Israel.

Read an Excerpt

From Chapter 7: Writing Scripts for awk

As mentioned in the preface, this book describes POSIX awk; that is, the awk language as specified by the POSIX standard. Before diving into the details, we'll provide a bit of history.

The original awk was a nice little language. It first saw the light of day with Version 7 UNIX, around 1978. It caught on, and people used it for significant programming.

In 1985, the original authors, seeing that awk was being used for more serious programming than they had ever intended, decided to beef up the language. (See Chapter 11, A Flock of awks, for a description of the original awk, and all the things it did not have when compared to the new one.) The new version was finally released to the world at large in 1987, and it is this version that is still found on SunOS 4.1.x systems.

In 1989, for System V Release 4, awk was updated in some minor ways. This version became the basis for the awk feature list in the POSIX standard. POSIX clarified a number of things about awk, and added the CONVFMT variable (to be discussed later in this chapter).

As you read the rest of this book, bear in mind that the term awk refers to POSIX awk, and not to any particular implementation, whether the original one from Bell Labs, or any of the others discussed in Chapter 11. However, in the few cases where different versions have fundamental differences of behavior, that will be pointed out in the main body of the discussion.

Playing the Game

To write an awk script, you must become familiar with the rules of the game. The rules can be stated plainly and you will find them described in Appendix B, QuickReference for awk, rather than in this chapter. The goal of this chapter is not to describe the rules but to show you how to play the game. In this way, you will become acquainted with many of the features of the language and see examples that illustrate how scripts actually work. Some people prefer to begin by reading the rules, which is roughly equivalent to learning to use a program from its manual page or learning to speak a language by scanning its rules of grammar--not an easy task. Having a good grasp of the rules, however, is essential once you begin to use awk regularly. But the more you use awk, the faster the rules of the game become second nature. You learn them through trial and error--spending a long time trying to fix a silly syntax error such as a missing space or brace has a magical effect upon long-term memory. Thus, the best way to learn to write scripts is to begin writing them. As you make progress writing scripts, you will no doubt benefit from reading the rules (and rereading them) in Appendix B or the awk manpage or The AWK Programming Language book. You can do that later--let's get started now.

Hello, World

It has become a convention to introduce a programming language by demonstrating the "Hello, world" program. Showing this program works in awk will demonstrate just how unconventional awk is. In fact, it's necessary to show several different approaches to printing "Hello, world."

In the first example, we create a file named test that contains a single line. This example shows a script that contains the print statement:

$ echo 'this line of data is ignored' > test $ awk '{ print "Hello, world" }' test

Hello, world

This script has only a single action, which is enclosed in braces. That action is to execute the print statement for each line of input. In this case, the test file contains only a single line; thus, the action occurs once. Note that the input line is read but never output.

Now let's look at another example. Here, we use a file that contains the line "Hello, world."

$ cat test2 Hello, world $ awk '{ print }&39; test2 Hello, world

In this example, "Hello, world" appears in the input file. The same result is achieved because the print statement, without arguments, simply outputs each line of input. If there were additional lines of input, they would be output as well.

Both of these examples illustrate that awk is usually input-driven. That is, nothing happens unless there are lines of input on which to act. When you invoke the awk program, it reads the script that you supply, checking the syntax of your instructions. Then awk attempts to execute the instructions for each line of input. Thus, the print statement will not be executed unless there is input from the file.

To verify this for yourself, try entering the command line in the first example but omit the filename. You'll find that because awk expects input to come from the keyboard, it will wait until you give it input to process: press RETURN several times, then type an EOF (CTRL-D on most systems) to signal the end of input. For each time that you pressed RETURN, the action that prints "Hello, world" will be executed.

There is yet another way to write the "Hello, world" message and not have awk wait for input. This method associates the action with the BEGIN pattern. The BEGIN pattern specifies actions that are performed before the first line of input is read.

$ awk 'BEGIN { print "Hello, world" }' Hello, world

Awk prints the message, and then exits. If a program has only a BEGIN pattern, and no other statements, awk will not process any input files.

Awk's Programming Model

It's important to understand the basic model that awk offers the programmer. Part of the reason why awk is easier to learn than many programming languages is that it offers such a well-defined and useful model to the programmer.

An awk program consists of what we will call a main input look. a loop is a routine that is executed over and over again until some condition exists that terminates it. You don't write this loop, it is given--it exists as the framework within which the code that you do write will be executed. The main input loop in awk is a routine that reads one line of input from a file and makes it available for processing. The actions you write to do the processing assume that there is a line of input available. In another programming language, you would have to create the main input loop as part of your program. It would have to open the input file and read one line at a time. This is not necessary a lot of work, but it illustrates a basic awk shortcut and makes it easier for you to write your program.

The main input loop is executed as many times as there are lines of input. As you saw in the "Hello, world" examples, this loop does not execute until there is a line of input. It terminates when there is no more input to be read.

Awk allows you to write two special routines that can be executed before any input is read and after all input is read. These are the procedures associated with the BEGIN and END rules, respectively. In other words, you can do some preprocessing before the main input loop is ever executed and you can do some postprocessing after the main input loop has terminated. The BEGIN and END procedures are optional.

You can think of an awk script as having potentially three major parts: what happens before, what happens during, and what happens after processing the input. Figure 7-1 shows the relationship of these parts in the flow of control of an awk script...

Dedication; Preface; Scope of This Handbook; Availability of sed and awk; Obtaining Example Source Code; Conventions Used in This Handbook; About the Second Edition; Acknowledgments from the First Edition; Comments and Questions; Chapter 1: Power Tools for Editing; 1.1 May You Solve Interesting Problems; 1.2 A Stream Editor; 1.3 A Pattern-Matching Programming Language; 1.4 Four Hurdles to Mastering sed and awk; Chapter 2: Understanding Basic Operations; 2.1 Awk, by Sed and Grep, out of Ed; 2.2 Command-Line Syntax; 2.3 Using sed; 2.4 Using awk; 2.5 Using sed and awk Together; Chapter 3: Understanding Regular Expression Syntax; 3.1 That's an Expression; 3.2 A Line-Up of Characters; 3.3 I Never Metacharacter I Didn't Like; Chapter 4: Writing sed Scripts; 4.1 Applying Commands in a Script; 4.2 A Global Perspective on Addressing; 4.3 Testing and Saving Output; 4.4 Four Types of sed Scripts; 4.5 Getting to the PromiSed Land; Chapter 5: Basic sed Commands; 5.1 About the Syntax of sed Commands; 5.2 Comment; 5.3 Substitution; 5.4 Delete; 5.5 Append, Insert, and Change; 5.6 List; 5.7 Transform; 5.8 Print; 5.9 Print Line Number; 5.10 Next; 5.11 Reading and Writing Files; 5.12 Quit; Chapter 6: Advanced sed Commands; 6.1 Multiline Pattern Space; 6.2 A Case for Study; 6.3 Hold That Line; 6.4 Advanced Flow Control Commands; 6.5 To Join a Phrase; Chapter 7: Writing Scripts for awk; 7.1 Playing the Game; 7.2 Hello, World; 7.3 Awk's Programming Model; 7.4 Pattern Matching; 7.5 Records and Fields; 7.6 Expressions; 7.7 System Variables; 7.8 Relational and Boolean Operators; 7.9 Formatted Printing; 7.10 Passing Parameters Into a Script; 7.11 Information Retrieval; Chapter 8: Conditionals, Loops, and Arrays; 8.1 Conditional Statements; 8.2 Looping; 8.3 Other Statements That Affect Flow Control; 8.4 Arrays; 8.5 An Acronym Processor; 8.6 System Variables That Are Arrays; Chapter 9: Functions; 9.1 Arithmetic Functions; 9.2 String Functions; 9.3 Writing Your Own Functions; Chapter 10: The Bottom Drawer; 10.1 The getline Function; 10.2 The close( ) Function; 10.3 The system( ) Function; 10.4 A Menu-Based Command Generator; 10.5 Directing Output to Files and Pipes; 10.6 Generating Columnar Reports; 10.7 Debugging; 10.8 Limitations; 10.9 Invoking awk Using the #! Syntax; Chapter 11: A Flock of awks; 11.1 Original awk; 11.2 Freely Available awks; 11.3 Commercial awks; 11.4 Epilogue; Chapter 12: Full-Featured Applications; 12.1 An Interactive Spelling Checker; 12.2 Generating a Formatted Index; 12.3 Spare Details of the masterindex Program; Chapter 13: A Miscellany of Scripts; 13.1 uutot.awk—Report UUCP Statistics; 13.2 phonebill—Track Phone Usage; 13.3 combine—Extract Multipart uuencoded Binaries; 13.4 mailavg—Check Size of Mailboxes; 13.5 adj—Adjust Lines for Text Files; 13.6 readsource—Format Program Source Files for troff; 13.7 gent—Get a termcap Entry; 13.8 plpr—lpr Preprocessor; 13.9 transpose—Perform a Matrix Transposition; 13.10 m1—Simple Macro Processor; Appendix A: Quick Reference for sed; A.1 Command-Line Syntax; A.2 Syntax of sed Commands; A.3 Command Summary for sed; Appendix B: Quick Reference for awk; B.1 Command-Line Syntax; B.2 Language Summary for awk; B.3 Command Summary for awk; Appendix C: Supplement for Chapter 12; C.1 Full Listing of spellcheck.awk; C.2 Listing of masterindex Shell Script; C.3 Documentation for masterindex; C.3.1 Background Details; C.3.2 Coding Index Entries; C.3.3 Output Format; C.3.4 Compiling a Master Index; Colophon;

Introduction

This book is about a set of oddly named UNIX utilities sed and awk. These utilities have many things in common, including the use of regular expressions for pattern matching. Since pattern matching is such an important part of their use, this book explains UNIX regular expression syntax very thoroughly. Because there is a natural progression in learning from grep to sed to awk, we will be covering all three programs, although the focus is on sed and awk.

Sed and awk are tools used by users, programmers, and system administrators_ anyone working with text files. Sed, so called because it is a stream editor, is perfect for applying a series of edits to a number of files. Awk, named after its developers Aho,, Weinberger,, and Kernighan,, is a programming language that permits easy manipulation of structured data and the generation of formatted reports. This book emphasizes the POSIX definition of awk. In addition, the book briefly describes the original version of awk, before discussing three freely available versions of awk and two commercial ones, all of which implement POSIX awk.

The focus of this book is on writing scripts for sed and awk that quickly solve an assortment of problems for the user. Many of these scripts could be called "quickfixes." In addition, we'll cover scripts that solve larger problems that require more careful design and development.

Scope of This Handbook
Chapter 1, Power Tools for Editing, is an overview of the features and capabilities of sed and awk.

Chapter 2, Under standing, Basic Operations, demonstrates the basic operations of sed and awk, showing a progression in functionality from sed to awk. Both share a similarcommand-line syntax, accepting user instructions in the form of a script.

Chapter 3, Understanding Regular Expression Syntax, describes UNIX regular expression syntax in full detail. New users are often intimidated by these strange expressions, used for pattern matching. It is important to master regular expression syntax to get the most from sed and awk. The pattern-matching examples in this chapter largely rely on grep and egrep.

Chapter 4, Writing sed Scripts, begins a three-chapter section on sed. This chapter covers the basic elements of writing a sed script using only a few sed commands. It also presents a shell script that simplifies invoking sed scripts.

Chapter 5, Basic sed Commands and Chapter 6, Advanced sed Commands, divide the sed command set into basic and advanced commands. The basic commands are commands that parallel manual editing actions, while the advanced commands introduce simple programming capabilities. Among the advanced commands are those that manipulate the hold space, a set-aside temporary buffer.

Chapter 7, Writing Scripts for awk, begins a five-chapter section on awk. This chapter presents the primary features of this scripting language. A number of scripts are explained, including one that modifies the output of the ls command.

Chapter 8, Conditionals' Loops, and Arrays, describes how to use common programming constructs such as conditionals, loops, and arrays.

Chapter 9, Functions, describes how to use awk's built-in functions as well as how to write user-defined functions.

Chapter 10, The Bottom Drawer, covers a set of miscellaneous awk topics. It describes how to execute UNIX commands from an awk script and how to direct output to files and pipes. It then offers some (meager) advice on debugging awk scripts.

Chapter 11, A Flock of awks, describes the original V7 version of awk, the current Bell Labs awk, GNU awk (gawk) from the Free Software Foundation, and mawk, by Michael Brennan. The latter three all have freely available source code. This chapter also describes two commercial implementations, MKS awk and Thomson Automation awk (tawk), as well as VSAwk, which brings awk-like capabilities to the Visual Basic environment.

Chapter 12, Full-Featured Applications presents two longer, more complex awk scripts that together demonstrate nearly all the features of the language. The first script is an interactive spelling checker. The second script processes and formats the index for a book or a master index for a set of books.

Chapter 13, A Miscellany of Scripts, presents a number of user-contributed scripts that show different styles and techniques of writing scripts for sed and awk.

Appendix A, Quick Reference for sea, is a quick reference describing sea's commands and command-line options.

Appendix B. Quick Reference for awk is a quick reference to awk's command-line options and a full description of its scripting language.

Appendix C, Supplement for Chapter 12, presents the full listings for the spellcheck.awk script and the masterindex shell script described in Chapter 12.

Availability of sed and awk
Sed and awk were part of Version 7 UNIX (also known as "V7," and "Seventh Edition") and have been part of the standard distribution ever since. Sed has been unchanged since it was introduced.

The Free Software Foundation GNU project's version of sed is freely available, although not technically in the public domain. Source code for GNU sed is available via anonymous FTP documentation. to the host ftp.gnu.ai.mit.edu. It is in the file /pub/gnu/sed-2.05.tar.gz. This is a tar file compressed with the gzip program, whose source code is available in the same directory. There are many sites worldwide that "mirror" the files from the main GNU distribution site; if you know of one close to you, you should get the files from there. Be sure to use "binary" or "image" mode to transfer the file(s).

In 1985, the authors of awk extended the language, adding many useful features. Unfortunately, this new version remained inside AT&T for several years. It became part of UNIX System V as of Release 3.1. It can be found under the name of nawk, for new awk; the older version still exists under its original name. This is still the case on System V Release 4 systems.

On commercial UNIX systems, such as those from Hewlett-Packard, Sun, IBM, Digital, and others, the naming situation is more complicated. All of these systems have some version of both old and new awk, but what each vendor names each program varies. Some have oawk and awk, others have awk and nawk. The best advice we can give is to check your local documentation. Throughout this book, we use the term awk to describe POSIX awk. Specific implementations will be referred to by name, such as "gawk," or "the Bell Labs awk."

Chapter 11 discusses three freely available awks (including where to get them), as well as several commercial ones.

DOS Versions
Gawk, mawk, and GNU sed have been ported to DOS. There are files on the main GNU distribution site with pointers to DOS versions of these programs. In addition, gawk has been ported to OS/2, VMS, and Atari and Amiga microcomputers, with ports to other systems (Macintosh, Windows) in progress.

egrep, sed, and awk are available for MS-DOS-based machines as part of the MKS Toolkit (Mortice Kern Systems, Inc., Ontario, Canada). Their implementation of awk supports the features of POSIX awk.

The MKS Toolkit also includes the Korn shell, which means that many shell scripts written for the Bourne shell on UNIX systems can be run on a PC. While most users of the MKS Toolkit have probably already discovered these tools in UNIX, we hope that the benefits of these programs will be obvious to PC users who have not ventured into UNIX.

Thompson Automation Software has an awk compiler for UNIX, DOS, and Microsoft Windows. This version is interesting because it has a number of extensions to the language, and it includes an awk debugger, written in awk!

We have used a PC on occasion because Ventura Publisher is a terrific formatting package. One of the reasons we like it is that we can continue to use vi to create and edit the text files and use sed for writing editing scripts. We have used sed to write conversion programs that translate troff macros into Ventura stylesheet tags. We have also used it to insert tags in batch mode. This can save having to manually tag repeated elements in a file.

Sed and awk are also useful for writing conversion programs that handle different file formats.

Other Sources of Information About sed and awk
For a long time, the main source of information on these utilities was two articles contained in Volume 2 of the UNIX Programmer's Guide. The article awk-A Pattern Scanning and Processing Language (September 1, 197X) was written by the language's three authors. In 10 pages, it offers a brief tutorial and discusses several design and implementation issues. The article SED-A Non-Interactive Text Editor (August 15, 1978) was written by Lee E. McMahon. It is a reference that gives a full description of each function and includes some useful examples (using Coleridge's Xanadu as sample input).

In trade books, the most significant treatment of sed and awk appears in The UNIX Programming Environment by Brian W. Kcrnighan and Rob Pike (Prentice-Hall, 1984). The chapter entitled "Filters" not only explains how these programs work but shows how they can work together to build useful applications.

The authors of awk collaborated on a book describing the enhanced version: The AWK Programming Language (Addison-Wesley, 1988). It contains many full examples and demonstrates the broad range of areas where awk can be applied. It follows in the style of the UNIX Programming Environment, which at times makes it too dense for some readers who are new users. The source code for the example programs in the book can be found in the directory /netlib/research/awkbookcode on netlib.bell-labs.com

The IEEE Standard for Information and Technology Portable Operating System Interface (POSIX) Part 2: Shell and Utilities (Standard 1003.2-1992) describes both sed and awk. It is the "official" word on the features available for portable shell programs that use sed and awk. Since awk is a programming language in its own right, it is also the official word on portable awk programs.

In 1996, the Free Software Foundation published The GNU Awk User's Guide, by Arnold Robbins. This is the documentation for gawk, written in a more tutorial style than the Aho, Kernighan, and Weinberger book. It has two full chapters of examples, and covers POSIX awk. This book is also published by SSC under the title Effective AWKProgramming, and the Texinfo source for the book comes with the gawk distribution.

It is one of the current deficiencies of GNU sed that it has no documentation of its own, not even a manpage.

Most general introductions to UNIX introduce sed and awk in a long parade of utilities. Of these books, Henry McGilton and Rachel Morgan's Introducing the UNIX System offers the best treatment of basic editing skills, including use of all UNIX text editors.

UNIX Text Processing (Hayden Books, 1987), by the original author of this handbook and Tim O'Reilly, covers sed and awk in full, although we did not include the new version of awk. Readers of that book will find some parts duplicated in this book, but in general a different approach has been taken here. Whereas in the textbook we treat sed and awk separately, expecting only advanced users to tackle awk, here we try to present both programs in relation to one another. They are different tools that can be used individually or together to provide interesting opportunities for text processing.

Finally, in 1995 the Usenet newsgroup comp.lang.awk came into being. If you can't find what you need to know in one of the above books, you can post a question in the newsgroup, with a good chance that someone will be able to help you.

The newsgroup also has a "frequently asked questions" (FAQ) article that is posted regularly. Besides answering questions about awk, the FAQ lists many sites where you can obtain binaries of different versions of awk for different systems. You can retrieve the FAQ via FT P in the file called /pub/usenet/comp.lang.awk/faq from the host rtfm.mit.edu.

Sample Programs
The sample programs in this book were originally written and tested on a Mac IIci running A/UX 2.0 (UNIX System V Release 2) and a SparcStation 1 running SunOS 4.0. Programs requiring POSIX awk were re-testecl using gawk 3.0.0 as well as the August 1994 version of the Bell Labs awk from the Bell Labs FTP site (see Chapter 11 for the FTP details). Sed programs were retested with the SunOS 4.1.3 sed and GNU sed 2.05.

Obtaining Example Source Code
You can obtain the source code for the programs presented in this book from O'Reilly & Associates through their Internet server. The example programs in this book are available electronically in a number of ways: by FTP, Ftpmail, BITFTP, and UUCP. The cheapest, fastest, and easiest ways are listed first. If you read from the top down, the first one that works for you is probably the best. Use FTP if you are directly on the Internet. Use Ftpmail if you are not on the Internet, hut can

send and receive electronic mail to Internet sites (this includes CompuServe users). Use BITFTP if you can send electronic mail via BITNET. Use UUCP if none of the above works.

FTP
To use FTP, you need a machine with direct access to the Internet. A sample session is shown, with what you should type in boldface.

$ ftp ftp.ora.com
Connected to ftp.ora.com.
220 FTP server (Version 6.21 Tue Mar 10 22:09:55 EST 1992) ready.
Name (ftp.ora.com:yourname): anonymous
331 Guest login ok, send domain style e-mail address as password.
Password: yourname@domain .name (Use your user name and host here)
230 Guest login ok, access restrictions apply.
ftp> cd /published/oreilly/nuthell/sedawk_2
250 CWD command successful.
ftp> binary (Very important! You must specify binary transfer for compressed files.)
203 Type set to I.
ftp> get progs.tar.gz
200 PORT command successful.
150 Opening BINARY mode data connection for progs.tar.gz.
226 Transfer complete.
ftp> quit
221 Goodbye.

The file is a gzip compressed tar archive; extract the files from the archive by typing

$ gzcat progs.tar.gz | tar xvf -

System V systems require the following tar command instead:

$ gzcat progs . .tar. gz | tar xof

If gzcat is not available on your system, use separate gunzip and tar commands.

$ gunzip progs.tar.gz
$ tar xvf progs.tar

Ftpmail
Ftpmail is a mail server available to anyone who can send electronic mail to and receive it from Internet sites. This includes any company or service provider that allows email connections to the Internet. Here's how you do it. You send mail to ftpmail@online.ora.com. In the message body? give the FTP commands you want to run. The server will run anonymous FTP for you and mail the files back to you.

To get a complete help file, send a message with no subject and the single word "help" in the body. The following is a sample mail session that should get you the examples. This command sends you a listing of the files in the selected directory and the requested example files. The listing is useful if there's a later version of the examples you're interested in.

$ mail ftpmail@online.ora.com
Subject:
reply-to yourname@domain.name (Where you want files mailed)
open
cd /published/oreilly/nutshell/sedawk_2
dir
mode binary
uuencode
get progs.tar.gz
quit

A signature at the end of the message is acceptable as long as it appears after "quit."

BITFTP
BITFTP is a mail server for BITNET users. You send it electronic mail messages requesting files, and it sends you back the files by electronic mail. BITFTP currently serves only users who send it mail from nodes that are directly on BITNET, EARN, or NetNorth. To use BITFTP, send mail containing your ftp commands to BITFTP@PUCC. For a complete help file, send HELP as the message body. The following is the message body you send to BITFTP:

FTP ftp.ora.com NETDATA
USER anonymous
PASS yourname@yourhost.edu Put your Internet email address here (not your BITNET address)
CD /published/oreilly/nutshell/sedawk_2
DIR
BINARY
GET progs.tar.gz
QUIT

Once you've got the desired file, follow the directions under FTP to extract the files from the archive. Since you are probably not on a UNIX system, you may need to get versions of uudecode, gunzip, atob, and tar for your system. VMS, DOS, and Mac versions are available.

UUCP
UUCP is standard on virtually all UNIX systems and is available for IBM-compatible PCs and Apple Macintoshes. The examples are available by UUCP via modem from UUNET; UUNET's connect-time charges apply. If you or your company has an account with UUNET, you have a system somewhere with a direct UUCP connection to UUNET. Find that system, and type:

uucp uunet\ ! ~ /published/oreilly/nutshell/sedawk_2/progs. tar. gz yourhost\ ! ~ /yo urname/

The backslashes can be omitted if you use a Bourne-style shell (sh, ksh, bash, zsh, pdksh) instead of csh. The file should appear some time later (up to a day or more) in the directory /usr/spool/uucppublic/yourname. If you don't have an account, but would like one so that you can get electronic mail, contact UUNET at 703-206-5400. It's a good idea to get the file /publtshed/oreilly/ls-lR.Z as a short test file containing the filenames and sizes of all the files available. Once you've got the desired file, follow the directions under FT P to extract the files from the archive

Conventions Used in This Handbook
The following conventions are used in this hook:

Bold is used for statements and functions, identifiers, and program names.

Italic is used for file and directory names when they appear in the body of a paragraph as well as for data types and to emphasize new terms and concepts when they are introduced.

Constant Widthis used in examples to show the contents of files or the output from commands.

Constant Boldis used in examples to show command lines and options that should be typedliterally by the user. (For example, rm foo means to type "rm foo" exactly as itappears in the text or the example.)

"" are used to identify a code fragment in explanatory text. System messages and symbols are quoted as well.

$ is the UNIX Bourne shell or Korn shell prompt.

[ ] surrounds optional elements in a description of program syntax. (The brackets themselves should never be typed, unless otherwise noted.)

... stands for text (usually computer output) that's been omitted for clarity or to save space.

indicates a literal space. This symbol is used to make spaces visible in examples, as well as in the text.

indicates a literal TAB character. This symbol is used to make tabs visible in examples, as well as in the text.

The notation CTRL-X or ^X indicates use of control characters. It means hold down the "control" key while typing the character "x". We denote other keys similarly (e.g., RETURN indicates a carriage return). All examples of command lines are followed by a RETURN unless otherwise indicated.

About the Second Edition
Since this book was first published in 1990, it has become one of the most fundamental of the O'Reilly & Associates Nutshell Handbooks. Three important events occurred after it was written. The first was the publication of the POSIX standard for sea, and more importantly for awk. The second (perhaps due to the first) was the widespread availability of some version or other of new awk on all modern UNIX systems, both commercial ones and the freely available UNIX like systems such as NetBSD, FreeBSD, and Linux. The third was the source code availability of GNU sea, and three versions of awk, instead of just gawk.

For these and other reasons, O'Reilly & Associates decided that this handbook needed to be updated. The goals of the revision were to keep the flavor of the book intact ("if it ain't broke, don't fix it"), reorient the awk part of the book around POSIX awk, correct mistakes, and bring the book up to date.

Customer Reviews

Average Rating 4.5

( 3 )

Rating Distribution

5 Star

(1)

4 Star

(2)

3 Star

(0)

2 Star

(0)

1 Star

(0)

Your Rating:

Review Guidelines

Tell the world what you think of this product.

Add Recommendations

Your Name: Create a Pen Name or Leave Anonymously

Barnes & Noble.com Review Rules

Our reader reviews allow you to share your comments on titles you liked, or didn't, with others. By submitting an online review, you are representing to Barnes & Noble.com that all information contained in your review is original and accurate in all respects, and that the submission of such content by you and the posting of such content by Barnes & Noble.com does not and will not violate the rights of any third party. Please follow the rules below to help ensure that your review can be posted.

Reviews by Our Customers Under the Age of 13

We highly value and respect everyone's opinion concerning the titles we offer. However, we cannot allow persons under the age of 13 to have accounts at BN.com or to post customer reviews. Please see our Terms of Use for more details.

What to exclude from your review:

Please do not write about reviews, commentary, or information posted on the product page. If you see any errors in the information on the product page, please send us an email.

Reviews should not contain any of the following:

- HTML tags, profanity, obscenities, vulgarities, or comments that defame anyone
- Time-sensitive information such as tour dates, signings, lectures, etc.
- Single-word reviews. Other people will read your review to discover why you liked or didn't like the title. Be descriptive.
- Comments focusing on the author or that may ruin the ending for others
- Phone numbers, addresses, URLs
- Pricing and availability information or alternative ordering information
- Advertisements or commercial solicitation

Reminder:

- By submitting a review, you grant to Barnes & Noble.com and its sublicensees the royalty-free, perpetual, irrevocable right and license to use the review in accordance with the Barnes & Noble.com Terms of Use.
- Barnes & Noble.com reserves the right not to post any review -- particularly those that do not follow the terms and conditions of these Rules. Barnes & Noble.com also reserves the right to remove any review at any time without notice.
- See Terms of Use for other conditions and disclaimers.

Anonymous

Posted May 17, 2000

Indispensible tools for text manipulation

Unix has earned itself quite a reputation for its potent tools, used for batch editing of text files (like program output). Sed and Awk are two of these tools. Sed is a direct descendent of Ed, the original Unix line editor, which employs regular expressions, a powerful method for description of patterns in text, for operations like substitute, append or delete. Awk is a complete scripting language with programming structures like conditionals, loops, functions etc., developed in 1970's by Alfred Aho, Brian Kernighan and Peter Weinberger (hence A-W-K). The trio has also written a book on Awk.
Dale Dougherty (in the 2nd edition with Arnold Robbins, maintainer of GNU Awk and author of several more books on Awk programming language) have made a good job in making a thoroughly readable tutorial on Sed and Awk. However, it remains a mystery to me how they succeeded to fill no less than 407 pages with it. Mind you, Sed and Awk are not really some big monsters. There exist something like two dozens of operators in Sed (most of them you will probably never use), and the syntax of Awk mimics those of C programming language, so it is likely that you know it already. Once you grok the idea of regular expressions, you should become a proficient user of Awk in about 30 minutes.
In conclusion, go buy the book if your need to manipulate text files on Unix and you think you need a lengthy tutorial with a gentle learning curve. Otherwise, short references on Awk and Sed, like the ones in Unix Power Tools and a bunch of examples showing some tricks you might not think of, will probably be more useful. In addition, it is good to know that during the nineties, much of the focus has drifted from Awk to Perl, so you might consider a book on Perl as well.
Was this review helpful? Yes NoThank you for your feedback. Report this reviewThank you, this review has been flagged.
Anonymous

Posted May 9, 2000

Wonderful Book

You will get sick of hearing from your co-workers how 'they gotta get this book'. It's that good. Both of these fantastic tools helped out as we needed to provide detailed file information during the latest ILOVEYOU virus fiasco on over 250 servers.
Was this review helpful? Yes NoThank you for your feedback. Report this reviewThank you, this review has been flagged.
Anonymous

Posted August 1, 2009

No text was provided for this review.

If you find inappropriate content, please report it to Barnes & Noble

sed and awk

Pick Up in Store

Overview

All Available Formats + Editions

Customers Who Bought This Also Bought

More About This Book

Overview

Related Subjects

Editorial Reviews

From Barnes & Noble

Fatbrain Review

Product Details

Meet the Author

Read an Excerpt

Table of Contents

Introduction

Customer Reviews

Rating Distribution

5 Star

4 Star

3 Star

2 Star

1 Star

Your Rating:

Your Recommendations:

Barnes & Noble.com Review Rules

Reviews by Our Customers Under the Age of 13

What to exclude from your review:

Reviews should not contain any of the following:

Reminder:

Recommend other products that relate to your review. Just search for them below and share!

Indispensible tools for text manipulation

Wonderful Book