Read an Excerpt
This book is intended to survey the most important computeralgorithms in use today and to teach fundamental techniques tothe growing number of people in need of knowing them. It can beused as a textbook for a second, third, or fourth course incomputer science, after students have acquired some programmingskills and familiarity with computer systems, but before theyhave taken specialized courses in advanced areas of computerscience or computer applications.
Additionally, the book may be useful for self-study or as areference for those engaged in the development of computersystems or applications programs, since it contains a number ofimplementations of useful algorithms and detailed information ontheir performance characteristics. The broad perspective takenin the book makes it an appropriate introduction to the field.
The algorithms are expressed in the C++ programming language(versions of the book in Pascal and C are also available). Nospecific knowledge about the language is assumedthe treatmenthere is self-contained (though fast-paced). Readers who havesome familiarity with C++ will find the language a useful vehiclefor learning a variety of methods of practical interest. Readerswho have some familiarity with basic algorithms will find thetreatment here a useful vehicle for learning a variety offeatures of the C++ language, while at the same time learningsome new algorithms.
The book contains forty-five chapters grouped into eight majorparts: fundamentals, sorting, searching, string processing,geometric algorithms, graph algorithms, mathematical algorithmsand advancedtopics. A major goal in developing this book hasbeen to bring together the fundamental methods from these diverseareas, in order to provide access to the best methods known forsolving problems by computer. Some of the chapters giveintroductory treatments of advanced material. It is hoped thatthe descriptions here can give readers some understanding of thebasic properties of fundamental algorithms ranging from priorityqueues and hashing to simplex and the fast Fourier transform.
One or two previous courses in computer science or equivalentprogramming experience are recommended for a reader to be able toappreciate the material in this book: one course in programmingin high-level languages such as C++, C or Pascal, and perhapsanother course which teaches fundamental concepts of programmingsystems. This book is thus intended for anyone conversant with amodern programming language and with the basic features of moderncomputer systems. References that might help fill in gaps inone's background are suggested in the text.
Most of the mathematical material supporting the analytic resultsis self-contained (or labeled as "beyond the scope" of thisbook), so little specific preparation in mathematics is requiredfor the bulk of the book, though a certain amount deal withalgorithms related to more advanced mathematical materialtheseare intended to place the algorithms in context with othermethods throughout the book, not to teach the mathematicalmaterial. Thus the discussion of advanced mathematical conceptsis brief, general, and descriptive.
USE IN THE CURRICULUM
There is a great deal of flexibility in how the material here canbe taught. To a large extent, the individual chapters in thebook can be read independently of the others, though in somecases algorithms in one chapter make use of methods from aprevious chapter. The material can be adapted for use forvarious courses by selecting perhaps 25 or 30 of the 45 chapters,according to the taste of the instructor and the preparation ofthe students.
The book begins with an introductory section on data structuresand the design and analysis of algorithms. This sets the tonefor the rest of the book and provides a framework within whichmore advanced algorithms are treated. Some readers may skip orskim this section; others may learn the basics there.
An elementary course on "data structures and algorithms" mightomit some of the mathematical algorithms and some of the advancedtopics, then emphasize how various data structures are used inthe implementations. An intermediate course on "design andanalysis of algorithms" might omit some of the more practicallyoriented sections, then emphasize the identification and study ofthe ways in which algorithms achieve good asymptotic performance. A course on "software tools" might omit the mathematical andadvanced algorithmic material, then emphasize how to integratethe implementations given here into large programs or systems. Acourse on "algorithms" might take a survey approach and introduceconcepts from all these areas.
Some instructors may wish to add supplementary material to thecourses described above to reflect their particular orientation. For "data structures and algorithms," more mathematical analysiscould be added; and for "software tools", software engineeringtechniques could be covered in more depth. In this book,attention is paid to all these areas, but the emphasis is on thealgorithms themselves.
Earlier versions of this book have been used in recent years atscores of colleges and universities around the country as a textfor the second or third course in computer science and assupplemental reading for other courses. At Princeton, ourexperience has been that the breadth of coverage of material inthis book provides our majors with an introduction to computerscience that can be expanded upon in later courses on analysis ofalgorithms, systems programming and theoretical computer science,while at the same time providing all the students with a largeset of techniques that they can immediately put to good use.
There are 450 exercises ten following each chapter, thatgenerally divide into one of two types. Most are intended totest students' understanding of material in the next and askstudents to work through an example or apply concepts describedin the text. A few of them, however, involve implementing andputting together some of the algorithms, perhaps runningempirical studies to compare algorithms and to learn theirproperties.
ALGORITHMS OF PRACTICAL USE
The orientation of the book is toward algorithms likely to be ofpractical use. The emphasis is on teaching students the tools oftheir trade to the point that they can confidently implement, runand debug useful algorithms. Full implementations of the methodsdiscussed are included in the text, along with descriptions ofthe operations of these programs on a consistent set of examples. Indeed, as discussed in the epilog, hundreds of figures areincluded in the book that have been created by the algorithmsthemselves. Many algorithms are brought to light on an intuitivelevel through the visual dimension provided by these figures.
Characteristics of the algorithms and situations in which theymight be useful are discussed in detail. Though not emphasized,connections to the analysis of algorithms and theoreticalcomputer science are not ignored. When appropriate, empiricaland analytic results are discussed to illustrate why certainalgorithms are preferred. When interesting, the relationship ofthe practical algorithms being discussed to purely theoreticalresults is described. Specific information performancecharacteristics of algorithms is encapsulated throughout in"properties," important facts about the algorithms that deservefurther study.
Some algorithms are used in relatively small programs to solvespecific problems. Others play an integral part in relativelylarge systems. Many fundamental algorithms find application inboth domains. We indicate as appropriate how algorithms might bespecialized for use as problem-solving tools or generalized forintegration into much bigger programs. Such considerations areof particular interest for algorithms expressed in anobject-oriented language such as C++. In this book we providerelevant information that can be used to make intelligenttradeoffs between utility and performance in implementing widelyapplicable algorithms.
While there is little direct treatment of specific uses of thealgorithms in science and engineering applications, the potentialfor such use is mentioned when appropriate. Our experience hasbeen that when students learn good algorithms in a computerscience context early in their education, they are able to applythem to solve problems when warranted later on.
The programming language used throughout the book is C++ (Pascaland C versions of the book are also available). Any particularlanguage has advantages and disadvantagesour intention here isto provide access to the fundamental algorithms that have beendeveloped over the years to the growing number of people who aremoving to C++ as a primary language and using it forapplications. The programs can easily be translated to othermodern programming languages, since they are written in astylized form that are relatively language-independent. Indeed,many of the programs have been translated from Pascal, C, andother languages, though we try to use standard C++ idioms whenappropriate. On the other hand, C++ is particularly well-suitedto our task, because its basic support for data abstraction andmodular programming allows us to clearly express relationshipsamong algorithms and data structures.
Some of the programs can simplified by using more advancedlanguage feature, but this is true less often than one might think. Although language features are discussed when appropriate, thisbook is not intended to be reference work on C++ or onobject-oriented programming. While we use C++ classes heavily,we do not use templates, inheritance or virtual functions. Butthe algorithms are coded so as to ease the process of embeddingthem in large systems where such features can be used toadvantage in an object-oriented programming approach. Whenforced to make a choice, we concentrate on the algorithms, notlanguage feature or implementations details.
A goal of this book is to present the algorithms in as simple anddirect a form as possible. The programs are intended to be readnot by themselves, but as part of the surrounding text. Thisstyle was chosen as an alternative, for example, to having inlinecomments. The style is consistent whenever possible, so thatprograms that are similar look similar.
Many people gave me helpful feedback on earlier versions of thisbook. In particular, students at Princeton and Brown sufferedthrough preliminary versions ofthe material in this book in the1980's. Special thanks are due to Trina Avery, Tom Freeman andJanet Incerpi for their help in producing the first edition. Iwould particularly like to thank Janet for converting the bookinto Tex format, adding the thousands of changes I made after the"last draft" of the first edition, guiding the files throughvarious systems to produce printed pages and even writing thescan-conversion routine for Tex used to produce draftmanuscripts, among many other things. Only after performing manyof these tasks myself for later versions do I truly appreciateJanet's contribution. I would also like to thank the manyreaders who provided me with detailed comments about the secondedition, including Guy Almes, Jay Gischer, Kennedy Lemke, UdiManber, Dana Richards, John Reif, M. Rosenfield, Stephen Seidman,and Michael Quinn.
Many of the designs in the figures are based on joint work withMarc Brown in the "electronic classroom" project at BrownUniversity in 1983. Marc's support and assistance in creatingthe designs (not tomention the system with which we worked) aregratefully acknowledged. I would like to acknowledge SarantosKapidakis' help in producing the endpapers.
This C++ version owes its existence to the persistence of KeithWollman who convinced me to proceed, and the patience of PeterGordon, who was confident that I would get around to doing so. Dave Hanson's willingness to answer questions about C and C++ wasinvaluable. I also would like to thank Darcy Cotten and SkipPlank for their help in producing the book.
Much of what I've written here I've learned from the teachingsand writings of Don Knuth, my advisor at Stanford. Though Donhad no direct influence on this work, his presence may be felt inthe book, for it was he who put the study of algorithms on ascientific footing that makes a work such as this possible.
I am very thankful for the support of Brown University and INRIAwhere I did most of the work on the book, and the Institute forDefense Analyses and the Xerox Palo Alto Research Center, where Idid some work on the book while visiting. Many parts of the bookare dependent on research that has been generously supported bythe National Science Foundation and the Office of Naval Research.Finally, I would like to thank Bill Bowen, Aaron Lemonick, andNeil Rudenstine at Princeton University for their support inbuilding an academic environment in which I was able to preparethis book, despite numerous other responsibilities.