Pub. Date:
Crypto: How the Code Rebels Beat the Government--Saving Privacy in the Digital Age

Crypto: How the Code Rebels Beat the Government--Saving Privacy in the Digital Age

by Steven Levy


View All Available Formats & Editions
Choose Expedited Shipping at checkout for delivery by Thursday, October 21


If you've ever made a secure purchase with your credit card over the Internet, then you have seen cryptography, or "crypto", in action. From Stephen Levy—the author who made "hackers" a household word—comes this account of a revolution that is already affecting every citizen in the twenty-first century. Crypto tells the inside story of how a group of "crypto rebels"—nerds and visionaries turned freedom fighters—teamed up with corporate interests to beat Big Brother and ensure our privacy on the Internet. Levy's history of one of the most controversial and important topics of the digital age reads like the best futuristic fiction.

Related collections and offers

Product Details

ISBN-13: 9780140244328
Publisher: Penguin Publishing Group
Publication date: 12/31/2001
Edition description: Reissue
Pages: 368
Sales rank: 695,726
Product dimensions: 5.38(w) x 7.95(h) x 1.01(d)

About the Author

Steven Levy is the author of Hackers, which has been in print for more than fifteen years, as well as Insanely Great: The Life & Times of Macintosh, the Computer That Changed Everything. He is also Newsweek's chief technology writer and has been a contributing writer to Wired since its inception. He lives in New York City with his wife and son.

Read an Excerpt

Chapter One

Mary Fischer loathed Whitfield Diffie on sight. He was a type she knew all too well, an MIT brainiac whose arrogance was a smoke screen for a massive personality disorder. The year of the meeting was 1969; the location a hardware store near Central Square in Cambridge, Massachusetts. Over his shoulder he carried a length of wire apparently destined for service as caging material for some sort of pet. This was a typical purchase for Diffie, whose exotic animal collection included a nine-foot python, a skunk, and a rare genetta genetta, a furry mongooselike creature whose gland secretions commonly evoked severe allergic reactions in people. It lived on a diet of live rats and at unpredictable moments would nip startled human admirers with needlelike fangs. An owner of such a creature would normally be of interest to Mary Fischer, an animal lover who at that very moment had a squirrel in her pocket. At home she also had a skunk as well as two dogs, a fox, a white-wing trumpeter bird, and two South American kinkajous. Diffie saw that she was buying some cage clips and abruptly focused his attention on her.

In future years, Whit Diffie would be known-extraordinarily well known-as the codiscoverer of public key cryptography, an iconographic figure with his shoulder-length blond hair, Buffalo Bill beard, and his bespoke suits cut by London tailors. But back in those days he was a wiry, crew-cut youth with "the angriest face I'd ever seen," Fischer says, and he immediately began peppering Mary Fischer with questions. You keep exotic animals? Then you'll need this, and this, and this. He took things out of her hands and putother things in as he lectured. His rudeness appalled Mary. But she hadn't yet cracked his code.

Mary Fischer didn't know that Diffie was spending prodigious amounts of time thinking about problems in computer security and their mathematical implications. She had no idea that he was casting about for a new way to preserve secrets. All she knew was that Whit Diffie was unappetizing and he loved animals. But animals meant a lot to her, and soon Diffie and his girlfriend began visiting Mary and her husband, sometimes accompanied by their creatures. The skunks got along, some ferrets were exchanged, and Diffie's visits to her home became routine.

Mary began to reconsider her initial repulsion to Diffie. But, in his failure to decode her, he seemed generally oblivious to her. On his visits he interacted only with the man of the house. After Mary and her husband moved to New Jersey, where he started veterinary school, she would sometimes pick up the ringing phone and hear Diffie's cuttingly precise voice brusquely ask for her spouse, as if she were an answering service. One day she made her feelings plain. "Look," she said, "I understand I'm not as bright as you and some of your friends, and I understand your friendship is primarily with my husband. But I don't really think it would kill you to say hello."

The message got through. Diffie's demeanor toward Mary dramatically improved, and she was not just startled but saddened when one day in 1971 he told her that he was going to travel for a while. Mary didn't know yet that Whit Diffie was preparing himself for a solitary-and romantic-quest, looking for answers to questions that the United States government didn't want asked. The odds against his success were astronomical, because he was confronting a near complete blockade of relevant information on a subject that, on its most sophisticated levels, was almost unspeakably obscure. What were the odds against such an unheralded outsider's transforming an entire field with an original discovery that would redefine the ground rules for personal privacy in the computer era?

The length of those odds would shorten with the role of Diffie's courtship of Mary Fischer in overcoming them-and a scientific breakthrough would result that affects every citizen in the digital age. "The discovery of public key," says Fischer, "was a romance."

Bailey Whitfield Diffie was born on the eve of D-Day, June 5, 1944. His professor father had just completed a wartime sabbatical in government service. (Though he disliked Communists-more for their humorless single-mindedness than their ideology-Whit Diffie's father was a passionate antifascist and often lectured against the repressive movement in Europe.) Both of Whit's parents were educated people. Bailey Wallace Diffie taught Iberian history and culture at City College in New York. Diffie's mother was the former Justine Louise Whitfield, a stockbroker's daughter from Tennessee who met her husband while working in the foreign service in Spain. She was a writer and scholar who studied Madame de Sévigné, a figure in the court of Louis XIII and Louis XIV.

Whit Diffie was always an independent sort. As one early friend remarked, "That kid had an alternative lifestyle at age five." Diffie didn't learn to read until he was ten years old. There was no question of disability; he simply preferred that his parents read to him, which seemingly they did, quite patiently. Apparently both parents understood that their son was extremely intelligent and obstinately contrarian, so they didn't press him. Finally, in the fifth grade, Diffie spontaneously worked his way through a tome called The Space Cat, and immediately progressed to the Oz books.

Later that year his teacher at P.S. 178-"Her name was Mary Collins, and if she is still alive I'd like to find her," Diffie would say decades later-spent an afternoon explaining something that would stick with him for a very long time: the basics of cryptography. Specifically, she described how one would go about solving something known as a substitution cipher.

Diffie found cryptography a delightfully conspiratorial means of expression. Its users collaborate to keep secrets in a world of prying eyes. A sender attempts this by transforming a private message to an altered state, a sort of mystery language: encryption. Once the message is transformed into a cacophonous babble, potential eavesdroppers are foiled. Only those in possession of the rules of transformation can restore the disorder back to the harmony of the message as it was first inscribed: decryption. Those who don't have that knowledge and try to decrypt messages without the secret "keys" are practicing "cryptanalysis."

A substitution cipher is one where someone creates ciphertext (the scrambled message) by switching the letters of the original message, or plaintext, with other letters according to a prearranged plan. The most basic of these has come to be known as the Caesar cipher, supposedly used by Julius Caesar himself. This system simply moved every character in the plaintext to the letter that occurs three notches later in the alphabet. (For instance, a Caesar cipher with its "key" of three would change A to D, B to E, and so on.) Slightly more challenging to an armchair cryptanalyst is a cryptosystem that matches every letter in the alphabet to one in a second, randomly rearranged alphabet. Newspaper pages often feature a daily "cryptogram" that encodes an aphorism or pithy quote in such a manner. These are by and large easy to crack because of the identifiable frequency of certain letters and the all-too-often predictable way they are distributed in words.

Like countless other curious young boys before him, Whit Diffie was thrilled by the process. In his history of cryptography, The Codebreakers, author David Kahn probes the emotional lures of secret writing, citing Freud's theory that the child's impulse to learn is tied to the desire to view the forbidden. "If you're a guy, you're trying to look up women's skirts," says Kahn. "When you get down to it basically, that's what it is, an urge to learn." For many, the fascination of crypto also deals with the thrill that comes from cracking encoded messages. Every intercepted ciphertext is, in effect, an invitation to assume the role of eavesdropper, intruder, voyeur. In any case, it wasn't the prospect of breaking codes that excited Whit Diffie but the more subtle pursuit of creating codes to protect information. "I never became a very good puzzle solver, and I never worked on solving codes very much then or later," he now says. He would always prefer keeping secrets to violating the secrets of others.

Diffie's response to Miss Collins's cryptography lesson was characteristic. He ignored her homework assignment, but independently pursued the subject in his own methodical, relentless fashion. He was particularly interested in her off-the-cuff remark that there were more complicated ciphers, including a foolproof "U.S. Code." He begged his father to check out all the books in the City College library that dealt with cryptography. Bailey Diffie promptly returned with an armload of books. Two of them were written for children; Diffie quickly devoured those. But then he got bogged down in Helen Forché Gaines's Cryptanalysis, a rather sophisticated 1939 tome.

Gaines offered a well-organized set of challenges that would provide hardworking amateurs an education in classical cryptographic systems. Many of these were refinements of advances made centuries ago, which in turn were more complicated variations of the earlier substitution ciphers. The best known were the polyalphabetic systems, first hatched in Vatican catacombs and later revealed in the early 1500s by a German monk named Johannes Trithemius. Published in 1518-two years after his death-Trithemius's Polygraphia introduced the use of tables, or tableaux, wherein each line was a separate, reshuffled alphabet. When you encoded your message, you transformed the first character of the text using the alphabet on the first line of the tableau. For the second character of your message you'd repeat the process with the scrambled alphabet on the second line, and so on.

On the heels of Trithemius came the innovations of a sixteenth-century French diplomat named Blaise de Vigenére. Here was a man who had penetrated the soul of crypto. "All things in the world constitute a cipher," he once observed. "All nature is merely a cipher and a secret writing." In the most famous of almost two dozen books he produced after his retirement from the diplomatic service, Vigenère produced devastating variations on previous polyalphabetic systems, adding complexity with a less predictable tableaux and "autokeys" that made use of the plaintext itself as a streaming key. The Vigenère system won a lasting reputation for security-it was known as le chiffre indéchiffrable-so much so that until almost the twentieth century, some armchair cryptographers believed that a certain streamlined version of the system was the sine qua non of cryptosystems.

Actually, by the time Diffie encountered them, the cryptologic arts had progressed dramatically since Vigenère. Still, Diffie's juvenile inquiries led him to think that Vigenère was the endpoint of the subject. Bored by the thought that cryptography was a problem already solved, he didn't delve too deeply into Gaines's book. His obsession with codes faded. At the time, he also felt that everybody was interested in codes, and, as a dogged contrarian, "this made it seem vulgar to me," he later recalled. "Instead, I learned about ancient fortifications, military maps, camouflage, poison gas, and germ warfare." He came to share his interests with a small group of teenage friends, and even considered pursuing a career in the armed forces, checking out the ROTC programs of universities he was interested in. But only one of Diffie's militia-minded clique actually enlisted in one of the armed services-and died in Vietnam.

Ultimately it was mathematics, not munitions, which dictated Diffie's choice of college. Math offered one thing that history did not: a sense of absolute truth. "I think that one of the central dilemmas of Whit's life has been to figure out what is really true," explains Mary Fischer, who says that early in the boy's life, Diffie's father was called to school and told that his son was a genius. As Fischer tells it, Bailey Diffie's reaction was to offer a ruse, in hopes that it would provoke discipline. He told Diffie that he wasn't as bright as other boys, but if he worked harder than those favored with high intelligence and applied himself, he might be able to achieve something. "With some children that might have worked," says Fischer, "but with Whit it was a bad tactic. It shook him for years, and I think it gave Whit a real hunger for what was ground-zero truth."

Though Diffie performed competently in school, he never did apply himself to the degree his father hoped. He was sometimes unruly in class; he worked best with material untainted by the stigma of having been assigned. Once a calculus teacher, fed up with Whit's noise-making, remarked, "One day you'll be roasting marshmallows in here!" and sure enough, the next class Diffie brought a Sterno canto toast the marshmallows a friend smuggled into school. He failedto fulfill the requirements for a full academic diploma, settling for a minimal distinction known as a general diploma. Nor did he attend graduation; he left with his father on a European trip. (The great tragedy of Diffie's high school years was the death of his mother; he still avoids talking about it.) Only stratospheric scores on standardized tests enabled him to enter the Massachusetts Institute of Technology in 1961.

"I wasn't a very good student there, either," Diffie admits. He was, however, dazzled by the brainpower of the student body, a collection of incandescent outcasts, visionaries, and prodigies, some of whom could solve in a minute problems that would take Diffie a day to complete. Of these mental luminaries, Whitfield Diffie might have seemed the least likely to produce a world-changing breakthrough. But since his brilliant friends were human beings and not high-powered automata, their trajectories proved far from predictable. Some of the very brightest wound up cycling through esoteric computer simulations, or proselytizing smart drugs, or teaching Transcendental Meditation.

Contemporaries from MIT recall Diffie vividly as a quirky teenager with blond hair sticking out from his head by two inches ("You wanted to take a lawn mower to it," says a friend). He bounded through campus on tiptoe, a weird walk that became an unmistakable signature in motion. But he was noted for his deep understanding of numbers as well.

He also took up computer programming-at first, Diffie now says, to get out of the draft. "I thought of computers as very low class," he says. "I thought of myself as a pure mathematician and was interested in partial differential equations and topology and things like that." But by 1965, when Diffie graduated from MIT, the Vietnam War was raging and he found himself deeply disenchanted with the trappings of armed conflict. "I had become a peacenik," he says. Not to mention a full-blown eccentric. He and his girlfriend lived in a small Cambridge apartment that eventually became packed with glass-walled tanks to hold their prodigious collection of exotic fauna. An aficionado of Chinese food, Diffie was also known for carrying around a pair of elegant chopsticks, much the way a serious billiard player totes his favorite cue.

To avoid the draft, Diffie accepted a job at the Mitre Corporation, which, as a defense contractor, could shelter its young employees from military service. His work had no direct connection to the war effort: he worked under a mathematician named Roland Silver, teaming up with another colleague to write a software package called Math-lab, which later evolved into a well-known symbolic mathematical manipulation system called Macsyma. (Though few knew of the nature of his contribution, the nerd cognoscenti understood that Diffie's work here involved a virtuosic mastery of arithmetic, numbers theory, and computer programming.)

Best of all, Diffie's team did not have to work at the Mitre offices but, in 1966, became a resident guest of the esteemed Marvin Minsky in the MIT artificial intelligence lab. During the three years he worked there, Diffie became part of this storied experiment in making machines smart, in pushing the frontiers of computer programming and in establishing an information-sharing ethos as the ground zero of computer culture. One aspect of this hacker-oriented society would turn out to be particularly relevant to the direction that Diffie's interests were heading. Just as some words in various languages have no meaning to drastically different civilizations (why would a tropical society need to speak of "snow"?), the AI lab had no technological equivalent for a term like "proprietary." Information was assumed to be as accessible as the air itself. As a consequence, there were no software locks on the operating system written by the MIT wizards.

Unlike his peers, however, Diffie believed that technology should offer a sense of privacy. And unlike some of his hacker colleagues, whose greatest kick came from playing in forbidden computer playgrounds, Diffie was drawn to questions of what software could be written to ensure that someone's files could not be accessed by intruders. To be sure, he participated in the literal safecracking that was a standard hobby in the AI lab: a favorite hacker pastime involved discovering new ways of opening government-approved secure safes. But Diffie got more of a kick from the protection of a strongly built safe than the rush of breaking a poorly designed system of locks and tumblers. He liked to keep his things in high-security filing cabinets and military safes.

In the information age, however, the ultimate information stronghold resides in software, not hardware: virtual safes protecting precious data. Information, after all, represents the treasure of the modern age, as valuable as all the doubloons and bangles of previous eras. The field charged with this responsibility back then was computer security, then in a nascent stage. Not many people bothered to discuss its philosophical underpinnings. But Diffie would often engage his boss in conversations on security. Inevitably, cryptography entered into their discussions.

Silver had some knowledge in the field, and the elder man opened Diffie's eyes to things unimaginable in his fifth-grade independent study. One day the pair sat in the cafeteria at Tech Square, the boxy nine-story building whose upper levels housed the AI lab, and Silver carefully explained to Diffie how modern cryptosystems worked.

Naturally, they depended on machinery. The machines that did the work-whether electromechanical devices like the Enigma cipher machines used by the Germans in World War II, or a contemporary computer-driven system-scrambled messages and documents by applying a unique recipe that would change the message, character by character. (The recipe for those transformations would be a set of complicated mathematical formulas or algorithms.) Only someone who had an identical machine or software program could reverse the process and divine the plaintext, with use of the special numerical key that had helped encrypt it.

In the case of the Enigma machines, that key involved "settings," the positions of the various code wheels that determined how each letter would be changed. Each day the encrypters would reset the wheels in a different way; those receiving the message would already have been informed of what those settings should be on that given day. That's why the Allied coup of recovering live Enigma machines-the key intelligence breakthrough of World War II-was only part of the elaborate codebreaking process that took place at Bletchley Park in England. The cryptanalysts also had to learn the process by which the Axis foes made their settings; then they could conduct what was known as a "brute force" attack that required going through all the possible combinations of settings. This could be efficiently done only by creating machines that were the forerunners of modern computers.

With computers, the equivalent of Enigma settings would become a digital key, a long string of numbers that would help determine how the system would transform the original message. Of course, the intended recipient of the message had to have not only the same computer program, but also that same key. But both mechanical and digital systems had two components: a so-called black box with the rules of transformation and a key that you'd feed into the black box along with your everyday message in plain English. Such was the background for what Silver talked about to Diffie that day-but not being privy to government secrets, he actually knew few of the details. He was able to explain, however, how computer cryptosystems generated a series of digits that represented a keystream, and how that would be "xor-ed" with the plaintext stream to get a ciphertext. (As any computer scientist knows, an xor operation involves pairing a digital bit with another bit, and generating a one or zero depending on whether they match.) If the key is suitably unpredictable, your output would be the most imponderable string of gibberish imaginable, recoverable (one hoped) only by using that same key to reverse the process.

Imponderable, of course, is a relative term, but those who devised cryptosystems had a standard to live up to: randomness. The idea was to create ciphertext that appeared to be as close to a random string of characters as possible. Otherwise, a smart, dedicated, and resourceful codebreaker could seize upon even the most subtle of patterns and eventually reconstruct the original message. A totally random stream could produce uncrackable code-this essentially represented the most secure form of encryption possible, the so-called one-time pad, a system that provided a truly randomly chosen substitute for every letter in the plaintext. One-time pads were the only cryptographic solution that was mathematically certain to be impervious to cryptanalysis.

The problem with one-time pads, however, was that for every character in the message, you needed a different number in the "key material" that originally transformed readable plaintext into jumbled ciphertext. In other words, a key for a one-time pad system had to be at least as long as the message and couldn't be used more than once. The unwieldiness of the process made it difficult to implement in the field. Even serious attempts to deploy one-time pads were commonly undermined by those tempted to save time and energy by reusing a pad.

His conversations with Silver excited Diffie. The subject of "pseudo-randomness" was clearly of importance to both the mathematical and real worlds, where security and privacy depended on the effectiveness of those codes. How close to randomness could we go? Obviously, there was a lot of work going on to discover the answer to that question-but the work was going on behind steep barriers erected and maintained by the government's intelligence agencies.

In fact, just about all the news about modern cryptography was behind that barrier. Everyone else had to rely on the same texts Whitfield Diffie had encountered in the fifth grade. And they didn't talk about how one went about changing the orderly procession of ones and zeros in a computer message to a different set of totally inscrutable ones and zeros using state-of-the-art stuff like Fibonacci generators, shift registers, or nonlinear feedback logic. Diffie resented this. "A well-developed technology is being kept secret!" he thought. He began to stew over this injustice. One day, walking with Silver along Mass Avenue near the railroad tracks, he spilled his concerns. Cryptography is vital to human privacy! he railed. Maybe, he suggested, passionate researchers in the public sector should attempt to liberate the subject. "If we put our minds to it," he told Silver, "we could rediscover a lot of that material." That is, they could virtually declassify it.

Silver was skeptical. "A lot of very smart people work at the NSA," he said, referring to the National Security Agency, the U.S. government's citadel of cryptography. After all, Silver explained, this organization had not only some of the best brains in the country, but billions of dollars in support. Its workers had years of experience and full access to recent cryptographic discoveries and techniques unknown to the hoi polloi-however intelligent-without high security clearances. The agency had supercomputers in its basement that made even MIT's state-of-the-art mainframe computers look like pocket calculators. How could outsiders like Diffie and Silver hope to match that?

Silver also told Diffie a story about his own NSA experience years earlier while writing a random number generator for the Digital Equipment Corporation's PDP-1 machine. He needed some information: his reasons were noncryptographic; he simply had a certain mathematical need, a polynomial number with some particular properties. He was sure that a friend of his at the NSA would know the answer instantly, and he put in a call. "Yes, I do know," said the friend. What was it? After a very long silence, during which Silver assumed that the friend was asking permission, the NSA scientist returned to the phone. Silver heard, in a conspiratorial whisper, "x to the twenty-fifth, plus x to the seventh, plus one."

Diffie was outraged at this secretiveness. He'd heard about the NSA, of course, but hadn't known that much about it. Just what was this organization, which acted as if it actually owned mathematical truths?

Created by President Truman's top-secret order in the fall of 1952, the National Security Agency was a multibillion-dollar organization that operated totally in the "black" region of government, where only those who could prove a "need to know" were entitled to knowledge. (It was not until five years after its founding that a government document even acknowledged its existence.) The NSA's cryptographic mission is twofold: to maintain the security of government information and to gather foreign intelligence. The double-sided nature of its duty led the NSA to organize itself into two major divisions: Communications Security, or COMSEC, which tries to devise codes that cannot be broken, and Communications Intelligence, or COMINT, which collects and decodes information from around the world. (Since the latter function most often involves intercepting and interpreting electronic information, it is more broadly referred to as signals intelligence, or SIGINT.) Over the years the NSA has established a vast network of listening devices and sensors to gather signals from even the most obscure reaches of the globe, an operation that expanded beyond the planetary atmosphere when the satellite era began in the 1960s.

In the early 1970s, none of this was discussed publicly. Within the Beltway, people in the know jokingly referred to the organizational acronym as No Such Agency. Those very few members of Congress who had oversight responsibility for intelligence funding would learn what had to be conveyed only in shielded rooms, swept for listening devices.

Access to the organization's headquarters at Fort George Meade, Maryland, was, as one might imagine, severely limited. A triple-barbed-wired and electrified fence kept outsiders at bay. To work within the gates, of course, one had to survive a rigid vetting.

"By joining NSA," reads the introduction to a handbook presented to new hires, "you have been given an opportunity to participate in the activities of one of the most important intelligence organizations of the United States government. At the same time you have assumed a trust which carries with it a most important individual responsibility-the safeguarding of sensitive information vital to the security of our nation."

Since all the salient information about modern crypto was withheld from public view, outsiders could only guess at what happened in "The Fort." The NSA undoubtedly operated the most sophisticated snooping operation in the world. It was universally assumed (though never admitted) that no foreign phone call, radio broadcast, or telegraph transmission was safe from the agency's global vacuum cleaner. Signals were sucked up and the content analyzed with multi-MIPS computers, combing the text for anything of value. (These suspicions were later confirmed with leaks of Project Echelon, the NSA's ambitious program to monitor foreign communications.) Were the results worth the billions of dollars and the questionable morality of the effort itself? This was something known only to the very few government officials who received briefings on the fabled intercepts-and even they were dependent on the quality of information that came from the agency itself.

What's more, the NSA considered itself the sole repository of cryptographic information in the country-not just that used by the civilian government and all the armed forces, as the law dictated, but that used by the private sector as well. Ultimately, the triple-depth electrified and barbed-wire fence surrounding its headquarters was not only a physical barrier but a metaphor for the NSA's near-fanatical drive to hide information about itself and its activities. In the United States of America, serious crypto existed only behind the Triple Fence.

Every day the NSA pored over new ideas for cryptographic systems submitted by would-be innovators in the field. "Their ideas disappear into the black maw of the NSA, and may see service in American cryptography," wrote David Kahn, "but security prevents the inventor from ever knowing this-and may enable the agency or its employees to utilize his ideas without compensation." But even those who did not submit ideas were not free of the NSA's stranglehold. The agency monitored all patent requests concerning cryptography and had the legal power to classify any of those it deemed too powerful to fall into the public domain.

As he learned more about the NSA, Whit Diffie came to feel a bit foolish that despite his having heard of the agency, the extent of its power had only belatedly dawned on him. Diffie had actually visited the Institute for Defense Analysis (IDA) at Princeton, a quasi-private outpost of the NSA, but he'd had only the vaguest idea about the organization's mission at the time. Not that it would have helped him get information from those crypto illuminati. One may socialize and even exchange thoughts with those who had ventured behind the Triple Fence, but only as long as those thoughts did not involve the forbidden subject of cryptography.

Cryptography, however, was exactly what Diffie wanted to talk about. He wanted to learn as much as he could, to have far-ranging conversations with the leaders in the field. Even the foot soldiers in the field would do. But he quickly became frustrated with those who would not, or could not, talk about it.

For instance, Diffie quizzed an MIT colleague named Dan Edwards, who would join the NSA after graduating. "He was extremely unhelpful," Diffie later reported, "failing to reveal things which were certainly not classified and which I later saw in the bibliography of his thesis." And when a colleague at Mitre went to work at IDA, Diffie asked him if he could share anything about his work. After a tantalizing pause: no.

Perhaps the idea of pursuing the forbidden was simply irresistible to a contrarian like Diffie. He kept thinking about crypto and the silent embargo against it. And the more he thought about the problem, the more he came to understand how deeply, deeply important the issue was. Especially in what he saw as the coming era of computational ubiquity. As more people used computers, wireless telephones, and other electronic devices, they would demand cryptography. Just as the invention of the telegraph upped the cryptographic ante by moving messages thousands of miles in the open, presenting a ripe opportunity for eavesdroppers of every stripe, the computer age would be moving billions of messages previously committed to paper into the realm of bits. Unencrypted, those bits were low-hanging fruit for snoopers. Could cryptography, that science kept intentionally opaque by the forces of government, help out? The answer was as clear as plaintext. Of course it could!

Right at MIT there was an excellent example of a need for a cryptographic solution to a big problem. The main computer system there was called Compatible Time Sharing System (CTSS). It was one of the first that used time-sharing, an arrangement by which several users could work on the machine simultaneously. Obviously, the use of a shared computer required some protocols to protect the privacy of each person's information. CTSS performed this by assigning a password to each user; his or her files would be in the equivalent of a locked mini-storage space, and each password would be the equivalent of the key that unlocked the door to that area. Passwords were distributed and maintained by a human being, the system operator. This central authority figure in essence controlled the privacy of every user. Even if he or she were scrupulously honest about protecting the passwords, the very fact that they existed within a centralized system provided an opportunity for compromise. Outside authorities had a clear shot at that information: simply present the system operator with a subpoena. "That person would sell you out," says Diffie, "because he had no interest in defying the order and going to jail to protect your data."

Diffie believed in what he called "a decentralized view of authority."

By creating the proper cryptographic tools, he felt, you could solve the problem-by transferring the data protection from a disinterested third party to the actual user, the one whose privacy was actually at risk. He fantasized about a company that would invent and implement such tools. He even had a name for this imaginary concern: Privacy Protection, Incorporated.

But in Diffie's fantasy, it was someone else who devised the solution, someone else who founded the company-not him. Though he was becoming absolutely sure that the problems of maintaining privacy in a non-crypto-protected world were insurmountable, he assumed that others would be better qualified, better motivated, more practically oriented than he to create the crypto to tackle such problems. So he tried to convince others to work on the solution. With little success. "None of the people I tried to get interested in the subject did anything," he recalls.

So Diffie kept working on his main interest, which lay in a mathematical problem called "proof of correctness." But he kept researching what he could on crypto, though at this point his efforts were far from methodical. One day at the Cambridge Public Library, Diffie was browsing the recent acquisitions and came across The Broken Seal by Ladislas Farago, a book about the pre-Pearl Harbor codebreaking efforts. He read a bit of it right there, and he certainly thought it worth reading further. But he never did. (Worse, he came to confuse this book with another book published at that time, David Kahn's The Codebreakers, which delayed his reading of the more important work.)

Similarly, one day at Mitre, a colleague moving out of his office gave Diffie a 1949 paper by Claude Shannon. The legendary father of information theory had been teaching at MIT since 1956, but Diffie had never met him, a slight, introverted professor who lived a quiet family life, pursuing a variety of interests from reading science fiction to listening to jazz. (Presumably, by the time Shannon had reached his sixties, he had put aside the unicycle he had once mastered.)

Shannon's impact on cryptography was considerable. After receiving an MIT doctorate in 1940, he had worked for Bell Telephone Laboratories during the war, specializing in secrecy systems. The work was classified, of course, but in the late part of the decade the two key papers in Shannon's wartime work found their way into the public domain. In 1948, Shannon's seminal article on information, "Mathematical Theory of Communication," ran in the Bell System Technical Journal, and subtly set the stage for the digital epoch. A year later, "Communication Theory of Secrecy Systems" appeared in the same journal.

Both efforts were highly technical; those without advanced math degrees could barely venture a few paragraphs without being snared in a thicket of thorny equations and formulas. But Shannon had a sense of clarity that enabled him to send a clear signal through the noise of high-level math. In the latter paper, he clearly and concisely examined the basic cryptographic relationship from scratch, addressing the "general mathematical structure and properties of secrecy systems." He even provided a diagram of the classic cryptanalytic situation, beginning with a box representing the original message. This was transformed by an "encipherer" with access to a "key source." The message would move to the "decipherer," who'd use the same key source to return the message to its original form. But there was another line branching out from the cryptogram. It led to the "enemy cryptanalyst," who might be able to intercept the encrypted message. That third party was always to be assumed. The challenge was to make it impossible for that enemy to crack the cryptogram.

The concepts of signal and noise loomed large in Shannon's view of cryptology. He saw crypto as a high-stakes zero-sum game between secret keeper and foe, where a successful secret was a signal that could not be teased out of the apparent noise. In his sixty-page discussion of the matter, he masterfully clarified the dilemma of both encrypter and enemy. The gift of the Shannon paper was undoubtedly one of the most valuable that a prospective cryptographer like Diffie could hope for in the late 1960s. Diffie himself would later consider it the last worthwhile unclassified paper published for over twenty years.

Too bad that Whit Diffie, still pursuing knowledge in a scattershot manner, waited several years before actually reading it.

In 1969, Diffie finally left Mitre. His funding had run out, and now that he was approaching the draft cutoff age, he had the freedom to leave. He had never really liked Cambridge very much. In high school, Diffie had hung out with the left-liberal and even the red diaper set, and led a full social life, with folk-singing parties and lots of friendly girls. Though similar scenes undoubtedly existed in Cambridge, "I just didn't find them," Diffie now moans. But at the University of California at Berkeley, where he spent a summer after his freshman year, Diffie found a place among the left-leaning protest crowd. "I really believe in the radical viewpoint," he says. "And I have always believed that one's politics and the character of his particular work are inseparable."

So Diffie and his girlfriend moved west, and Diffie went to work at John McCarthy's Stanford Artificial Intelligence Lab. Supposedly, he would continue working on proof of correctness and other mathematical problems that applied to computer science. But in conversations with McCarthy, Diffie was led into a deeper consideration of privacy concerns. A pioneer in time-sharing, McCarthy understood that soon computer terminals would find their way into the home. Inevitably, he believed, the nature of work itself would change, as the electronic office became something that moved out of the cloistered world of computer scientists and hackers and deep into the mainstream. This would open up not only a thicket of security problems, but also a host of novel challenges that almost no one was thinking about in 1969: If work products became electronic-produced on computer and sent over digital networks-how would people duplicate the customary forms of authentication (the means to verify that the author of a document was actually the person he or she claimed to be)? What would be the computerized version of a receipt? How could you get a computer-generated equivalent of a signed contract? Even if people were given unique "digital signatures"-say, a long, randomly generated number bequeathed to a single person-the nature of digital media, in which something can be copied in milliseconds, would seem to make such an identifier pointless. If you "signed" such a number to a contract, what would stop someone from simply scooping up the signature, making a perfect copy, and affixing it to other documents, contracts, and bank checks? If even the possibility of such unauthorized signed copies existed, the signature would be worthless. "I didn't sign this," someone could say. "Someone copied my signature!" Diffie began to wonder how one could begin to fix this apparently inherent flaw in the concept of digital commerce.

Diffie and McCarthy spent hours in rambling discussions on issues like authentication and the problems of distributing electronic keys. But Diffie still was more interested in letting others create the solution. In the summer of 1972, however, machinations in Washington, D.C., indirectly changed his course.

The government, under the aegis of the Defense Department's Advanced Research Projects Agency (ARPA), had recently begun a program to link major research institutions. This was known as the ARPAnet, a system that would one day transmogrify into today's Internet. ARPA's director of information-processing techniques, Larry Roberts, realized that such a computer network, the first computer net to link multiple sites and handle hundreds if not thousands of users, would need a way to keep messages secure, and the obvious way of doing that was to devise new crypto solutions. But when Roberts approached the NSA, he got a quick brush-off. Ultimately, he enlisted the help of Bolt Baranek Newman, the Boston-based company that helped set up ARPAnet in the first place. In the meantime, he mentioned the problem to his friend John McCarthy, who encouraged people at Stanford to concoct some crypto programs. They began working on what Diffie later called "a very complicated system combining the effects of several linear congruential random number generators."

Since Diffie's girlfriend was on that team, he also was drawn into the effort. Naturally, his curiosity led him to study the system closely. As he came to understand it, he found himself dissatisfied with its lack of efficiency. Diffie believed that if cryptography were to be used in a computer system, it was essential that users not have to suffer performance lags. Ideally, encryption should add but a tiny-or imperceptible-increment to the time it took to perform a function like copying a file. Diffie went over the group's basic encoding algorithm and eventually wrote a routine that ran much faster. In the process-now that he was actually doing some cryptography-he began to spend even more time thinking about the larger issue of how to advance the field. Later that year he went to Cambridge and saw Roland Silver again; Diffie now had much more hands-on expertise to bring to a discussion of crypto, and their rich exchange fueled his interest even more.

By now Diffie had finally gotten around to reading David Kahn's The Codebreakers. Since Diffie was a very slow, methodical reader, tackling a book of a thousand densely packed pages was a major undertaking for him. "He traveled everywhere with that book in hand," says his friend Harriet Fell. "If you invited him to dinner, he'd come with The Codebreakers." But Diffie found the hundreds of hours he spent on the book to be well worth the trouble.

Indeed, The Codebreakers was a landmark work-and one that the government had not wanted to see published. Kahn was a Newsday reporter who, as a twelve-year-old, had been thrilled, like Diffie and countless other boys, with his first exposure to the mysteries of secret writing. That moment first came on a visit to the local Great Neck (Long Island, New York) library, where the cover to a potboiler history called Secret and Urgent, by Fletcher Pratt, was on display. "This was about 1942 or '43," recalls Kahn. "That dust jacket was terrific; it had letters and numbers swirling out of the cosmos. I was hooked." The hook sank deeper when he actually read the book and learned about how ciphers worked. The youngster joined what was then probably the most sophisticated cryptography organization outside the government, the American Cryptogram Association. Which wasn't saying much. "It was a bunch of amateurs," he says. "They solved cryptograms as puzzles, and used a little publication with articles on how to solve them." Many of the members were elderly, or at least had time on their hands. There was even an offshoot called the Bedwarmers. "These were people with polio, or were in some sort of clinic, or were paralyzed," says Kahn. "They couldn't move around very well so they solved puzzles." Such was the scope of crypto work outside the government.

Unlike Diffie, however, Kahn loved to solve the puzzles himself, and kept his interest into adulthood. He discussed some sophisticated schemes with some fellow Cryptogram Association members. "Otherwise, you were totally isolated," he says. "This was an unknown field; nobody knew anything about it." But he didn't detect a more general interest in cryptography until 1961, when two NSA cryptographers defected to the USSR and held a press conference about their experience. This was revelatory to Kahn; despite diligently monitoring all the public literature about cryptography, he had hardly known that the NSA existed! Still, since he knew something about crypto, he dared to ask editors at the New York Times Magazine if they would like a backgrounder on the subject. They did, and he produced it.

The day after the story's publication, Kahn received three book contract offers. He turned them down since they were from paperback publishers and he wanted his work between boards. He got his wish a week later when an editor named Peter Ritner asked him to do a hardcover for Macmillan. Kahn wrote up an outline for a general book about codes, and received a $2000 advance. But as he began working on the introductory section, his research efforts kept kicking up more and more interesting stories from disparate sources. By the time he reached page 250 of his "preliminary chapter"-he had barely gotten to the Renaissance-he realized that he was really writing the comprehensive history of cryptology.

Two years into the project, Kahn quit his job to focus his efforts full time on the book. He lived off his savings, bunking at his parents' house and eating meals cooked by his grandmother. He wrote hundreds of letters, spent days in the New York Public Library, and, most important, connected with people who had never previously told their stories. A high-ranking Department of Defense official allowed him access to two important World War II codebreakers-an astonishing event given how Cold War politics decreed that revealing any information of this sort was virtually treason-if he agreed to submit his notes from the interviews to the government. "I guess the [Defense official] didn't know what he was getting into," reasons Kahn, "and when the notes got submitted to the NSA, the government panicked, and said I had to [disregard the information]. I respectfully declined."

Kahn also constructed, with the help of an important confidential source, the first public account of the extent of the NSA's power, constructing it from the bits and pieces that had dribbled out over the years. But the most explosive details of Kahn's book lay in its methodical explanation of how cryptography works, and how the NSA used it. When The Codebreakers was finished in 1965, it contained the most complete description of the operations of Fort Meade that had ever been compiled without an EYES-ONLY stamp on each page.

Quite correctly, officials at the National Security Agency had come to view Kahn's book as a literary hand grenade, with the potential for serious damage to the government's carefully maintained ramparts of secrecy. In his NSA exposé The Puzzle Palace, author James Bamford wrote that "innumerable hours of meetings and discussions, involving the highest levels of the agency, including the director, were spent in an attempt to sandbag the book." Countermeasures considered behind the Triple Fence ranged from outright purchase of the copyright to a break-in at Kahn's home. Kahn, who had moved to Paris to work for the Herald Tribune, was placed on the NSA's "watch list," enabling eavesdroppers to read his mail and monitor his conversations.

To Kahn's dismay, in March 1966 his editor sent the manuscript off to the Pentagon for its scrutiny and comments. Of course, it was then shipped to Fort Meade. The Defense Department wrote Macmillan's chairman that publishing The Codebreakers "would not be in the national interest." But Macmillan didn't bend, less because of backbone, Kahn guesses, than the fact that by that point in the production process "they had too much money put into it."

So the NSA took an extraordinary step. In July 1966, its director, Lt. Gen. Marshall S. Carter-a man so secretive that his name never appeared in newspapers-flew to New York City and met with the chairman of the publishing company, its legal counsel, and Kahn's editor, Peter Ritner. After attacking Kahn's reputation and expertise, Carter finally made a personal appeal for three specific deletions. A few days later, Ritner presented Kahn with the request. The actual deletions struck Kahn as surprisingly inconsequential. "It didn't really hurt the book, so I took the three things out," Kahn says. "But I insisted that we put in a statement to the effect that the book had been submitted to the Department of Defense. In the end that had a good effect, because right-wing reviewers could otherwise have said the book was destroying the republic. Now they couldn't."

While The Codebreakers never made the New York Times bestseller list, it became a steady seller, going through dozens of printings. And it did not, as the NSA had hysterically predicted, bring an abrupt close to the American century. It did, however, enlighten a new generation of cryptographers who would dare to work outside of the government's wall of secrecy. And its prime student was Whitfield Diffie.

"I read it more carefully than anyone had ever read it.... Kahn's book to me is like the Vedas," he explains, citing the centuries-old Indian text. "There's an expression I learned: 'If a man loses his cow, he looks for it in the Vedas.'"

By the time Whitfield Diffie finished The Codebreakers, he was no longer depending on others to tackle the great problems of cryptography. He was personally, passionately engaged in them himself. They consumed his waking dreams. They were now his obsession.

Why had Diffie's once-intermittent interest become such a consuming passion? Behind every great cryptographer, it seems, there is a driving pathology. Though Diffie's quest was basically an intellectual challenge, he had come to take it very personally. Beneath his casual attire and streaming blond hair, Diffie was a proud and determined man. He had an unusual drive for getting at what he considered the bedrock truth of any issue. This led to a fascination with protecting and uncovering secrets, especially important secrets that were desperately held. "Ostensibly, my reason for getting interested in this was its importance to personal privacy," he now says. "But I was also fascinated with investigating this business that people wouldn't tell you about." It was as if solving this conundrum would provide a more general meaning to the world at large. "I guess in a very real sense I'm a Gnostic," he says. "I had been looking all my life for some great mystery.... I think somewhere deep in my mind is the notion that if I could learn just the right thing, I would be saved."

And then, Diffie's quest to discover truths in cryptography became intertwined with another sort of romance: his courtship of Mary Fischer.

Table of Contents

The Loner
The Standard
Public Key
Prime Time
Selling Crypto
Patents and Keys
Crypto Anarchy
The Clipper Chip
Slouching Toward Crypto
Epilogue: The Open Secret

What People are Saying About This

From the Publisher

"Gripping and illuminating." —The Wall Street Journal

"A great David-and-Goliath story—humble hackers hoodwink sinister spooks." —Time

Customer Reviews