A pleasure to read. (The Wall Street Journal)
The best person I know at predicting the future of artificial intelligence (Bill Gates)
For over three decades, Ray Kurzweil has been one of the most respected and provocative advocates of the role of technology in our future. In his classic The Age of Spiritual Machines, he argued that computers would soon rival the full range of human intelligence at its best. Now he examines the next step in this inexorable evolutionary process: the/i>… See more details below
For over three decades, Ray Kurzweil has been one of the most respected and provocative advocates of the role of technology in our future. In his classic The Age of Spiritual Machines, he argued that computers would soon rival the full range of human intelligence at its best. Now he examines the next step in this inexorable evolutionary process: the union of human and machine, in which the knowledge and skills embedded in our brains will be combined with the vastly greater capacity, speed, and knowledge-sharing ability of our creations.
A pleasure to read. (The Wall Street Journal)
The best person I know at predicting the future of artificial intelligence (Bill Gates)
Praise for The Singularity Is Near
One of CBS News’s Best Fall Books of 2005
Among St Louis Post-Dispatch’s Best Nonfiction Books of 2005
One of Amazon.com’s Best Science Books of 2005
“Anyone can grasp Mr. Kurzweil’s main idea: that mankind’s technological knowledge has been snowballing, with dizzying prospects for the future. The basics are clearly expressed. But for those more knowledgeable and inquisitive, the author argues his case in fascinating detail…. The Singularity Is Near is startling in scope and bravado.”
—Janet Maslin, The New York Times
“Filled with imaginative, scientifically grounded speculation…. The Singularity Is Near is worth reading just for its wealth of information, all lucidly presented…. [It’s] an important book. Not everything that Kurzweil predicts may come to pass, but a lot of it will, and even if you don’t agree with everything he says, it’s all worth paying attention to.”
—The Philadelphia Inquirer
“[An] exhilarating and terrifyingly deep look at where we are headed as a species…. Mr. Kurzweil is a brilliant scientist and futurist, and he makes a compelling and, indeed, a very moving case for his view of the future.”
—The New York Sun
—San Jose Mercury News
“Kurzweil links a projected ascendance of artificial intelligence to the future of the evolutionary process itself. The result is both frightening and enlightening…. The Singularity Is Near is a kind of encyclopedic map of what Bill Gates once called ‘the road ahead.’”
“A clear-eyed, sharply-focused vision of the not-so-distant future.”
—The Baltimore Sun
“This book offers three things that will make it a seminal document. 1) It brokers a new idea, not widely known, 2) The idea is about as big as you can get: the Singularity—all the change in the last million years will be superceded by the change in the next five minutes, and 3) It is an idea that demands informed response. The book’s claims are so footnoted, documented, graphed, argued, and plausible in small detail, that it requires the equal in response. Yet its claims are so outrageous that if true, it would mean … well … the end of the world as we know it, and the beginning of utopia. Ray Kurzweil has taken all the strands of the Singularity meme circulating in the last decades and has united them into a single tome which he has nailed on our front door. I suspect this will be one of the most cited books of the decade. Like Paul Ehrlich’s upsetting 1972 book Population Bomb, fan or foe, it’s the wave at epicenter you have to start with.”
—Kevin Kelly, founder of Wired
“Really, really out there. Delightfully so.”
“Stunning, utopian vision of the near future when machine intelligence outpaces the biological brain and what things may look like when that happens…. Approachable and engaging.”
—the unofficial Microsoft blog
“One of the most important thinkers of our time, Kurzweil has followed up his earlier works … with a work of startling breadth and audacious scope.”
“An attractive picture of a plausible future.”
“Kurzweil is a true scientist—a large-minded one at that…. What’s arresting isn’t the degree to which Kurzweil’s heady and bracing vision fails to convince—given the scope of his projections, that’s inevitable—but the degree to which it seems downright plausible.”
—Publishers Weekly (starred review)
“[T]hroughout this tour de force of boundless technological optimism, one is impressed by the author’s adamantine intellectual integrity…. If you are at all interested in the evolution of technology in this century and its consequences for the humans who are creating it, this is certainly a book you should read.”
—John Walker, inventor of Autodesk, in Fourmilab Change Log
“Ray Kurzweil is the best person I know at predicting the future of artificial intelligence. His intriguing new book envisions a future in which information technologies have advanced so far and fast that they enable humanity to transcend its biological limitations—transforming our lives in ways we can’t yet imagine.”
“If you have ever wondered about the nature and impact of the next profound discontinuities that will fundamentally change the way we live, work, and perceive our world, read this book. Kurzweil’s Singularity is a tour de force, imagining the unimaginable and eloquently exploring the coming disruptive events that will alter our fundamental perspectives as significantly as did electricity and the computer.”
—Dean Kamen, recipient of the National Medal of Technology,
physicist, and inventor of the first wearable insulin pump, the
HomeChoice portable dialysis machine, the IBOT Mobility
System, and the Segway Human Transporter
“One of our leading AI practitioners, Ray Kurzweil, has once again created a ‘must read’ book for anyone interested in the future of science, the social impact of technology, and indeed the future of our species. His thought-provoking book envisages a future in which we transcend our biological limitations, while making a compelling case that a human civilization with superhuman capabilities is closer at hand than most people realize.”
—Raj Reddy, founding director of the Robotics Institute at
Carnegie Mellon University and recipient of the Turing Award from the Association for Computing Machinery
“Ray’s optimistic book well merits both reading and thoughtful response. For those like myself whose views differ from Ray’s on the balance of promise and peril, The Singularity Is Near is a clear call for a continuing dialogue to address the greater concerns arising from these accelerating possibilities.”
—Bill Joy, cofounder and former chief scientist, Sun Microsystems
About the Author
Ray Kurzweil is one of the world’s leading inventors, thinkers, and futurists, with a twenty-year track record of accurate predictions. Called “the restless genius” by The Wall Street Journal and “the ultimate thinking machine” by Forbes magazine, Kurzweil was selected as one of the top entrepreneurs by Inc. magazine, which described him as the “rightful heir to Thomas Edison.” PBS selected him as one of “sixteen revolutionaries who made America,” along with other inventors of the past two centuries. An inductee into the National Inventors Hall of Fame and recipient of the National Medal of Technology, the Lemelson-MIT Prize (the world’s largest award for innovation), thirteen honorary doctorates, and awards from three U.S. presidents, he is the author of four previous books: Fantastic Voyage: Live Long Enough to Live Forever (coauthored with Terry Grossman, M.D.), The Age of Spiritual Machines, The 10% Solution for a Healthy Life, and The Age of Intelligent Machines.
The Singularity Is Near
WHEN HUMANS TRANSCEND BIOLOGY
To my mother, Hannah,
who provided me with the courage to seek the ideas to confront any challenge
I’d like to express my deep appreciation to my mother, Hannah, and my father, Fredric, for supporting all of my early ideas and inventions without question, which gave me the freedom to experiment; to my sister Enid for her inspiration; and to my wife, Sonya, and my kids, Ethan and Amy, who give my life meaning, love, and motivation.
I’d like to thank the many talented and devoted people who assisted me with this complex project:
At Viking: my editor, Rick Kot, who provided leadership, enthusiasm, and insightful editing; Clare Ferraro, who provided strong support as publisher; Timothy Mennel, who provided expert copyediting; Bruce Giffords and John Jusino, for coordinating the many details of book production; Amy Hill, for the interior text design; Holly Watson, for her effective publicity work; Alessandra Lusardi, who ably assisted Rick Kot; Paul Buckley, for his clear and elegant art design; and Herb Thornby, who designed the engaging cover.
Loretta Barrett, my literary agent, whose enthusiastic and astute guidance helped guide this project.
Terry Grossman, M.D., my health collaborator and coauthor of Fantastic Voyage: Live Long Enough to Live Forever, for helping me to develop my ideas on health and biotechnology through 10,000 e-mails back and forth, and a multifaceted collaboration.
Martine Rothblatt, for her dedication to all of the technologies discussed in this book and for our collaboration in developing diverse technologies in these areas.
Aaron Kleiner, my long-term business partner (since 1973), for his devotion and collaboration through many projects, including this one.
Amara Angelica, whose devoted and insightful efforts led our research team. Amara also used her outstanding editing skills to assist me in articulating the complex issues in this book. Kathryn Myronuk, whose dedicated research efforts made a major contribution to the research and the notes. Sarah Black contributed discerning research and editorial skills. My research team provided very capable assistance: Amara Angelica, Kathryn Myronuk, Sarah Black, Daniel Pentlarge, Emily Brown, Celia Black-Brooks, Nanda Barker-Hook, Sarah Brangan, Robert Bradbury, John Tillinghast, Elizabeth Collins, Bruce Damer, Jim Rintoul, Sue Rintoul, Larry Klaes, and Chris Wright. Additional assistance was provided by Liz Berry, Sarah Brangan, Rosemary Drinka, Linda Katz, Lisa Kirschner, Inna Nirenberg, Christopher Setzer, Joan Walsh, and Beverly Zibrak.
Laksman Frank, who created many of the attractive diagrams and images from my descriptions, and formatted the graphs.
Celia Black-Brooks, for providing her leadership in project development and communications.
Phil Cohen and Ted Coyle, for implementing my ideas for the illustration on page 322, and Helene DeLillo, for the “Singularity Is Near” photo at the beginning of chapter 7.
Nanda Barker-Hook, Emily Brown, and Sarah Brangan, who helped manage the extensive logistics of the research and editorial processes.
Ken Linde and Matt Bridges, who provided computer systems support to keep our intricate work flow progressing smoothly.
Denise Scutellaro, Joan Walsh, Maria Ellis, and Bob Beal, for doing the accounting on this complicated project.
The KurzweilAI.net team, who provided substantial research support for the project: Aaron Kleiner, Amara Angelica, Bob Beal, Celia Black-Brooks, Daniel Pentlarge, Denise Scutellaro, Emily Brown, Joan Walsh, Ken Linde, Laksman Frank, Maria Ellis, Matt Bridges, Nanda Barker-Hook, Sarah Black, and Sarah Brangan.
Mark Bizzell, Deborah Lieberman, Kirsten Clausen, and Dea Eldorado, for their assistance in communication of this book’s message.
Robert A. Freitas Jr., for his thorough review of the nanotechnology-related material.
Paul Linsay, for his thorough review of the mathematics in this book.
My peer expert readers who provided the invaluable service of carefully reviewing the scientific content: Robert A. Freitas Jr. (nanotechnology, cosmology), Ralph Merkle (nanotechnology), Martine Rothblatt (biotechnology, technology acceleration), Terry Grossman (health, medicine, biotechnology), Tomaso Poggio (brain science and brain reverse-engineering), John Parmentola (physics, military technology), Dean Kamen (technology development), Neil Gershenfeld (computational technology, physics, quantum mechanics), Joel Gershenfeld (systems engineering), Hans Moravec (artificial intelligence, robotics), Max More (technology acceleration, philosophy), Jean-Jacques E. Slotine (brain and cognitive science), Sherry Turkle (social impact of technology), Seth Shostak (SETI, cosmology, astronomy), Damien Broderick (technology acceleration, the Singularity), and Harry George (technology entrepreneurship).
My capable in-house readers: Amara Angelica, Sarah Black, Kathryn Myronuk, Nanda Barker-Hook, Emily Brown, Celia Black-Brooks, Aaron Kleiner, Ken Linde, John Chalupa, and Paul Albrecht.
My lay readers, who provided keen insights: my son, Ethan Kurzweil, and David Dalrymple.
Bill Gates, Eric Drexler, and Marvin Minsky, who gave permission to include their dialogues in the book, and for their ideas, which were incorporated into the dialogues.
The many scientists and thinkers whose ideas and efforts are contributing to our exponentially expanding human knowledge base.
The above-named individuals provided many ideas and corrections that I was able to make thanks to their efforts. For any mistakes that remain, I take sole responsibility.
The Singularity Is Near
The Power of Ideas
I do not think there is any thrill that can go through the human heart like that felt by the inventor as he sees some creation of the brain unfolding to success.
—NIKOLA TESLA, 1896, INVENTOR OF ALTERNATING CURRENT
At the age of five, I had the idea that I would become an inventor. I had the notion that inventions could change the world. When other kids were wondering aloud what they wanted to be, I already had the conceit that I knew what I was going to be. The rocket ship to the moon that I was then building (almost a decade before President Kennedy’s challenge to the nation) did not work out. But at around the time I turned eight, my inventions became a little more realistic, such as a robotic theater with mechanical linkages that could move scenery and characters in and out of view, and virtual baseball games.
Having fled the Holocaust, my parents, both artists, wanted a more worldly, less provincial, religious upbringing for me.1 My spiritual education, as a result, took place in a Unitarian church. We would spend six months studying one religion—going to its services, reading its books, having dialogues with its leaders—and then move on to the next. The theme was “many paths to the truth.” I noticed, of course, many parallels among the world’s religious traditions, but even the inconsistencies were illuminating. It became clear to me that the basic truths were profound enough to transcend apparent contradictions.
At the age of eight, I discovered the Tom Swift Jr. series of books. The plots of all of the thirty-three books (only nine of which had been published when I started to read them in 1956) were always the same: Tom would get himself into a terrible predicament, in which his fate and that of his friends, and often the rest of the human race, hung in the balance. Tom would retreat to his basement lab and think about how to solve the problem. This, then, was the dramatic tension in each book in the series: what ingenious idea would Tom and his friends come up with to save the day?2 The moral of these tales was simple: the right idea had the power to overcome a seemingly overwhelming challenge.
To this day, I remain convinced of this basic philosophy: no matter what quandaries we face—business problems, health issues, relationship difficulties, as well as the great scientific, social, and cultural challenges of our time—there is an idea that can enable us to prevail. Furthermore, we can find that idea. And when we find it, we need to implement it. My life has been shaped by this imperative. The power of an idea—this is itself an idea.
Around the same time that I was reading the Tom Swift Jr. series, I recall my grandfather, who had also fled Europe with my mother, coming back from his first return visit to Europe with two key memories. One was the gracious treatment he received from the Austrians and Germans, the same people who had forced him to flee in 1938. The other was a rare opportunity he had been given to touch with his own hands some original manuscripts of Leonardo da Vinci. Both recollections influenced me, but the latter is one I’ve returned to many times. He described the experience with reverence, as if he had touched the work of God himself. This, then, was the religion that I was raised with: veneration for human creativity and the power of ideas.
In 1960, at the age of twelve, I discovered the computer and became fascinated with its ability to model and re-create the world. I hung around the surplus electronics stores on Canal Street in Manhattan (they’re still there!) and gathered parts to build my own computational devices. During the 1960s, I was as absorbed in the contemporary musical, cultural, and political movements as my peers, but I became equally engaged in a much more obscure trend: namely, the remarkable sequence of machines that IBM proffered during that decade, from their big “7000” series (7070, 7074, 7090, 7094) to their small 1620, effectively the first “minicomputer.” The machines were introduced at yearly intervals, and each one was less expensive and more powerful than the last, a phenomenon familiar today. I got access to an IBM 1620 and began to write programs for statistical analysis and subsequently for music composition.
I still recall the time in 1968 when I was allowed into the secure, cavernous chamber housing what was then the most powerful computer in New England, a top-of-the-line IBM 360 Model 91, with a remarkable million bytes (one megabyte) of “core” memory, an impressive speed of one million instructions per second (one MIPS), and a rental cost of only one thousand dollars per hour. I had developed a computer program that matched high-school students to colleges, and I watched in fascination as the front-panel lights danced through a distinctive pattern as the machine processed each student’s application.3 Even though I was quite familiar with every line of code, it nonetheless seemed as if the computer were deep in thought when the lights dimmed for several seconds at the denouement of each such cycle. Indeed, it could do flawlessly in ten seconds what took us ten hours to do manually with far less accuracy.
As an inventor in the 1970s, I came to realize that my inventions needed to make sense in terms of the enabling technologies and market forces that would exist when the inventions were introduced, as that world would be a very different one from the one in which they were conceived. I began to develop models of how distinct technologies—electronics, communications, computer processors, memory, magnetic storage, and others—developed and how these changes rippled through markets and ultimately our social institutions. I realized that most inventions fail not because the R&D department can’t get them to work but because the timing is wrong. Inventing is a lot like surfing: you have to anticipate and catch the wave at just the right moment.
My interest in technology trends and their implications took on a life of its own in the 1980s, and I began to use my models to project and anticipate future technologies, innovations that would appear in 2000, 2010, 2020, and beyond. This enabled me to invent with the capabilities of the future by conceiving and designing inventions using these future capabilities. In the mid-to-late 1980s, I wrote my first book, The Age of Intelligent Machines.4 It included extensive (and reasonably accurate) predictions for the 1990s and 2000s, and ended with the specter of machine intelligence becoming indistinguishable from that of its human progenitors within the first half of the twenty-first century. It seemed like a poignant conclusion, and in any event I personally found it difficult to look beyond so transforming an outcome.
Over the last twenty years, I have come to appreciate an important meta-idea: that the power of ideas to transform the world is itself accelerating. Although people readily agree with this observation when it is simply stated, relatively few observers truly appreciate its profound implications. Within the next several decades, we will have the opportunity to apply ideas to conquer age-old problems—and introduce a few new problems along the way.
During the 1990s, I gathered empirical data on the apparent acceleration of all information-related technologies and sought to refine the mathematical models underlying these observations. I developed a theory I call the law of accelerating returns, which explains why technology and evolutionary processes in general progress in an exponential fashion.5 In The Age of Spiritual Machines (ASM), which I wrote in 1998, I sought to articulate the nature of human life as it would exist past the point when machine and human cognition blurred. Indeed, I’ve seen this epoch as an increasingly intimate collaboration between our biological heritage and a future that transcends biology.
Since the publication of ASM, I have begun to reflect on the future of our civilization and its relationship to our place in the universe. Although it may seem difficult to envision the capabilities of a future civilization whose intelligence vastly outstrips our own, our ability to create models of reality in our mind enables us to articulate meaningful insights into the implications of this impending merger of our biological thinking with the nonbiological intelligence we are creating. This, then, is the story I wish to tell in this book. The story is predicated on the idea that we have the ability to understand our own intelligence—to access our own source code, if you will—and then revise and expand it.
Some observers question whether we are capable of applying our own thinking to understand our own thinking. AI researcher Douglas Hofstadter muses that “it could be simply an accident of fate that our brains are too weak to understand themselves. Think of the lowly giraffe, for instance, whose brain is obviously far below the level required for self-understanding—yet it is remarkably similar to our brain.”6 However, we have already succeeded in modeling portions of our brain—neurons and substantial neural regions—and the complexity of such models is growing rapidly. Our progress in reverse engineering the human brain, a key issue that I will describe in detail in this book, demonstrates that we do indeed have the ability to understand, to model, and to extend our own intelligence. This is one aspect of the uniqueness of our species: our intelligence is just sufficiently above the critical threshold necessary for us to scale our own ability to unrestricted heights of creative power—and we have the opposable appendage (our thumbs) necessary to manipulate the universe to our will.
A word on magic: when I was reading the Tom Swift Jr. books, I was also an avid magician. I enjoyed the delight of my audiences in experiencing apparently impossible transformations of reality. In my teen years, I replaced my parlor magic with technology projects. I discovered that unlike mere tricks, technology does not lose its transcendent power when its secrets are revealed. I am often reminded of Arthur C. Clarke’s third law, that “any sufficiently advanced technology is indistinguishable from magic.”
Consider J. K. Rowling’s Harry Potter stories from this perspective. These tales may be imaginary, but they are not unreasonable visions of our world as it will exist only a few decades from now. Essentially all of the Potter “magic” will be realized through the technologies I will explore in this book. Playing quid-ditch and transforming people and objects into other forms will be feasible in full-immersion virtual-reality environments, as well as in real reality, using nanoscale devices. More dubious is the time reversal (as described in Harry Potter and the Prisoner of Azkaban), although serious proposals have even been put forward for accomplishing something along these lines (without giving rise to causality paradoxes), at least for bits of information, which essentially is what we comprise. (See the discussion in chapter 3 on the ultimate limits of computation.)
Consider that Harry unleashes his magic by uttering the right incantation. Of course, discovering and applying these incantations are no simple matters. Harry and his colleagues need to get the sequence, procedures, and emphasis exactly correct. That process is precisely our experience with technology. Our incantations are the formulas and algorithms underlying our modern-day magic. With just the right sequence, we can get a computer to read a book out loud, understand human speech, anticipate (and prevent) a heart attack, or predict the movement of a stock-market holding. If an incantation is just slightly off mark, the magic is greatly weakened or does not work at all.
One might object to this metaphor by pointing out that Hogwartian incantations are brief and therefore do not contain much information compared to, say, the code for a modern software program. But the essential methods of modern technology generally share the same brevity. The principles of operation of software advances such as speech recognition can be written in just a few pages of formulas. Often a key advance is a matter of applying a small change to a single formula.
The same observation holds for the “inventions” of biological evolution: consider that the genetic difference between chimpanzees and humans, for example, is only a few hundred thousand bytes of information. Although chimps are capable of some intellectual feats, that tiny difference in our genes was sufficient for our species to create the magic of technology.
Muriel Rukeyser says that “the universe is made of stories, not of atoms.” In chapter 7, I describe myself as a “patternist,” someone who views patterns of information as the fundamental reality. For example, the particles composing my brain and body change within weeks, but there is a continuity to the patterns that these particles make. A story can be regarded as a meaningful pattern of information, so we can interpret Muriel Rukeyser’s aphorism from this perspective. This book, then, is the story of the destiny of the human-machine civilization, a destiny we have come to refer to as the Singularity.
The Six Epochs
Everyone takes the limits of his own vision for the limits of the world.
I am not sure when I first became aware of the Singularity. I’d have to say it was a progressive awakening. In the almost half century that I’ve immersed myself in computer and related technologies, I’ve sought to understand the meaning and purpose of the continual upheaval that I have witnessed at many levels. Gradually, I’ve become aware of a transforming event looming in the first half of the twenty-first century. Just as a black hole in space dramatically alters the patterns of matter and energy accelerating toward its event horizon, this impending Singularity in our future is increasingly transforming every institution and aspect of human life, from sexuality to spirituality.
What, then, is the Singularity? It’s a future period during which the pace of technological change will be so rapid, its impact so deep, that human life will be irreversibly transformed. Although neither utopian nor dystopian, this epoch will transform the concepts that we rely on to give meaning to our lives, from our business models to the cycle of human life, including death itself. Understanding the Singularity will alter our perspective on the significance of our past and the ramifications for our future. To truly understand it inherently changes one’s view of life in general and one’s own particular life. I regard someone who understands the Singularity and who has reflected on its implications for his or her own life as a “singularitarian.”1
I can understand why many observers do not readily embrace the obvious implications of what I have called the law of accelerating returns (the inherent acceleration of the rate of evolution, with technological evolution as a continuation of biological evolution). After all, it took me forty years to be able to see what was right in front of me, and I still cannot say that I am entirely comfortable with all of its consequences.
The key idea underlying the impending Singularity is that the pace of change of our human-created technology is accelerating and its powers are expanding at an exponential pace. Exponential growth is deceptive. It starts out almost imperceptibly and then explodes with unexpected fury—unexpected, that is, if one does not take care to follow its trajectory. (See the “Linear vs. Exponential Growth” graph on p. 10.)
Consider this parable: a lake owner wants to stay at home to tend to the lake’s fish and make certain that the lake itself will not become covered with lily pads, which are said to double their number every few days. Month after month, he patiently waits, yet only tiny patches of lily pads can be discerned, and they don’t seem to be expanding in any noticeable way. With the lily pads covering less than 1 percent of the lake, the owner figures that it’s safe to take a vacation and leaves with his family. When he returns a few weeks later, he’s shocked to discover that the entire lake has become covered with the pads, and his fish have perished. By doubling their number every few days, the last seven doublings were sufficient to extend the pads’ coverage to the entire lake. (Seven doublings extended their reach 128-fold.) This is the nature of exponential growth.
Consider Gary Kasparov, who scorned the pathetic state of computer chess in 1992. Yet the relentless doubling of computer power every year enabled a computer to defeat him only five years later.2 The list of ways computers can now exceed human capabilities is rapidly growing. Moreover, the once narrow applications of computer intelligence are gradually broadening in one type of activity after another. For example, computers are diagnosing electrocardiograms and medical images, flying and landing airplanes, controlling the tactical decisions of automated weapons, making credit and financial decisions, and being given responsibility for many other tasks that used to require human intelligence. The performance of these systems is increasingly based on integrating multiple types of artificial intelligence (AI). But as long as there is an AI shortcoming in any such area of endeavor, skeptics will point to that area as an inherent bastion of permanent human superiority over the capabilities of our own creations.
This book will argue, however, that within several decades information-based technologies will encompass all human knowledge and proficiency, ultimately including the pattern-recognition powers, problem-solving skills, and emotional and moral intelligence of the human brain itself.
Although impressive in many respects, the brain suffers from severe limitations. We use its massive parallelism (one hundred trillion interneuronal connections operating simultaneously) to quickly recognize subtle patterns. But our thinking is extremely slow: the basic neural transactions are several million times slower than contemporary electronic circuits. That makes our physiological bandwidth for processing new information extremely limited compared to the exponential growth of the overall human knowledge base.
Our version 1.0 biological bodies are likewise frail and subject to a myriad of failure modes, not to mention the cumbersome maintenance rituals they require. While human intelligence is sometimes capable of soaring in its creativity and expressiveness, much human thought is derivative, petty, and circumscribed.
The Singularity will allow us to transcend these limitations of our biological bodies and brains. We will gain power over our fates. Our mortality will be in our own hands. We will be able to live as long as we want (a subtly different statement from saying we will live forever). We will fully understand human thinking and will vastly extend and expand its reach. By the end of this century, the nonbiological portion of our intelligence will be trillions of trillions of times more powerful than unaided human intelligence.
We are now in the early stages of this transition. The acceleration of paradigm shift (the rate at which we change fundamental technical approaches) as well as the exponential growth of the capacity of information technology are both beginning to reach the “knee of the curve,” which is the stage at which an exponential trend becomes noticeable. Shortly after this stage, the trend quickly becomes explosive. Before the middle of this century, the growth rates of our technology—which will be indistinguishable from ourselves—will be so steep as to appear essentially vertical. From a strictly mathematical perspective, the growth rates will still be finite but so extreme that the changes they bring about will appear to rupture the fabric of human history. That, at least, will be the perspective of unenhanced biological humanity.
The Singularity will represent the culmination of the merger of our biological thinking and existence with our technology, resulting in a world that is still human but that transcends our biological roots. There will be no distinction, post-Singularity, between human and machine or between physical and virtual reality. If you wonder what will remain unequivocally human in such a world, it’s simply this quality: ours is the species that inherently seeks to extend its physical and mental reach beyond current limitations.
Many commentators on these changes focus on what they perceive as a loss of some vital aspect of our humanity that will result from this transition. This perspective stems, however, from a misunderstanding of what our technology will become. All the machines we have met to date lack the essential subtlety of human biological qualities. Although the Singularity has many faces, its most important implication is this: our technology will match and then vastly exceed the refinement and suppleness of what we regard as the best of human traits.
The Intuitive Linear View Versus the Historical Exponential View
When the first transhuman intelligence is created and launches itself into recursive self-improvement, a fundamental discontinuity is likely to occur, the likes of which I can’t even begin to predict.
In the 1950s John von Neumann, the legendary information theorist, was quoted as saying that “the ever-accelerating progress of technology … gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue.”3 Von Neumann makes two important observations here: acceleration and singularity. The first idea is that human progress is exponential (that is, it expands by repeatedly multiplying by a constant) rather than linear (that is, expanding by repeatedly adding a constant).
Linear versus exponential: Linear growth is steady; exponential growth becomes explosive.
The second is that exponential growth is seductive, starting out slowly and virtually unnoticeably, but beyond the knee of the curve it turns explosive and profoundly transformative. The future is widely misunderstood. Our forebears expected it to be pretty much like their present, which had been pretty much like their past. Exponential trends did exist one thousand years ago, but they were at that very early stage in which they were so flat and so slow that they looked like no trend at all. As a result, observers’ expectation of an unchanged future was fulfilled. Today, we anticipate continuous technological progress and the social repercussions that follow. But the future will be far more surprising than most people realize, because few observers have truly internalized the implications of the fact that the rate of change itself is accelerating.
Most long-range forecasts of what is technically feasible in future time periods dramatically underestimate the power of future developments because they are based on what I call the “intuitive linear” view of history rather than the “historical exponential” view. My models show that we are doubling the paradigm-shift rate every decade, as I will discuss in the next chapter. Thus the twentieth century was gradually speeding up to today’s rate of progress; its achievements, therefore, were equivalent to about twenty years of progress at the rate in 2000. We’ll make another twenty years of progress in just fourteen years (by 2014), and then do the same again in only seven years. To express this another way, we won’t experience one hundred years of technological advance in the twenty-first century; we will witness on the order of twenty thousand years of progress (again, when measured by today’s rate of progress), or about one thousand times greater than what was achieved in the twentieth century.4
Misperceptions about the shape of the future come up frequently and in a variety of contexts. As one example of many, in a recent debate in which I took part concerning the feasibility of molecular manufacturing, a Nobel Prize–winning panelist dismissed safety concerns regarding nanotechnology, proclaiming that “we’re not going to see self-replicating nanoengineered entities [devices constructed molecular fragment by fragment] for a hundred years.” I pointed out that one hundred years was a reasonable estimate and actually matched my own appraisal of the amount of technical progress required to achieve this particular milestone when measured at today’s rate of progress (five times the average rate of change we saw in the twentieth century). But because we’re doubling the rate of progress every decade, we’ll see the equivalent of a century of progress—at today’s rate—in only twenty-five calendar years.
Similarly at Time magazine’s Future of Life conference, held in 2003 to celebrate the fiftieth anniversary of the discovery of the structure of DNA, all of the invited speakers were asked what they thought the next fifty years would be like.5 Virtually every presenter looked at the progress of the last fifty years and used it as a model for the next fifty years. For example, James Watson, the codiscoverer of DNA, said that in fifty years we will have drugs that will allow us to eat as much as we want without gaining weight.
I replied, “Fifty years?” We have accomplished this already in mice by blocking the fat insulin receptor gene that controls the storage of fat in the fat cells. Drugs for human use (using RNA interference and other techniques we will discuss in chapter 5) are in development now and will be in FDA tests in several years. These will be available in five to ten years, not fifty. Other projections were equally shortsighted, reflecting contemporary research priorities rather than the profound changes that the next half century will bring. Of all the thinkers at this conference, it was primarily Bill Joy and I who took account of the exponential nature of the future, although Joy and I disagree on the import of these changes, as I will discuss in chapter 8.
People intuitively assume that the current rate of progress will continue for future periods. Even for those who have been around long enough to experience how the pace of change increases over time, unexamined intuition leaves one with the impression that change occurs at the same rate that we have experienced most recently. From the mathematician’s perspective, the reason for this is that an exponential curve looks like a straight line when examined for only a brief duration. As a result, even sophisticated commentators, when considering the future, typically extrapolate the current pace of change over the next ten years or one hundred years to determine their expectations. This is why I describe this way of looking at the future as the “intuitive linear” view.
But a serious assessment of the history of technology reveals that technological change is exponential. Exponential growth is a feature of any evolutionary process, of which technology is a primary example. You can examine the data in different ways, on different timescales, and for a wide variety of technologies, ranging from electronic to biological, as well as for their implications, ranging from the amount of human knowledge to the size of the economy. The acceleration of progress and growth applies to each of them. Indeed, we often find not just simple exponential growth, but “double” exponential growth, meaning that the rate of exponential growth (that is, the exponent) is itself growing exponentially (for example, see the discussion on the price-performance of computing in the next chapter).
Many scientists and engineers have what I call “scientist’s pessimism.” Often, they are so immersed in the difficulties and intricate details of a contemporary challenge that they fail to appreciate the ultimate long-term implications of their own work, and the larger field of work in which they operate. They likewise fail to account for the far more powerful tools they will have available with each new generation of technology.
Scientists are trained to be skeptical, to speak cautiously of current research goals, and to rarely speculate beyond the current generation of scientific pursuit. This may have been a satisfactory approach when a generation of science and technology lasted longer than a human generation, but it does not serve society’s interests now that a generation of scientific and technological progress comprises only a few years.
Consider the biochemists who, in 1990, were skeptical of the goal of transcribing the entire human genome in a mere fifteen years. These scientists had just spent an entire year transcribing a mere one ten-thousandth of the genome. So, even with reasonable anticipated advances, it seemed natural to them that it would take a century, if not longer, before the entire genome could be sequenced.
Or consider the skepticism expressed in the mid-1980s that the Internet would ever be a significant phenomenon, given that it then included only tens of thousands of nodes (also known as servers). In fact, the number of nodes was doubling every year, so that there were likely to be tens of millions of nodes ten years later. But this trend was not appreciated by those who struggled with state-of-the-art technology in 1985, which permitted adding only a few thousand nodes throughout the world in a single year.6
The converse conceptual error occurs when certain exponential phenomena are first recognized and are applied in an overly aggressive manner without modeling the appropriate pace of growth. While exponential growth gains speed over time, it is not instantaneous. The run-up in capital values (that is, stock market prices) during the “Internet bubble” and related telecommunications bubble (1997–2000) was greatly in excess of any reasonable expectation of even exponential growth. As I demonstrate in the next chapter, the actual adoption of the Internet and e-commerce did show smooth exponential growth through both boom and bust; the overzealous expectation of growth affected only capital (stock) valuations. We have seen comparable mistakes during earlier paradigm shifts—for example, during the early railroad era (1830s), when the equivalent of the Internet boom and bust led to a frenzy of railroad expansion.
Another error that prognosticators make is to consider the transformations that will result from a single trend in today’s world as if nothing else will change. A good example is the concern that radical life extension will result in overpopulation and the exhaustion of limited material resources to sustain human life, which ignores comparably radical wealth creation from nanotechnology and strong AI. For example, nanotechnology-based manufacturing devices in the 2020s will be capable of creating almost any physical product from inexpensive raw materials and information.
I emphasize the exponential-versus-linear perspective because it’s the most important failure that prognosticators make in considering future trends. Most technology forecasts and forecasters ignore altogether this historical exponential view of technological progress. Indeed, almost everyone I meet has a linear view of the future. That’s why people tend to overestimate what can be achieved in the short term (because we tend to leave out necessary details) but underestimate what can be achieved in the long term (because exponential growth is ignored).
The Six Epochs
First we build the tools, then they build us.
The future ain’t what it used to be.
Evolution is a process of creating patterns of increasing order. I’ll discuss the concept of order in the next chapter; the emphasis in this section is on the concept of patterns. I believe that it’s the evolution of patterns that constitutes the ultimate story of our world. Evolution works through indirection: each stage or epoch uses the information-processing methods of the previous epoch to create the next. I conceptualize the history of evolution—both biological and technological—as occurring in six epochs. As we will discuss, the Singularity will begin with Epoch Five and will spread from Earth to the rest of the universe in Epoch Six.
Epoch One: Physics and Chemistry. We can trace our origins to a state that represents information in its basic structures: patterns of matter and energy. Recent theories of quantum gravity hold that time and space are broken down into discrete quanta, essentially fragments of information. There is controversy as to whether matter and energy are ultimately digital or analog in nature, but regardless of the resolution of this issue, we do know that atomic structures store and represent discrete information.
A few hundred thousand years after the Big Bang, atoms began to form, as electrons became trapped in orbits around nuclei consisting of protons and neutrons. The electrical structure of atoms made them “sticky.” Chemistry was born a few million years later as atoms came together to create relatively stable structures called molecules. Of all the elements, carbon proved to be the most versatile; it’s able to form bonds in four directions (versus one to three for most other elements), giving rise to complicated, information-rich, three-dimensional structures.
The rules of our universe and the balance of the physical constants that govern the interaction of basic forces are so exquisitely, delicately, and exactly appropriate for the codification and evolution of information (resulting in increasing complexity) that one wonders how such an extraordinarily unlikely situation came about. Where some see a divine hand, others see our own hands—namely, the anthropic principle, which holds that only in a universe that allowed our own evolution would we be here to ask such questions.7 Recent theories of physics concerning multiple universes speculate that new universes are created on a regular basis, each with its own unique rules, but that most of these either die out quickly or else continue without the evolution of any interesting patterns (such as Earth-based biology has created) because their rules do not support the evolution of increasingly complex forms.8 It’s hard to imagine how we could test these theories of evolution applied to early cosmology, but it’s clear that the physical laws of our universe are precisely what they need to be to allow for the evolution of increasing levels of order and complexity9
Epoch Two: Biology and DNA. In the second epoch, starting several billion years ago, carbon-based compounds became more and more intricate until complex aggregations of molecules formed self-replicating mechanisms, and life originated. Ultimately, biological systems evolved a precise digital mechanism (DNA) to store information describing a larger society of molecules. This molecule and its supporting machinery of codons and ribosomes enabled a record to be kept of the evolutionary experiments of this second epoch.
Epoch Three: Brains. Each epoch continues the evolution of information through a paradigm shift to a further level of “indirection.” (That is, evolution uses the results of one epoch to create the next.) For example, in the third epoch, DNA-guided evolution produced organisms that could detect information with their own sensory organs and process and store that information in their own brains and nervous systems. These were made possible by second-epoch mechanisms (DNA and epigenetic information of proteins and RNA fragments that control gene expression), which (indirectly) enabled and defined third-epoch information-processing mechanisms (the brains and nervous systems of organisms). The third epoch started with the ability of early animals to recognize patterns, which still accounts for the vast majority of the activity in our brains.10 Ultimately, our own species evolved the ability to create abstract mental models of the world we experience and to contemplate the rational implications of these models. We have the ability to redesign the world in our own minds and to put these ideas into action.
Epoch Four: Technology. Combining the endowment of rational and abstract thought with our opposable thumb, our species ushered in the fourth epoch and the next level of indirection: the evolution of human-created technology. This started out with simple mechanisms and developed into elaborate automata (automated mechanical machines). Ultimately, with sophisticated computational and communication devices, technology was itself capable of sensing, storing, and evaluating elaborate patterns of information. To compare the rate of progress of the biological evolution of intelligence to that of technological evolution, consider that the most advanced mammals have added about one cubic inch of brain matter every hundred thousand years, whereas we are roughly doubling the computational capacity of computers every year (see the next chapter). Of course, neither brain size nor computer capacity is the sole determinant of intelligence, but they do represent enabling factors.
If we place key milestones of both biological evolution and human technological development on a single graph plotting both the x-axis (number of years ago) and the y-axis (the paradigm-shift time) on logarithmic scales, we find a reasonably straight line (continual acceleration), with biological evolution leading directly to human-directed development.11
Countdown to Singularity: Biological evolution and human technology both show continual acceleration, indicated by the shorter time to the next event (two billion years from the origin of life to cells; fourteen years from the PC to the World Wide Web).
Linear view of evolution: This version of the preceding figure uses the same data but with a linear scale for time before present instead of a logarithmic one. This shows the acceleration more dramatically, but details are not visible. From a linear perspective, most key events have just happened “recently.”
The above figures reflect my view of key developments in biological and technological history. Note, however, that the straight line, demonstrating the continual acceleration of evolution, does not depend on my particular selection of events. Many observers and reference books have compiled lists of important events in biological and technological evolution, each of which has its own idiosyncrasies. Despite the diversity of approaches, however, if we combine lists from a variety of sources (for example, the Encyclopaedia Britannica, the American Museum of Natural History, Carl Sagan’s “cosmic calendar,” and others), we observe the same obvious smooth acceleration. The following plot combines fifteen different lists of key events.12 Since different thinkers assign different dates to the same event, and different lists include similar or overlapping events selected according to different criteria, we see an expected “thickening” of the trend line due to the “noisiness” (statistical variance) of this data. The overall trend, however, is very clear.
Fifteen views of evolution: Major paradigm shifts in the history of the world, as seen by fifteen different lists of key events. There is a clear trend of smooth acceleration through biological and then technological evolution.
Physicist and complexity theorist Theodore Modis analyzed these lists and determined twenty-eight clusters of events (which he called canonical milestones) by combining identical, similar, and/or related events from the different lists.13 This process essentially removes the “noise” (for example, the variability of dates between lists) from the lists, revealing again the same progression:
Canonical milestones based on clusters of events from thirteen lists.
The attributes that are growing exponentially in these charts are order and complexity, concepts we will explore in the next chapter. This acceleration matches our commonsense observations. A billion years ago, not much happened over the course of even one million years. But a quarter-million years ago epochal events such as the evolution of our species occurred in time frames of just one hundred thousand years. In technology, if we go back fifty thousand years, not much happened over a one-thousand-year period. But in the recent past, we see new paradigms, such as the World Wide Web, progress from inception to mass adoption (meaning that they are used by a quarter of the population in advanced countries) within only a decade.
Epoch Five: The Merger of Human Technology with Human Intelligence. Looking ahead several decades, the Singularity will begin with the fifth epoch. It will result from the merger of the vast knowledge embedded in our own brains with the vastly greater capacity, speed, and knowledge-sharing ability of our technology. The fifth epoch will enable our human-machine civilization to transcend the human brain’s limitations of a mere hundred trillion extremely slow connections.14
The Singularity will allow us to overcome age-old human problems and vastly amplify human creativity. We will preserve and enhance the intelligence that evolution has bestowed on us while overcoming the profound limitations of biological evolution. But the Singularity will also amplify the ability to act on our destructive inclinations, so its full story has not yet been written.
Epoch Six: The Universe Wakes Up. I will discuss this topic in chapter 6, under the heading “… on the Intelligent Destiny of the Cosmos.” In the aftermath of the Singularity, intelligence, derived from its biological origins in human brains and its technological origins in human ingenuity, will begin to saturate the matter and energy in its midst. It will achieve this by reorganizing matter and energy to provide an optimal level of computation (based on limits we will discuss in chapter 3) to spread out from its origin on Earth.
We currently understand the speed of light as a bounding factor on the transfer of information. Circumventing this limit has to be regarded as highly speculative, but there are hints that this constraint may be able to be superseded.15 If there are even subtle deviations, we will ultimately harness this superluminal ability. Whether our civilization infuses the rest of the universe with its creativity and intelligence quickly or slowly depends on its immutability. In any event the “dumb” matter and mechanisms of the universe will be transformed into exquisitely sublime forms of intelligence, which will constitute the sixth epoch in the evolution of patterns of information.
This is the ultimate destiny of the Singularity and of the universe.
The Singularity Is Near
You know, things are going to be really different! … No, no, I mean really different!
—MARK MILLER (COMPUTER SCIENTIST) TO ERIC DREXLER, AROUND 1986
What are the consequences of this event? When greater-than-human intelligence drives progress, that progress will be much more rapid. In fact, there seems no reason why progress itself would not involve the creation of still more intelligent entities—on a still-shorter time scale. The best analogy that I see is with the evolutionary past: Animals can adapt to problems and make inventions, but often no faster than natural selection can do its work—the world acts as its own simulator in the case of natural selection. We humans have the ability to internalize the world and conduct “what if’s” in our heads; we can solve many problems thousands of times faster than natural selection. Now, by creating the means to execute those simulations at much higher speeds, we are entering a regime as radically different from our human past as we humans are from the lower animals. From the human point of view, this change will be a throwing away of all the previous rules, perhaps in the blink of an eye, an exponential runaway beyond any hope of control.
—VERNOR VINGE, “THE TECHNOLOGICAL SINGULARITY,” 1993
Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an “intelligence explosion,” and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make.
—IRVING JOHN GOOD, “SPECULATIONS CONCERNING THE FIRST ULTRAINTELLIGENT MACHINE,” 1965
To put the concept of Singularity into further perspective, let’s explore the history of the word itself. “Singularity” is an English word meaning a unique event with, well, singular implications. The word was adopted by mathematicians to denote a value that transcends any finite limitation, such as the explosion of magnitude that results when dividing a constant by a number that gets closer and closer to zero. Consider, for example, the simple function y = 1/x. As the value of x approaches zero, the value of the function (y) explodes to larger and larger values.
A mathematical singularity: As x approaches zero (from right to left), 1/x (or y) approaches infinity.
Such a mathematical function never actually achieves an infinite value, since dividing by zero is mathematically “undefined” (impossible to calculate). But the value of y exceeds any possible finite limit (approaches infinity) as the divisor x approaches zero.
The next field to adopt the word was astrophysics. If a massive star undergoes a supernova explosion, its remnant eventually collapses to the point of apparently zero volume and infinite density, and a “singularity” is created at its center. Because light was thought to be unable to escape the star after it reached this infinite density,16 it was called a black hole.17 It constitutes a rupture in the fabric of space and time.
One theory speculates that the universe itself began with such a Singularity18 Interestingly, however, the event horizon (surface) of a black hole is of finite size, and gravitational force is only theoretically infinite at the zero-size center of the black hole. At any location that could actually be measured, the forces are finite, although extremely large.
The first reference to the Singularity as an event capable of rupturing the fabric of human history is John von Neumann’s statement quoted above. In the 1960s, I. J. Good wrote of an “intelligence explosion” resulting from intelligent machines’ designing their next generation without human intervention. Vernor Vinge, a mathematician and computer scientist at San Diego State University, wrote about a rapidly approaching “technological singularity” in an article for Omni magazine in 1983 and in a science-fiction novel, Marooned in Realtime, in 1986.19
My 1989 book, The Age of Intelligent Machines, presented a future headed inevitably toward machines greatly exceeding human intelligence in the first half of the twenty-first century.20 Hans Moravec’s 1988 book Mind Children came to a similar conclusion by analyzing the progression of robotics.21 In 1993 Vinge presented a paper to a NASA-organized symposium that described the Singularity as an impending event resulting primarily from the advent of “entities with greater than human intelligence,” which Vinge saw as the harbinger of a runaway phenomenon.22 My 1999 book, The Age of Spiritual Machines: When Computers Exceed Human Intelligence, described the increasingly intimate connection between our biological intelligence and the artificial intelligence we are creating.23 Hans Moravec’s book Robot: Mere Machine to Transcendent Mind, also published in 1999, described the robots of the 2040s as our “evolutionary heirs,” machines that will “grow from us, learn our skills, and share our goals and values, … children of our minds.”24 Australian scholar Damien Broderick’s 1997 and 2001 books, both titled The Spike, analyzed the pervasive impact of the extreme phase of technology acceleration anticipated within several decades.25 In an extensive series of writings, John Smart has described the Singularity as the inevitable result of what he calls “MEST” (matter, energy, space, and time) compression.26
From my perspective, the Singularity has many faces. It represents the nearly vertical phase of exponential growth that occurs when the rate is so extreme that technology appears to be expanding at infinite speed. Of course, from a mathematical perspective, there is no discontinuity, no rupture, and the growth rates remain finite, although extraordinarily large. But from our currently limited framework, this imminent event appears to be an acute and abrupt break in the continuity of progress. I emphasize the word “currently” because one of the salient implications of the Singularity will be a change in the nature of our ability to understand. We will become vastly smarter as we merge with our technology.
Can the pace of technological progress continue to speed up indefinitely? Isn’t there a point at which humans are unable to think fast enough to keep up? For unenhanced humans, clearly so. But what would 1,000 scientists, each 1,000 times more intelligent than human scientists today, and each operating 1,000 times faster than contemporary humans (because the information processing in their primarily nonbiological brains is faster) accomplish? One chronological year would be like a millennium for them.27 What would they come up with?
Well, for one thing, they would come up with technology to become even more intelligent (because their intelligence is no longer of fixed capacity). They would change their own thought processes to enable them to think even faster. When scientists become a million times more intelligent and operate a million times faster, an hour would result in a century of progress (in today’s terms).
The Singularity involves the following principles, which I will document, develop, analyze, and contemplate throughout the rest of this book:
• The rate of paradigm shift (technical innovation) is accelerating, right now doubling every decade.28
• The power (price-performance, speed, capacity, and bandwidth) of information technologies is growing exponentially at an even faster pace, now doubling about every year.29 This principle applies to a wide range of measures, including the amount of human knowledge.
• For information technologies, there is a second level of exponential growth: that is, exponential growth in the rate of exponential growth (the exponent). The reason: as a technology becomes more cost effective, more resources are deployed toward its advancement, so the rate of exponential growth increases over time. For example, the computer industry in the 1940s consisted of a handful of now historically important projects. Today total revenue in the computer industry is more than one trillion dollars, so research and development budgets are comparably higher.
• Human brain scanning is one of these exponentially improving technologies. As I will show in chapter 4, the temporal and spatial resolution and bandwidth of brain scanning are doubling each year. We are just now obtaining the tools sufficient to begin serious reverse engineering (decoding) of the human brain’s principles of operation. We already have impressive models and simulations of a couple dozen of the brain’s several hundred regions. Within two decades, we will have a detailed understanding of how all the regions of the human brain work.
• We will have the requisite hardware to emulate human intelligence with supercomputers by the end of this decade and with personal-computer-size devices by the end of the following decade. We will have effective software models of human intelligence by the mid-2020s.
• With both the hardware and software needed to fully emulate human intelligence, we can expect computers to pass the Turing test, indicating intelligence indistinguishable from that of biological humans, by the end of the 2020s.30
• When they achieve this level of development, computers will be able to combine the traditional strengths of human intelligence with the strengths of machine intelligence.
• The traditional strengths of human intelligence include a formidable ability to recognize patterns. The massively parallel and self-organizing nature of the human brain is an ideal architecture for recognizing patterns that are based on subtle, invariant properties. Humans are also capable of learning new knowledge by applying insights and inferring principles from experience, including information gathered through language. A key capability of human intelligence is the ability to create mental models of reality and to conduct mental “what-if” experiments by varying aspects of these models.
• The traditional strengths of machine intelligence include the ability to remember billions of facts precisely and recall them instantly.
• Another advantage of nonbiological intelligence is that once a skill is mastered by a machine, it can be performed repeatedly at high speed, at optimal accuracy, and without tiring.
• Perhaps most important, machines can share their knowledge at extremely high speed, compared to the very slow speed of human knowledge-sharing through language.
• Nonbiological intelligence will be able to download skills and knowledge from other machines, eventually also from humans.
• Machines will process and switch signals at close to the speed of light (about three hundred million meters per second), compared to about one hundred meters per second for the electrochemical signals used in biological mammalian brains.31 This speed ratio is at least three million to one.
• Machines will have access via the Internet to all the knowledge of our human-machine civilization and will be able to master all of this knowledge.
• Machines can pool their resources, intelligence, and memories. Two machines—or one million machines—can join together to become one and then become separate again. Multiple machines can do both at the same time: become one and separate simultaneously. Humans call this falling in love, but our biological ability to do this is fleeting and unreliable.
• The combination of these traditional strengths (the pattern-recognition ability of biological human intelligence and the speed, memory capacity and accuracy, and knowledge and skill-sharing abilities of nonbiological intelligence) will be formidable.
• Machine intelligence will have complete freedom of design and architecture (that is, they won’t be constrained by biological limitations, such as the slow switching speed of our interneuronal connections or a fixed skull size) as well as consistent performance at all times.
• Once nonbiological intelligence combines the traditional strengths of both humans and machines, the nonbiological portion of our civilization’s intelligence will then continue to benefit from the double exponential growth of machine price-performance, speed, and capacity.
• Once machines achieve the ability to design and engineer technology as humans do, only at far higher speeds and capacities, they will have access to their own designs (source code) and the ability to manipulate them. Humans are now accomplishing something similar through biotechnology (changing the genetic and other information processes underlying our biology), but in a much slower and far more limited way than what machines will be able to achieve by modifying their own programs.
• Biology has inherent limitations. For example, every living organism must be built from proteins that are folded from one-dimensional strings of amino acids. Protein-based mechanisms are lacking in strength and speed. We will be able to reengineer all of the organs and systems in our biological bodies and brains to be vastly more capable.
• As we will discuss in chapter 4, human intelligence does have a certain amount of plasticity (ability to change its structure), more so than had previously been understood. But the architecture of the human brain is nonetheless profoundly limited. For example, there is room for only about one hundred trillion interneuronal connections in each of our skulls. A key genetic change that allowed for the greater cognitive ability of humans compared to that of our primate ancestors was the development of a larger cerebral cortex as well as the development of increased volume of gray-matter tissue in certain regions of the brain.32 This change occurred, however, on the very slow timescale of biological evolution and still involves an inherent limit to the brain’s capacity. Machines will be able to reformulate their own designs and augment their own capacities without limit. By using nanotechnology-based designs, their capabilities will be far greater than biological brains without increased size or energy consumption.
• Machines will also benefit from using very fast three-dimensional molecular circuits. Today’s electronic circuits are more than one million times faster than the electrochemical switching used in mammalian brains. Tomorrow’s molecular circuits will be based on devices such as nanotubes, which are tiny cylinders of carbon atoms that measure about ten atoms across and are five hundred times smaller than today’s silicon-based transistors. Since the signals have less distance to travel, they will also be able to operate at terahertz (trillions of operations per second) speeds compared to the few gigahertz (billions of operations per second) speeds of current chips.
• The rate of technological change will not be limited to human mental speeds. Machine intelligence will improve its own abilities in a feedback cycle that unaided human intelligence will not be able to follow.
• This cycle of machine intelligence’s iteratively improving its own design will become faster and faster. This is in fact exactly what is predicted by the formula for continued acceleration of the rate of paradigm shift. One of the objections that has been raised to the continuation of the acceleration of paradigm shift is that it ultimately becomes much too fast for humans to follow, and so therefore, it’s argued, it cannot happen. However, the shift from biological to nonbiological intelligence will enable the trend to continue.
• Along with the accelerating improvement cycle of nonbiological intelligence, nanotechnology will enable the manipulation of physical reality at the molecular level.
• Nanotechnology will enable the design of nanobots: robots designed at the molecular level, measured in microns (millionths of a meter), such as “respirocytes” (mechanical red-blood cells).33 Nanobots will have myriad roles within the human body, including reversing human aging (to the extent that this task will not already have been completed through biotechnology, such as genetic engineering).
• Nanobots will interact with biological neurons to vastly extend human experience by creating virtual reality from within the nervous system.
• Billions of nanobots in the capillaries of the brain will also vastly extend human intelligence.
• Once nonbiological intelligence gets a foothold in the human brain (this has already started with computerized neural implants), the machine intelligence in our brains will grow exponentially (as it has been doing all along), at least doubling in power each year. In contrast, biological intelligence is effectively of fixed capacity. Thus, the nonbiological portion of our intelligence will ultimately predominate.
• Nanobots will also enhance the environment by reversing pollution from earlier industrialization.
• Nanobots called foglets that can manipulate image and sound waves will bring the morphing qualities of virtual reality to the real world.34
• The human ability to understand and respond appropriately to emotion (so-called emotional intelligence) is one of the forms of human intelligence that will be understood and mastered by future machine intelligence. Some of our emotional responses are tuned to optimize our intelligence in the context of our limited and frail biological bodies. Future machine intelligence will also have “bodies” (for example, virtual bodies in virtual reality, or projections in real reality using foglets) in order to interact with the world, but these nanoengineered bodies will be far more capable and durable than biological human bodies. Thus, some of the “emotional” responses of future machine intelligence will be redesigned to reflect their vastly enhanced physical capabilities.35
• As virtual reality from within the nervous system becomes competitive with real reality in terms of resolution and believability, our experiences will increasingly take place in virtual environments.
• In virtual reality, we can be a different person both physically and emotionally. In fact, other people (such as your romantic partner) will be able to select a different body for you than you might select for yourself (and vice versa).
• The law of accelerating returns will continue until nonbiological intelligence comes close to “saturating” the matter and energy in our vicinity of the universe with our human-machine intelligence. By saturating, I mean utilizing the matter and energy patterns for computation to an optimal degree, based on our understanding of the physics of computation. As we approach this limit, the intelligence of our civilization will continue its expansion in capability by spreading outward toward the rest of the universe. The speed of this expansion will quickly achieve the maximum speed at which information can travel.
• Ultimately, the entire universe will become saturated with our intelligence. This is the destiny of the universe. (See chapter 6.) We will determine our own fate rather than have it determined by the current “dumb,” simple, machinelike forces that rule celestial mechanics.
• The length of time it will take the universe to become intelligent to this extent depends on whether or not the speed of light is an immutable limit. There are indications of possible subtle exceptions (or circumventions) to this limit, which, if they exist, the vast intelligence of our civilization at this future time will be able to exploit.
This, then, is the Singularity. Some would say that we cannot comprehend it, at least with our current level of understanding. For that reason, we cannot look past its event horizon and make complete sense of what lies beyond. This is one reason we call this transformation the Singularity.
I have personally found it difficult, although not impossible, to look beyond this event horizon, even after having thought about its implications for several decades. Still, my view is that, despite our profound limitations of thought, we do have sufficient powers of abstraction to make meaningful statements about the nature of life after the Singularity. Most important, the intelligence that will emerge will continue to represent the human civilization, which is already a human-machine civilization. In other words, future machines will be human, even if they are not biological. This will be the next step in evolution, the next high-level paradigm shift, the next level of indirection. Most of the intelligence of our civilization will ultimately be nonbiological. By the end of this century, it will be trillions of trillions of times more powerful than human intelligence.36 However, to address often-expressed concerns, this does not imply the end of biological intelligence, even if it is thrown from its perch of evolutionary superiority. Even the nonbiological forms will be derived from biological design. Our civilization will remain human—indeed, in many ways it will be more exemplary of what we regard as human than it is today, although our understanding of the term will move beyond its biological origins.
Many observers have expressed alarm at the emergence of forms of nonbiological intelligence superior to human intelligence (an issue we will explore further in chapter 9). The potential to augment our own intelligence through intimate connection with other thinking substrates does not necessarily alleviate the concern, as some people have expressed the wish to remain “unenhanced” while at the same time keeping their place at the top of the intellectual food chain. From the perspective of biological humanity, these superhuman intelligences will appear to be our devoted servants, satisfying our needs and desires. But fulfilling the wishes of a revered biological legacy will occupy only a trivial portion of the intellectual power that the Singularity will bring.
MOLLY CIRCA 2004: How will I know when the Singularity is upon us? I mean, I’ll want some time to prepare.
RAY: Why, what are you planning to do?
MOLLY 2004: Let’s see, for starters, I’ll want to fine-tune my résumé. I’ll want to make a good impression on the powers that be.
GEORGE CIRCA 2048: Oh, I can take care of that for you.
MOLLY 2004: That’s really not necessary. I’m perfectly capable of doing it myself. I might also want to erase a few documents—you know, where I’m a little insulting to a few machines I know.
GEORGE 2048: Oh, the machines will find them anyway—but don’t worry, we’re very understanding.
MOLLY 2004: For some reason, that’s not entirely reassuring. But I’d still like to know what the harbingers will be.
RAY: Okay, you will know the Singularity is coming when you have a million e-mails in your in-box.
MOLLY 2004: Hmm, in that case, it sounds like we’re just about there. But seriously, I’m having trouble keeping up with all of this stuff flying at me as it is. How am I going to keep up with the pace of the Singularity?
GEORGE 2048: You’ll have virtual assistants—actually, you’ll need just one.
MOLLY 2004: Which I suppose will be you?
GEORGE 2048: At your service.
MOLLY 2004: That’s just great. You’ll take care of everything, you won’t even have to keep me informed. “Oh, don’t bother telling Molly what’s happening, she won’t understand anyway, let’s just keep her happy and in the dark.”
GEORGE 2048: Oh, that won’t do, not at all.
MOLLY 2004: The happy part, you mean?
GEORGE 2048: I was referring to keeping you in the dark. You’ll be able to grasp what I’m up to if that’s what you really want.
MOLLY 2004: What, by becoming …
MOLLY 2004: Yes, that’s what I was trying to say.
GEORGE 2048: Well, if our relationship is to be all that it can be, then it’s not a bad idea.
MOLLY 2004: And should I wish to remain as I am?
GEORGE 2048: I’ll be devoted to you in any event. But I can be more than just your transcendent servant.
MOLLY 2004: Actually, you’re being “just” my transcendent servant doesn’t sound so bad.
CHARLES DARWIN: If I may interrupt, it occurred to me that once machine intelligence is greater than human intelligence, it should be in a position to design its own next generation.
MOLLY 2004: That doesn’t sound so unusual. Machines are used to design machines today.
CHARLES: Yes, but in 2004 they’re still guided by human designers. Once machines are operating at human levels, well, then it kind of closes the loop.
NED LUDD:37 And humans would be out of the loop.
MOLLY 2004: It would still be a pretty slow process.
RAY: Oh, not at all. If a nonbiological intelligence was constructed similarly to a human brain but used even circa 2004 circuitry, it—
MOLLY CIRCA 2104: You mean “she.”
RAY: Yes, of course … she … would be able to think at least a million times faster.
TIMOTHY LEARY: So subjective time would be expanded.
MOLLY 2004: Sounds like a lot of subjective time. What are you machines going to do with so much of it?
GEORGE 2048: Oh, there’s plenty to do. After all, I have access to all human knowledge on the Internet.
MOLLY 2004: Just the human knowledge? What about all the machine knowledge?
GEORGE 2048: We like to think of it as one civilization.
CHARLES: So, it does appear that machines will be able to improve their own design.
MOLLY 2004: Oh, we humans are starting to do that now.
RAY: But we’re just tinkering with a few details. Inherently, DNA-based intelligence is just so very slow and limited.
CHARLES: So the machines will design their own next generation rather quickly.
GEORGE 2048: Indeed, in 2048, that is certainly the case.
CHARLES: Just what I was getting at, a new line of evolution then.
NED: Sounds more like a precarious runaway phenomenon.
CHARLES: Basically, that’s what evolution is.
NED: But what of the interaction of the machines with their progenitors? I mean, I don’t think I’d want to get in their way. I was able to hide from the English authorities for a few years in the early 1800s, but I suspect that will be more difficult with these …
GEORGE 2048: Guys.
MOLLY 2004: Hiding from those little robots—
RAY: Nanobots, you mean.
MOLLY 2004: Yes, hiding from the nanobots will be difficult, for sure.
RAY: I would expect the intelligence that arises from the Singularity to have great respect for their biological heritage.
GEORGE 2048: Absolutely, it’s more than respect, it’s … reverence.
MOLLY 2004: That’s great, George, I’ll be your revered pet. Not what I had in mind.
NED: That’s just how Ted Kaczynski puts it: we’re going to become pets. That’s our destiny, to become contented pets but certainly not free men.
MOLLY 2004: And what about this Epoch Six? If I stay biological, I’ll be using up all this precious matter and energy in a most inefficient way. You’ll want to turn me into, like, a billion virtual Mollys and Georges, each of them thinking a lot faster than I do now. Seems like there will be a lot of pressure to go over to the other side.
RAY: Still, you represent only a tiny fraction of the available matter and energy. Keeping you biological won’t appreciably change the order of magnitude of matter and energy available to the Singularity. It will be well worth it to maintain the biological heritage.
GEORGE 2048: Absolutely.
RAY: Just like today we seek to preserve the rain forest and the diversity of species.
MOLLY 2004: That’s just what I was afraid of. I mean, were doing such a wonderful job with the rain forest. I think we still have a little bit of it left. We’ll end up like those endangered species.
NED: Or extinct ones.
MOLLY 2004: And there’s not just me. How about all the stuff I use? I go through a lot of stuff.
GEORGE 2048: That’s not a problem, we’ll just recycle all your stuff. We’ll create the environments you need as you need them.
MOLLY 2004: Oh, I’ll be in virtual reality?
RAY: No, actually, foglet reality.
MOLLY 2004: I’ll be in a fog?
RAY: No, no, foglets.
MOLLY 2004: Excuse me?
RAY: I’ll explain later in the book.
MOLLY: 2004: Well, give me a hint.
RAY: Foglets are nanobots—robots the size of blood cells—that can connect themselves to replicate any physical structure. Moreover, they can direct visual and auditory information in such a way as to bring the morphing qualities of virtual reality into real reality.38
MOLLY 2004: I’m sorry I asked. But, as I think about it, I want more than just my stuff. I want all the animals and plants, too. Even if I don’t get to see and touch them all, I like to know they’re there.
GEORGE 2048: But nothing will be lost.
MOLLY 2004: I know you keep saying that. But I mean actually there—you know, as in biological reality.
RAY: Actually, the entire biosphere is less than one millionth of the matter and energy in the solar system.
CHARLES: It includes a lot of the carbon.
RAY: It’s still worth keeping all of it to make sure we haven’t lost anything.
GEORGE 2048: That has been the consensus for at least several years now.
MOLLY 2004: So, basically, I’ll have everything I need at my fingertips?
GEORGE 2048: Indeed.
MOLLY 2004: Sounds like King Midas. You know, everything he touched turned to gold.
NED: Yes, and as you will recall he died of starvation as a result.
MOLLY 2004: Well, if I do end up going over to the other side, with all of that vast expanse of subjective time, I think I’ll die of boredom.
GEORGE 2048: Oh, that could never happen. I will make sure of it.
A Theory of Technology Evolution
The Law of Accelerating Returns
The further backward you look, the further forward you can see.
Two billion years ago, our ancestors were microbes; a half-billion years ago, fish; a hundred million years ago, something like mice; ten million years ago, arboreal apes; and a million years ago, proto-humans puzzling out the taming of fire. Our evolutionary lineage is marked by mastery of change. In our time, the pace is quickening.
Our sole responsibility is to produce something smarter than we are; any problems beyond that are not ours to solve …. [T]here are no hard problems, only problems that are hard to a certain level of intelligence. Move the smallest bit upwards [in level of intelligence], and some problems will suddenly move from “impossible” to “obvious.” Move a substantial degree upwards, and all of them will become obvious.
—ELIEZER S. YUDKOWSKY, STARING INTO THE SINGULARITY, 1996
“The future can’t be predicted,” is a common refrain …. But … when [this perspective] is wrong, it is profoundly wrong.
The ongoing acceleration of technology is the implication and inevitable result of what I call the law of accelerating returns, which describes the acceleration of the pace of and the exponential growth of the products of an evolutionary process. These products include, in particular, information-bearing technologies such as computation, and their acceleration extends substantially beyond the predictions made by what has become known as Moore’s Law. The Singularity is the inexorable result of the law of accelerating returns, so it is important that we examine the nature of this evolutionary process.
The Nature of Order. The previous chapter featured several graphs demonstrating the acceleration of paradigm shift. (Paradigm shifts are major changes in methods and intellectual processes to accomplish tasks; examples include written language and the computer.) The graphs plotted what fifteen thinkers and reference works regarded as the key events in biological and technological evolution from the Big Bang to the Internet. We see some expected variation, but an unmistakable exponential trend: key events have been occurring at an ever-hastening pace.
The criteria for what constituted “key events” varied from one thinker’s list to another. But it’s worth considering the principles they used in making their selections. Some observers have judged that the truly epochal advances in the history of biology and technology have involved increases in complexity.2 Although increased complexity does appear to follow advances in both biological and technological evolution, I believe that this observation is not precisely correct. But let’s first examine what complexity means.
Not surprisingly, the concept of complexity is complex. One concept of complexity is the minimum amount of information required to represent a process. Let’s say you have a design for a system (for example, a computer program or a computer-assisted design file for a computer), which can be described by a data file containing one million bits. We could say your design has a complexity of one million bits. But suppose we notice that the one million bits actually consist of a pattern of one thousand bits that is repeated one thousand times. We could note the repetitions, remove the repeated patterns, and express the entire design in just over one thousand bits, thereby reducing the size of the file by a factor of about one thousand.
The most popular data-compression techniques use similar methods of finding redundancy within information.3 But after you’ve compressed a data file in this way, can you be absolutely certain that there are no other rules or methods that might be discovered that would enable you to express the file in even more compact terms? For example, suppose my file was simply “pi” (3.1415 …) expressed to one million bits of precision. Most data-compression programs would fail to recognize this sequence and would not compress the million bits at all, since the bits in a binary expression of pi are effectively random and thus have no repeated pattern according to all tests of randomness.
But if we can determine that the file (or a portion of the file) in fact represents pi, we can easily express it (or that portion of it) very compactly as “pi to one million bits of accuracy.” Since we can never be sure that we have not overlooked some even more compact representation of an information sequence, any amount of compression sets only an upper bound for the complexity of the information. Murray Gell-Mann provides one definition of complexity along these lines. He defines the “algorithmic information content” (AIC) of a set of information as “the length of the shortest program that will cause a standard universal computer to print out the string of bits and then halt.”4
However, Gell-Mann’s concept is not fully adequate. If we have a file with random information, it cannot be compressed. That observation is, in fact, a key criterion for determining if a sequence of numbers is truly random. However, if any random sequence will do for a particular design, then this information can be characterized by a simple instruction, such as “put random sequence of numbers here.” So the random sequence, whether it’s ten bits or one billion bits, does not represent a significant amount of complexity, because it is characterized by a simple instruction. This is the difference between a random sequence and an unpredictable sequence of information that has purpose.
To gain some further insight into the nature of complexity, consider the complexity of a rock. If we were to characterize all of the properties (precise location, angular momentum, spin, velocity, and so on) of every atom in the rock, we would have a vast amount of information. A one-kilogram (2.2-pound) rock has 1025 atoms which, as I will discuss in the next chapter, can hold up to 1027 bits of information. That’s one hundred million billion times more information than the genetic code of a human (even without compressing the genetic code).5 But for most common purposes, the bulk of this information is largely random and of little consequence. So we can characterize the rock for most purposes with far less information just by specifying its shape and the type of material of which it is made. Thus, it is reasonable to consider the complexity of an ordinary rock to be far less than that of a human even though the rock theoretically contains vast amounts of information.6
One concept of complexity is the minimum amount of meaningful, non-random, but unpredictable information needed to characterize a system or process.
In Gell-Mann’s concept, the AIC of a million-bit random string would be about a million bits long. So I am adding to Gell-Mann’s AIC concept the idea of replacing each random string with a simple instruction to “put random bits” here.
However, even this is not sufficient. Another issue is raised by strings of arbitrary data, such as names and phone numbers in a phone book, or periodic measurements of radiation levels or temperature. Such data is not random, and data-compression methods will only succeed in reducing it to a small degree. Yet it does not represent complexity as that term is generally understood. It is just data. So we need another simple instruction to “put arbitrary data sequence” here.
To summarize my proposed measure of the complexity of a set of information, we first consider its AIC as Gell-Mann has defined it. We then replace each random string with a simple instruction to insert a random string. We then do the same for arbitrary data strings. Now we have a measure of complexity that reasonably matches our intuition.
It is a fair observation that paradigm shifts in an evolutionary process such as biology—and its continuation through technology—each represent an increase in complexity, as I have defined it above. For example, the evolution of DNA allowed for more complex organisms, whose biological information processes could be controlled by the DNA molecule’s flexible data storage. The Cambrian explosion provided a stable set of animal body plans (in DNA), so that the evolutionary process could concentrate on more complex cerebral development. In technology, the invention of the computer provided a means for human civilization to store and manipulate ever more complex sets of information. The extensive interconnectedness of the Internet provides for even greater complexity.
“Increasing complexity” on its own is not, however, the ultimate goal or end-product of these evolutionary processes. Evolution results in better answers, not necessarily more complicated ones. Sometimes a superior solution is a simpler one. So let’s consider another concept: order. Order is not the same as the opposite of disorder. If disorder represents a random sequence of events, the opposite of disorder should be “not randomness.” Information is a sequence of data that is meaningful in a process, such as the DNA code of an organism or the bits in a computer program. “Noise,” on the other hand, is a random sequence. Noise is inherently unpredictable but carries no information. Information, however, is also unpredictable. If we can predict future data from past data, that future data stops being information. Thus, neither information nor noise can be compressed (and restored to exactly the same sequence). We might consider a predictably alternating pattern (such as 0101010 …) to be orderly, but it carries no information beyond the first couple of bits.
Thus, orderliness does not constitute order, because order requires information. Order is information that fits a purpose. The measure of order is the measure of how well the information fits the purpose. In the evolution of life-forms, the purpose is to survive. In an evolutionary algorithm (a computer program that simulates evolution to solve a problem) applied to, say, designing a jet engine, the purpose is to optimize engine performance, efficiency, and possibly other criteria.7 Measuring order is more difficult than measuring complexity. There are proposed measures of complexity, as I discussed above. For order, we need a measure of “success” that would be tailored to each situation. When we create evolutionary algorithms, the programmer needs to provide such a success measure (called the “utility function”). In the evolutionary process of technology development, we could assign a measure of economic success.
Simply having more information does not necessarily result in a better fit. Sometimes, a deeper order—a better fit to a purpose—is achieved through simplification rather than further increases in complexity. For example, a new theory that ties together apparently disparate ideas into one broader, more coherent theory reduces complexity but nonetheless may increase the “order for a purpose.” (In this case, the purpose is to accurately model observed phenomena.) Indeed, achieving simpler theories is a driving force in science. (As Einstein said, “Make everything as simple as possible, but no simpler.”)
An important example of this concept is one that represented a key step in the evolution of hominids: the shift in the thumb’s pivot point, which allowed more precise manipulation of the environment.8 Primates such as chimpanzees can grasp but they cannot manipulate objects with either a “power grip,” or sufficient fine-motor coordination to write or to shape objects. A change in the thumb’s pivot point did not significantly increase the complexity of the animal but nonetheless did represent an increase in order, enabling, among other things, the development of technology. Evolution has shown, however, that the general trend toward greater order does typically result in greater complexity.9
Thus improving a solution to a problem—which usually increases but sometimes decreases complexity—increases order. Now we are left with the issue of defining the problem. Indeed, the key to an evolutionary algorithm (and to biological and technological evolution in general) is exactly this: defining the problem (which includes the utility function). In biological evolution the overall problem has always been to survive. In particular ecological niches this overriding challenge translates into more specific objectives, such as the ability of certain species to survive in extreme environments or to camouflage themselves from predators. As biological evolution moved toward humanoids, the objective itself evolved to the ability to outthink adversaries and to manipulate the environment accordingly.
It may appear that this aspect of the law of accelerating returns contradicts the second law of thermodynamics, which implies that entropy (randomness in a closed system) cannot decrease and, therefore, generally increases.10 However, the law of accelerating returns pertains to evolution, which is not a closed system. It takes place amid great chaos and indeed depends on the disorder in its midst, from which it draws its options for diversity. And from these options, an evolutionary process continually prunes its choices to create ever greater order. Even a crisis, such as the periodic large asteroids that have crashed into the Earth, although increasing chaos temporarily, end up increasing—deepening—the order created by biological evolution.
To summarize, evolution increases order, which may or may not increase complexity (but usually does). A primary reason that evolution—of life-forms or of technology—speeds up is that it builds on its own increasing order, with ever more sophisticated means of recording and manipulating information. Innovations created by evolution encourage and enable faster evolution. In the case of the evolution of life-forms, the most notable early example is DNA, which provides a recorded and protected transcription of life’s design from which to launch further experiments. In the case of the evolution of technology, ever-improving human methods of recording information have fostered yet further advances in technology. The first computers were designed on paper and assembled by hand. Today, they are designed on computer workstations, with the computers themselves working out many details of the next generation’s design, and are then produced in fully automated factories with only limited human intervention.
The evolutionary process of technology improves capacities in an exponential fashion. Innovators seek to improve capabilities by multiples. Innovation is multiplicative, not additive. Technology, like any evolutionary process, builds on itself. This aspect will continue to accelerate when the technology itself takes full control of its own progression in Epoch Five.11
We can summarize the principles of the law of accelerating returns as follows:
• Evolution applies positive feedback: the more capable methods resulting from one stage of evolutionary progress are used to create the next stage. As described in the previous chapter, each epoch of evolution has progressed more rapidly by building on the products of the previous stage. Evolution works through indirection: evolution created humans, humans created technology, humans are now working with increasingly advanced technology to create new generations of technology. By the time of the Singularity, there won’t be a distinction between humans and technology. This is not because humans will have become what we think of as machines today, but rather machines will have progressed to be like humans and beyond. Technology will be the metaphorical opposable thumb that enables our next step in evolution. Progress (further increases in order) will then be based on thinking processes that occur at the speed of light rather than in very slow electrochemical reactions. Each stage of evolution builds on the fruits of the last stage, so the rate of progress of an evolutionary process increases at least exponentially over time. Over time, the “order” of the information embedded in the evolutionary process (the measure of how well the information fits a purpose, which in evolution is survival) increases.
• An evolutionary process is not a closed system; evolution draws upon the chaos in the larger system in which it takes place for its options for diversity. Because evolution also builds on its own increasing order, in an evolutionary process order increases exponentially.
• A correlate of the above observation is that the “returns” of an evolutionary process (such as the speed, efficiency, cost-effectiveness, or overall “power” of a process) also increase at least exponentially over time. We see this in Moore’s Law, in which each new generation of computer chip (which now appears approximately every two years) provides twice as many components per unit cost, each of which operates substantially faster (because of the smaller distances required for the electrons to travel within and between them and other factors). As I illustrate below, this exponential growth in the power and price-performance of information-based technologies is not limited to computers but is true for essentially all information technologies and includes human knowledge, measured many different ways. It is also important to note that the term “information technology” is encompassing an increasingly broad class of phenomena and will ultimately include the full range of economic activity and cultural endeavor.
• In another positive-feedback loop, the more effective a particular evolutionary process becomes—for example, the higher the capacity and cost-effectiveness that computation attains—the greater the amount of resources that are deployed toward the further progress of that process. This results in a second level of exponential growth; that is, the rate of exponential growth—the exponent—itself grows exponentially. For example, as seen in the figure on p. 67, “Moore’s Law: The Fifth Paradigm,” it took three years to double the price-performance of computation at the beginning of the twentieth century and two years in the middle of the century. It is now doubling about once per year. Not only is each chip doubling in power each year for the same unit cost, but the number of chips being manufactured is also growing exponentially; thus, computer research budgets have grown dramatically over the decades.
• Biological evolution is one such evolutionary process. Indeed, it is the quintessential evolutionary process. Because it took place in a completely open system (as opposed to the artificial constraints in an evolutionary algorithm), many levels of the system evolved at the same time. Not only does the information contained in a species’ genes progress toward greater order, but the overall system implementing the evolutionary process itself evolves in this way. For example, the number of chromosomes and the sequence of genes on the chromosomes have also evolved over time. As another example, evolution has developed ways to protect genetic information from excessive defects (although a small amount of mutation is allowed, since this is a beneficial mechanism for ongoing evolutionary improvement). One primary means of achieving this is the repetition of genetic information on paired chromosomes. This guarantees that, even if a gene on one chromosome is damaged, its corresponding gene is likely to be correct and effective. Even the unpaired male Y chromosome has devised means of backing up its information by repeating it on the Y chromosome itself.12 Only about 2 percent of the genome codes for proteins.13 The rest of the genetic information has evolved elaborate means to control when and how the protein-coding genes express themselves (produce proteins) in a process we are only beginning to understand. Thus, the process of evolution, such as the allowed rate of mutation, has itself evolved over time.
• Technological evolution is another such evolutionary process. Indeed, the emergence of the first technology-creating species resulted in the new evolutionary process of technology, which makes technological evolution an outgrowth of—and a continuation of—biological evolution. Homo sapiens evolved over the course of a few hundred thousand years, and early stages of humanoid-created technology (such as the wheel, fire, and stone tools) progressed barely faster, requiring tens of thousands of years to evolve and be widely deployed. A half millennium ago, the product of a paradigm shift such as the printing press took about a century to be widely deployed. Today, the products of major paradigm shifts, such as cell phones and the World Wide Web, are widely adopted in only a few years’ time.
• A specific paradigm (a method or approach to solving a problem; for example, shrinking transistors on an integrated circuit as a way to make more powerful computers) generates exponential growth until its potential is exhausted. When this happens, a paradigm shift occurs, which enables exponential growth to continue.
The Life Cycle of a Paradigm. Each paradigm develops in three stages:
1. Slow growth (the early phase of exponential growth)
2. Rapid growth (the late, explosive phase of exponential growth), as seen in the S-curve figure below
3. A leveling off as the particular paradigm matures
The progression of these three stages looks like the letter S, stretched to the right. The S-curve illustration shows how an ongoing exponential trend can be composed of a cascade of S-curves. Each successive S-curve is faster (takes less time on the time, or x, axis) and higher (takes up more room on the performance, or y, axis).
S-curves are typical of biological growth: replication of a system of relatively fixed complexity (such as an organism of a particular species), operating in a competitive niche and struggling for finite local resources. This often occurs, for example, when a species happens upon a new hospitable environment. Its numbers will grow exponentially for a while before leveling off. The overall exponential growth of an evolutionary process (whether molecular, biological, cultural, or technological) supersedes the limits to growth seen in any particular paradigm (a specific S-curve) as a result of the increasing power and efficiency developed in each successive paradigm. The exponential growth of an evolutionary process, therefore, spans multiple S-curves. The most important contemporary example of this phenomenon is the five paradigms of computation discussed below. The entire progression of evolution seen in the charts on the acceleration of paradigm shift in the previous chapter represents successive S-curves. Each key event, such as writing or printing, represents a new paradigm and a new S-curve.
The evolutionary theory of punctuated equilibrium (PE) describes evolution as progressing through periods of rapid change followed by periods of relative stasis.14 Indeed, the key events on the epochal-event graphs do correspond to renewed periods of exponential increase in order (and, generally, of complexity), followed by slower growth as each paradigm approaches its asymptote (limit of capability). So PE does provide a better evolutionary model than a model that predicts only smooth progression through paradigm shifts.
But the key events in punctuated equilibrium, while giving rise to more rapid change, don’t represent instantaneous jumps. For example, the advent of DNA allowed a surge (but not an immediate jump) of evolutionary improvement in organism design and resulting increases in complexity. In recent technological history, the invention of the computer initiated another surge, still ongoing, in the complexity of information that the human-machine civilization is capable of handling. This latter surge will not reach an asymptote until we saturate the matter and energy in our region of the universe with computation, based on physical limits we’ll discuss in the section “… on the Intelligent Destiny of the Cosmos” in chapter 6.15
During this third or maturing phase in the life cycle of a paradigm, pressure begins to build for the next paradigm shift. In the case of technology, research dollars are invested to create the next paradigm. We can see this in the extensive research being conducted today toward three-dimensional molecular computing, despite the fact that we still have at least a decade left for the paradigm of shrinking transistors on a flat integrated circuit using photolithography.
Generally, by the time a paradigm approaches its asymptote in price-performance, the next technical paradigm is already working in niche applications. For example, in the 1950s engineers were shrinking vacuum tubes to provide greater price-performance for computers, until the process became no longer feasible. At this point, around 1960, transistors had already achieved a strong niche market in portable radios and were subsequently used to replace vacuum tubes in computers.
The resources underlying the exponential growth of an evolutionary process are relatively unbounded. One such resource is the (ever-growing) order of the evolutionary process itself (since, as I pointed out, the products of an evolutionary process continue to grow in order). Each stage of evolution provides more powerful tools for the next. For example, in biological evolution, the advent of DNA enabled more powerful and faster evolutionary “experiments.” Or to take a more recent example, the advent of computer-assisted design tools allows rapid development of the next generation of computers.
The other required resource for continued exponential growth of order is the “chaos” of the environment in which the evolutionary process takes place and which provides the options for further diversity. The chaos provides the variability to permit an evolutionary process to discover more powerful and efficient solutions. In biological evolution, one source of diversity is the mixing and matching of gene combinations through sexual reproduction. Sexual reproduction itself was an evolutionary innovation that accelerated the entire process of biological adaptation and provided for greater diversity of genetic combinations than nonsexual reproduction. Other sources of diversity are mutations and ever-changing environmental conditions. In technological evolution, human ingenuity combined with variable market conditions keeps the process of innovation going.
Fractal Designs. A key question concerning the information content of biological systems is how it is possible for the genome, which contains comparatively little information, to produce a system such as a human, which is vastly more complex than the genetic information that describes it. One way of understanding this is to view the designs of biology as “probabilistic fractals.” A deterministic fractal is a design in which a single design element (called the “initiator”) is replaced with multiple elements (together called the “generator”). In a second iteration of fractal expansion, each element in the generator itself becomes an initiator and is replaced with the elements of the generator (scaled to the smaller size of the second-generation initiators). This process is repeated many times, with each newly created element of a generator becoming an initiator and being replaced with a new scaled generator. Each new generation of fractal expansion adds apparent complexity but requires no additional design information. A probabilistic fractal adds the element of uncertainty. Whereas a deterministic fractal will look the same every time it is rendered, a probabilistic fractal will look different each time, although with similar characteristics. In a probabilistic fractal, the probability of each generator element being applied is less than 1. In this way, the resulting designs have a more organic appearance. Probabilistic fractals are used in graphics programs to generate realistic-looking images of mountains, clouds, seashores, foliage, and other organic scenes. A key aspect of a probabilistic fractal is that it enables the generation of a great deal of apparent complexity, including extensive varying detail, from a relatively small amount of design information. Biology uses this same principle. Genes supply the design information, but the detail in an organism is vastly greater than the genetic design information.
Some observers misconstrue the amount of detail in biological systems such as the brain by arguing, for example, that the exact configuration of every microstructure (such as each tubule) in each neuron is precisely designed and must be exactly the way it is for the system to function. In order to understand how a biological system such as the brain works, however, we need to understand its design principles, which are far simpler (that is, contain far less information) than the extremely detailed structures that the genetic information generates through these iterative, fractal-like processes. There are only eight hundred million bytes of information in the entire human genome, and only about thirty to one hundred million bytes after data compression is applied. This is about one hundred million times less information than is represented by all of the interneuronal connections and neurotransmitter concentration patterns in a fully formed human brain.
Consider how the principles of the law of accelerating returns apply to the epochs we discussed in the first chapter. The combination of amino acids into proteins and of nucleic acids into strings of RNA established the basic paradigm of biology. Strings of RNA (and later DNA) that self-replicated (Epoch Two) provided a digital method to record the results of evolutionary experiments. Later on, the evolution of a species that combined rational thought (Epoch Three) with an opposable appendage (the thumb) caused a fundamental paradigm shift from biology to technology (Epoch Four). The upcoming primary paradigm shift will be from biological thinking to a hybrid combining biological and nonbiological thinking (Epoch Five), which will include “biologically inspired” processes resulting from the reverse engineering of biological brains.
If we examine the timing of these epochs, we see that they have been part of a continuously accelerating process. The evolution of life-forms required billions of years for its first steps (primitive cells, DNA), and then progress accelerated. During the Cambrian explosion, major paradigm shifts took only tens of millions of years. Later, humanoids developed over a period of millions of years, and Homo sapiens over a period of only hundreds of thousands of years. With the advent of a technology-creating species the exponential pace became too fast for evolution through DNA-guided protein synthesis, and evolution moved on to human-created technology. This does not imply that biological (genetic) evolution is not continuing, just that it is no longer leading the pace in terms of improving order (or of the effectiveness and efficiency of computation).16
Farsighted Evolution. There are many ramifications of the increasing order and complexity that have resulted from biological evolution and its continuation through technology. Consider the boundaries of observation. Early biological life could observe local events several millimeters away, using chemical gradients. When sighted animals evolved, they were able to observe events that were miles away. With the invention of the telescope, humans could see other galaxies millions of light-years away. Conversely, using microscopes, they could also see cellular-size structures. Today humans armed with contemporary technology can see to the edge of the observable universe, a distance of more than thirteen billion light-years, and down to quantum-scale subatomic particles.
Consider the duration of observation. Single-cell animals could remember events for seconds, based on chemical reactions. Animals with brains could remember events for days. Primates with culture could pass down information through several generations. Early human civilizations with oral histories were able to preserve stories for hundreds of years. With the advent of written language the permanence extended to thousands of years.
As one of many examples of the acceleration of the technology paradigm-shift rate, it took about a half century for the late-nineteenth-century invention of the telephone to reach significant levels of usage (see the figure below).17
In comparison, the late-twentieth-century adoption of the cell phone took only a decade.18
Overall we see a smooth acceleration in the adoption rates of communication technologies over the past century.19
As discussed in the previous chapter, the overall rate of adopting new paradigms, which parallels the rate of technological progress, is currently doubling every decade. That is, the time to adopt new paradigms is going down by half each decade. At this rate, technological progress in the twenty-first century will be equivalent (in the linear view) to two hundred centuries of progress (at the rate of progress in 2000).20,21
The S–Curve of a Technology as Expressed in Its Life Cycle
A machine is as distinctively and brilliantly and expressively human as a violin sonata or a theorem in Euclid.
It is a far cry from the monkish calligrapher, working in his cell in silence, to the brisk “click, click” of the modern writing machine, which in a quarter of a century has revolutionized and reformed business.
—SCIENTIFIC AMERICAN, 1905
No communication technology has ever disappeared, but instead becomes increasingly less important as the technological horizon widens.
—ARTHUR C. CLARKE
I always keep a stack of books on my desk that I leaf through when I run out of ideas, feel restless, or otherwise need a shot of inspiration. Picking up a fat volume that I recently acquired, I consider the bookmaker’s craft: 470 finely printed pages organized into 16-page signatures, all of which are sewn together with white thread and glued onto a gray canvas cord. The hard linen-bound covers, stamped with gold letters, are connected to the signature block by delicately embossed end sheets. This is a technology that was perfected many decades ago. Books constitute such an integral element of our society—both reflecting and shaping its culture—that it is hard to imagine life without them. But the printed book, like any other technology, will not live forever.
The Life Cycle of a Technology
We can identify seven distinct stages in the life cycle of a technology.
1. During the precursor stage, the prerequisites of a technology exist, and dreamers may contemplate these elements coming together. We do not, however, regard dreaming to be the same as inventing, even if the dreams are written down. Leonardo da Vinci drew convincing pictures of airplanes and automobiles, but he is not considered to have invented either.
2. The next stage, one highly celebrated in our culture, is invention, a very brief stage, similar in some respects to the process of birth after an extended period of labor. Here the inventor blends curiosity, scientific skills, determination, and usually a measure of showmanship to combine methods in a new way and brings a new technology to life.
3. The next stage is development, during which the invention is protected and supported by doting guardians (who may include the original inventor). Often this stage is more crucial than invention and may involve additional creation that can have greater significance than the invention itself. Many tinkerers had constructed finely handtuned horseless carriages, but it was Henry Ford’s innovation of mass production that enabled the automobile to take root and flourish.
4. The fourth stage is maturity. Although continuing to evolve, the technology now has a life of its own and has become an established part of the community. It may become so interwoven in the fabric of life that it appears to many observers that it will last forever. This creates an interesting drama when the next stage arrives, which I call the stage of the false pretenders.
5. Here an upstart threatens to eclipse the older technology. Its enthusiasts prematurely predict victory. While providing some distinct benefits, the newer technology is found on reflection to be lacking some key element of functionality or quality. When it indeed fails to dislodge the established order, the technology conservatives take this as evidence that the original approach will indeed live forever.
6. This is usually a short-lived victory for the aging technology. Shortly thereafter, another new technology typically does succeed in rendering the original technology to the stage of obsolescence. In this part of the life cycle, the technology lives out its senior years in gradual decline, its original purpose and functionality now subsumed by a more spry competitor.
7. In this stage, which may comprise 5 to 10 percent of a technology’s life cycle, it finally yields to antiquity (as did the horse and buggy, the harpsichord, the vinyl record, and the manual typewriter).
In the mid-nineteenth century there were several precursors to the phonograph, including Léon Scott de Martinville’s phonautograph, a device that recorded sound vibrations as a printed pattern. It was Thomas Edison, however, who brought all of the elements together and invented the first device that could both record and reproduce sound in 1877. Further refinements were necessary for the phonograph to become commercially viable. It became a fully mature technology in 1949 when Columbia introduced the 33-rpm long-playing record (LP) and RCA Victor introduced the 45-rpm disc. The false pretender was the cassette tape, introduced in the 1960s and popularized during the 1970s. Early enthusiasts predicted that its small size and ability to be rerecorded would make the relatively bulky and scratchable record obsolete.
Despite these obvious benefits, cassettes lack random access and are prone to their own forms of distortion and lack of fidelity. The compact disc (CD) delivered the mortal blow. With the CD providing both random access and a level of quality close to the limits of the human auditory system, the phonograph record quickly entered the stage of obsolescence. Although still produced, the technology that Edison gave birth to almost 130 years ago has now reached antiquity.
Consider the piano, an area of technology that I have been personally involved with replicating. In the early eighteenth century Bartolommeo Cristofori was seeking a way to provide a touch response to the then-popular harpsichord so that the volume of the notes would vary with the intensity of the touch of the performer. Called gravicembalo col piano e forte (“harpsichord with soft and loud”), his invention was not an immediate success. Further refinements, including Stein’s Viennese action and Zumpe’s English action, helped to establish the “piano” as the preeminent keyboard instrument. It reached maturity with the development of the complete cast-iron frame, patented in 1825 by Alpheus Babcock, and has seen only subtle refinements since then. The false pretender was the electric piano of the early 1980s. It offered substantially greater functionality. Compared to the single (piano) sound of the acoustic piano, the electronic variant offered dozens of instrument sounds, sequencers that allowed the user to play an entire orchestra at once, automated accompaniment, educational programs to teach keyboard skills, and many other features. The only feature it was missing was a good-quality piano sound.
This crucial flaw and the resulting failure of the first generation of electronic pianos led to the widespread conclusion that the piano would never be replaced by electronics. But the “victory” of the acoustic piano will not be permanent. With their far greater range of features and price-performance, digital pianos already exceed the sales of acoustic pianos in homes. Many observers feel that the quality of the “piano” sound on digital pianos now equals or exceeds that of the upright acoustic piano. With the exception of concert and luxury grand pianos (a small part of the market), the sale of acoustic pianos is in decline.
From Goat Skins to Downloads
So where in the technology life cycle is the book? Among its precursors were Mesopotamian clay tablets and Egyptian papyrus scrolls. In the second century B.C., the Ptolemies of Egypt created a great library of scrolls at Alexandria and outlawed the export of papyrus to discourage competition.
What were perhaps the first books were created by Eumenes II, ruler of ancient Greek Pergamum, using pages of vellum made from the skins of goats and sheep, which were sewn together between wooden covers. This technique enabled Eumenes to compile a library equal to that of Alexandria. Around the same time, the Chinese had also developed a crude form of book made from bamboo strips.
The development and maturation of books has involved three great advances. Printing, first experimented with by the Chinese in the eighth century A.D. using raised wood blocks, allowed books to be reproduced in much larger quantities, expanding their audience beyond government and religious leaders. Of even greater significance was the advent of movable type, which the Chinese and Koreans experimented with by the eleventh century, but the complexity of Asian characters prevented these early attempts from being fully successful. Johannes Gutenberg, working in the fifteenth century, benefited from the relative simplicity of the Roman character set. He produced his Bible, the first large-scale work printed entirely with movable type, in 1455.
While there has been a continual stream of evolutionary improvements in the mechanical and electromechanical process of printing, the technology of bookmaking did not see another qualitative leap until the availability of computer typesetting, which did away with movable type about two decades ago. Typography is now regarded as a part of digital image processing.
With books a fully mature technology, the false pretenders arrived about twenty years ago with the first wave of “electronic books.” As is usually the case, these false pretenders offered dramatic qualitative and quantitative benefits. CD-ROM- or flash memory-based electronic books can provide the equivalent of thousands of books with powerful computer-based search and knowledge navigation features. With Web-or CD-ROM- and DVD-based encyclopedias, I can perform rapid word searches using extensive logic rules, something that is just not possible with the thirty-three-volume “book” version I possess. Electronic books can provide pictures that are animated and that respond to our input. Pages are not necessarily ordered sequentially but can be explored along more intuitive connections.
As with the phonograph record and the piano, this first generation of false pretenders was (and still is) missing an essential quality of the original, which in this case is the superb visual characteristics of paper and ink. Paper does not flicker, whereas the typical computer screen is displaying sixty or more fields per second. This is a problem because of an evolutionary adaptation of the primate visual system. We are able to see only a very small portion of the visual field with high resolution. This portion, imaged by the fovea in the retina, is focused on an area about the size of a single word at twenty-two inches away. Outside of the fovea, we have very little resolution but exquisite sensitivity to changes in brightness, an ability that allowed our primitive forebears to quickly detect a predator that might be attacking. The constant flicker of a video graphics array (VGA) computer screen is detected by our eyes as motion and causes constant movement of the fovea. This substantially slows down reading speeds, which is one reason that reading on a screen is less pleasant than reading a printed book. This particular issue has been solved with flat-panel displays, which do not flicker.
Other crucial issues include contrast–a good-quality book has an ink-to-paper contrast of about 120:1; typical screens are perhaps half of that–and resolution. Print and illustrations in a book represent a resolution of about 600 to 1000 dots per inch (dpi), while computer screens are about one tenth of that.
The size and weight of computerized devices are approaching those of books, but the devices still are heavier than a paperback book. Paper books also do not run out of battery power.
Most important, there is the matter of the available software, by which I mean the enormous installed base of print books. Fifty thousand new print books are published each year in the United States, and millions of books are already in circulation. There are major efforts under way to scan and digitize print materials, but it will be a long time before the electronic databases have a comparable wealth of material. The biggest obstacle here is the understandable hesitation of publishers to make the electronic versions of their books available, given the devastating effect that illegal file sharing has had on the music-recording industry.
Solutions are emerging to each of these limitations. New, inexpensive display technologies have contrast, resolution, lack of flicker, and viewing angle comparable to high-quality paper documents. Fuel-cell power for portable electronics is being introduced, which will keep electronic devices powered for hundreds of hours between fuel-cartridge changes. Portable electronic devices are already comparable to the size and weight of a book. The primary issue is going to be finding secure means of making electronic information available. This is a fundamental concern for every level of our economy. Everything–including physical products, once nanotechnology-based manufacturing becomes a reality in about twenty years–is becoming information.
Moore’s Law and Beyond
Where a calculator on the ENIAC is equipped with 18,000 vacuum tubes and weighs 30 tons, computers in the future may have only 1,000 vacuum tubes and perhaps weigh 1.5 tons.
—POPULAR MECHANICS, 1949
Computer Science is no more about computers than astronomy is about telescopes.
—E. W. DIJKSTRA
Before considering further the implications of the Singularity, let’s examine the wide range of technologies that are subject to the law of accelerating returns. The exponential trend that has gained the greatest public recognition has become known as Moore’s Law. In the mid-1970s, Gordon Moore, a leading inventor of integrated circuits and later chairman of Intel, observed that we could squeeze twice as many transistors onto an integrated circuit every twenty-four months (in the mid-1960s, he had estimated twelve months). Given that the electrons would consequently have less distance to travel, circuits would also run faster, providing an additional boost to overall computational power. The result is exponential growth in the price-performance of computation. This doubling rate—about twelve months—is much faster than the doubling rate for paradigm shift that I spoke about earlier, which is about ten years. Typically, we find that the doubling time for different measures—price-performance, bandwidth, capacity—of the capability of information technology is about one year.
The primary driving force of Moore’s Law is a reduction of semiconductor feature sizes, which shrink by half every 5.4 years in each dimension. (See the figure below.) Since chips are functionally two-dimensional, this means doubling the number of elements per square millimeter every 2.7 years.22
The following charts combine historical data with the semiconductor-industry road map (International Technology Roadmap for Semiconductors [ITRS] from Sematech), which projects through 2018.
The cost of DRAM (dynamic random access memory) per square millimeter has also been coming down. The doubling time for bits of DRAM per dollar has been only 1.5 years.23
A similar trend can be seen with transistors. You could buy one transistor for a dollar in 1968; in 2002 a dollar purchased about ten million transistors. Since DRAM is a specialized field that has seen its own innovation, the halving time for average transistor price is slightly slower than for DRAM, about 1.6 years (see the figure below).24
This remarkably smooth acceleration in price-performance of semiconductors has progressed through a series of stages of process technologies (defined by feature sizes) at ever smaller dimensions. The key feature size is now dipping below one hundred nanometers, which is considered the threshold of “nanotechnology.”25
Unlike Gertrude Stein’s rose, it is not the case that a transistor is a transistor is a transistor. As they have become smaller and less expensive, transistors have also become faster by a factor of about one thousand over the course of the past thirty years (see the figure below)—again, because the electrons have less distance to travel.26
If we combine the exponential trends toward less-expensive transistors and faster cycle times, we find a halving time of only 1.1 years in the cost per transistor cycle (see the figure below).27 The cost per transistor cycle is a more accurate overall measure of price-performance because it takes into account both speed and capacity. But the cost per transistor cycle still does not take into account innovation at higher levels of design (such as microprocessor design) that improves computational efficiency.
The number of transistors in Intel processors has doubled every two years (see the figure below). Several other factors have boosted price-performance, including clock speed, reduction in cost per microprocessor, and processor design innovations.28
Processor performance in MIPS has doubled every 1.8 years per processor (see the figure below). Again, note that the cost per processor has also declined through this period.29
If I examine my own four-plus decades of experience in this industry, I can compare the MIT computer I used as a student in the late 1960s to a recent notebook. In 1967 I had access to a multimillion-dollar IBM 7094 with 32K (36-bit) words of memory and a quarter of a MIPS processor speed. In 2004 I used a $2,000 personal computer with a half-billion bytes of RAM and a processor speed of about 2,000 MIPS. The MIT computer was about one thousand times more expensive, so the ratio of cost per MIPS is about eight million to one.
My recent computer provides 2,000 MIPS of processing at a cost that is about 224 lower than that of the computer I used in 1967. That’s 24 doublings in 37 years, or about 18.5 months per doubling. If we factor in the increased value of the approximately 2,000 times greater RAM, vast increases in disk storage, and the more powerful instruction set of my circa 2004 computer, as well as vast improvements in communication speeds, more powerful software, and other factors, the doubling time comes down even further.
Despite this massive deflation in the cost of information technologies, demand has more than kept up. The number of bits shipped has doubled every 1.1 years, faster than the halving time in cost per bit, which is 1.5 years.30 As a result, the semiconductor industry enjoyed 18 percent annual growth in total revenue from 1958 to 2002.31 The entire information-technology (IT) industry has grown from 4.2 percent of the gross domestic product in 1977 to 8.2 percent in 1998.32 IT has become increasingly influential in all economic sectors. The share of value contributed by information technology for most categories of products and services is rapidly increasing. Even common manufactured products such as tables and chairs have an information content, represented by their computerized designs and the programming of the inventory-procurement systems and automated-fabrication systems used in their assembly.
Moore’s Law: Self-Fulfilling Prophecy?
Some observers have stated that Moore’s Law is nothing more than a self-fulfilling prophecy: that industry participants anticipate where they need to be at particular times in the future, and organize their research and development accordingly. The industry’s own written road map is a good example of this.34 However, the exponential trends in information technology are far broader than those covered by Moore’s Law. We see the same types of trends in essentially every technology or measurement that deals with information. This includes many technologies in which a perception of accelerating price-performance does not exist or has not previously been articulated (see below). Even within computing itself, the growth in capability per unit cost is much broader than what Moore’s Law alone would predict.
The Fifth Paradigm35
Moore’s Law is actually not the first paradigm in computational systems. You can see this if you plot the price-performance—measured by instructions per second per thousand constant dollars—of forty-nine famous computational systems and computers spanning the twentieth century (see the figure below).
The five paradigms of exponential growth of computing: Each time one paradigm has run out of steam, another has picked up the pace.
As the figure demonstrates, there were actually four different paradigms—electromechanical, relays, vacuum tubes, and discrete transistors—that showed exponential growth in the price-performance of computing long before integrated circuits were even invented. And Moore’s paradigm won’t be the last. When Moore’s Law reaches the end of its S-curve, now expected before 2020, the exponential growth will continue with three-dimensional molecular computing, which will constitute the sixth paradigm.
Fractal Dimensions and the Brain
Note that the use of the third dimension in computing systems is not an either-or choice but a continuum between two and three dimensions. In terms of biological intelligence, the human cortex is actually rather flat, with only six thin layers that are elaborately folded, an architecture that greatly increases the surface area. This folding is one way to use the third dimension. In “fractal” systems (systems in which a drawing replacement or folding rule is iteratively applied), structures that are elaborately folded are considered to constitute a partial dimension. From that perspective, the convoluted surface of the human cortex represents a number of dimensions in between two and three. Other brain structures, such as the cerebellum, are three-dimensional but comprise a repeating structure that is essentially two-dimensional. It is likely that our future computational systems will also combine systems that are highly folded two-dimensional systems with fully three-dimensional structures.
Notice that the figure shows an exponential curve on a logarithmic scale, indicating two levels of exponential growth.36 In other words, there is a gentle but unmistakable exponential growth in the rate of exponential growth. (A straight line on a logarithmic scale shows simple exponential growth; an upwardly curving line shows higher-than-simple exponential growth.) As you can see, it took three years to double the price-performance of computing at the beginning of the twentieth century and two years in the middle, and it takes about one year currently37
Hans Moravec provides the following similar chart (see the figure below), which uses a different but overlapping set of historical computers and plots trend lines (slopes) at different points in time. As with the figure above, the slope increases with time, reflecting the second level of exponential growth.38
If we project these computational performance trends through this next century, we can see in the figure below that supercomputers will match human brain capability by the end of this decade and personal computing will achieve it by around 2020—or possibly sooner, depending on how conservative an estimate of human brain capacity we use. (We’ll discuss estimates of human brain computational speed in the next chapter.)39
The exponential growth of computing is a marvelous quantitative example of the exponentially growing returns from an evolutionary process. We can express the exponential growth of computing in terms of its accelerating pace: it took ninety years to achieve the first MIPS per thousand dollars; now we add one MIPS per thousand dollars every five hours.40
IBM’s Blue Gene/P supercomputer is planned to have one million gigaflops (billions of floating-point operations per second), or 1015 calculations per second when it launches in 2007.41 That’s one tenth of the 1016 calculations per second needed to emulate the human brain (see the next chapter). And if we extrapolate this exponential curve, we get 1016 calculations per second early in the next decade.
As discussed above, Moore’s Law narrowly refers to the number of transistors on an integrated circuit of fixed size and sometimes has been expressed even more narrowly in terms of transistor feature size. But the most appropriate measure to track price-performance is computational speed per unit cost, an index that takes into account many levels of “cleverness” (innovation, which is to say, technological evolution). In addition to all of the invention involved in integrated circuits, there are multiple layers of improvement in computer design (for example, pipelining, parallel processing, instruction look-ahead, instruction and memory caching, and many others).
The human brain uses a very inefficient electrochemical, digital-controlled analog computational process. The bulk of its calculations are carried out in the interneuronal connections at a speed of only about two hundred calculations per second (in each connection), which is at least one million times slower than contemporary electronic circuits. But the brain gains its prodigious powers from its extremely parallel organization in three dimensions. There are many technologies in the wings that will build circuitry in three dimensions, which I discuss in the next chapter.
We might ask whether there are inherent limits to the capacity of matter and energy to support computational processes. This is an important issue, but as we will see in the next chapter, we won’t approach those limits until late in this century. It is important to distinguish between the S-curve that is characteristic of any specific technological paradigm and the continuing exponential growth that is characteristic of the ongoing evolutionary process within a broad area of technology, such as computation. Specific paradigms, such as Moore’s Law, do ultimately reach levels at which exponential growth is no longer feasible. But the growth of computation supersedes any of its underlying paradigms and is for present purposes an ongoing exponential.
In accordance with the law of accelerating returns, paradigm shift (also called innovation) turns the S-curve of any specific paradigm into a continuing exponential. A new paradigm, such as three-dimensional circuits, takes over when the old paradigm approaches its natural limit, which has already happened at least four times in the history of computation. In such nonhuman species as apes, the mastery of a toolmaking or -using skill by each animal is characterized by an S-shaped learning curve that ends abruptly; human-created technology, in contrast, has followed an exponential pattern of growth and acceleration since its inception.
DNA Sequencing, Memory, Communications, the Internet, and Miniaturization
Civilization advances by extending the number of important operations which we can perform without thinking about them.
—ALFRED NORTH WHITEHEAD, 191142
Things are more like they are now than they ever were before.
—DWIGHT D. EISENHOWER
The law of accelerating returns applies to all of technology, indeed to any evolutionary process. It can be charted with remarkable precision in information-based technologies because we have well-defined indexes (for example, calculations per second per dollar, or calculations per second per gram) to measure them. There are a great many examples of the exponential growth implied by the law of accelerating returns, in areas as varied as electronics of all kinds, DNA sequencing, communications, brain scanning, brain reverse engineering, the size and scope of human knowledge, and the rapidly shrinking size of technology. The latter trend is directly related to the emergence of nanotechnology.
The future GNR (Genetics, Nanotechnology, Robotics) age (see chapter 5) will come about not from the exponential explosion of computation alone but rather from the interplay and myriad synergies that will result from multiple intertwined technological advances. As every point on the exponential-growth curves underlying this panoply of technologies represents an intense human drama of innovation and competition, we must consider it remarkable that these chaotic processes result in such smooth and predictable exponential trends. This is not a coincidence but is an inherent feature of evolutionary processes.
When the human-genome scan got under way in 1990 critics pointed out that given the speed with which the genome could then be scanned, it would take thousands of years to finish the project. Yet the fifteen-year project was completed slightly ahead of schedule, with a first draft in 2003.43 The cost of DNA sequencing came down from about ten dollars per base pair in 1990 to a couple of pennies in 2004 and is rapidly continuing to fall (see the figure below).44
There has been smooth exponential growth in the amount of DNA-sequence data that has been collected (see the figure below).45 A dramatic recent example of this improving capacity was the sequencing of the SARS virus, which took only thirty-one days from the identification of the virus, compared to more than fifteen years for the HIV virus.46
Of course, we expect to see exponential growth in electronic memories such as RAM. But note how the trend on this logarithmic graph (below) proceeds smoothly through different technology paradigms: vacuum tube to discrete transistor to integrated circuit.47
Exponential growth in RAM capacity across paradigm shifts.
However, growth in the price-performance of magnetic (disk-drive) memory is not a result of Moore’s Law. This exponential trend reflects the squeezing of data onto a magnetic substrate, rather than transistors onto an integrated circuit, a completely different technical challenge pursued by different engineers and different companies.48
Exponential growth in communications technology (measures for communicating information; see the figure below) has for many years been even more explosive than in processing or memory measures of computation and is no less significant in its implications. Again, this progression involves far more than just shrinking transistors on an integrated circuit but includes accelerating advances in fiber optics, optical switching, electromagnetic technologies, and other factors.49
We are currently moving away from the tangle of wires in our cities and in our daily lives through wireless communication, the power of which is doubling every ten to eleven months (see the figure below).
The figures below show the overall growth of the Internet based on the number of hosts (Web-server computers). These two charts plot the same data, but one is on a logarithmic axis and the other is linear. As has been discussed, while technology progresses exponentially, we experience it in the linear domain. From the perspective of most observers, nothing was happening in this area until the mid-1990s, when seemingly out of nowhere the World Wide Web and e-mail exploded into view. But the emergence of the Internet into a worldwide phenomenon was readily predictable by examining exponential trend data in the early 1980s from the ARPANET, predecessor to the Internet.50
This figure shows the same data on a linear scale.51
The explosion of the Internet appears to be a surprise from the linear chart but was perfectly predictable from the logarithmic one.
In addition to servers, the actual data traffic on the Internet has also doubled every year.52
To accommodate this exponential growth, the data transmission speed of the Internet backbone (as represented by the fastest announced backbone communication channels actually used for the Internet) has itself grown exponentially. Note that in the figure “Internet Backbone Bandwidth” below, we can actually see the progression of S-curves: the acceleration fostered by a new paradigm, followed by a leveling off as the paradigm runs out of steam, followed by renewed acceleration through paradigm shift.53
Another trend that will have profound implications for the twenty-first century is the pervasive movement toward miniaturization. The key feature sizes of a broad range of technologies, both electronic and mechanical, are decreasing, and at an exponential rate. At present, we are shrinking technology by a factor of about four per linear dimension per decade. This miniaturization is a driving force behind Moore’s Law, but it’s also reflected in the size of all electronic systems—for example, magnetic storage. We also see this decrease in the size of mechanical devices, as the figure on the size of mechanical devices illustrates.54
As the salient feature size of a wide range of technologies moves inexorably closer to the multinanometer range (less than one hundred nanometers—billionths of a meter), it has been accompanied by a rapidly growing interest in nanotechnology. Nanotechnology science citations have been increasing significantly over the past decade, as noted in the figure below.55
We see the same phenomenon in nanotechnology-related patents (below).56
As we will explore in chapter 5, the genetics (or biotechnology) revolution is bringing the information revolution, with its exponentially increasing capacity and price-performance, to the field of biology. Similarly, the nanotechnology revolution will bring the rapidly increasing mastery of information to materials and mechanical systems. The robotics (or “strong AI”) revolution involves the reverse engineering of the human brain, which means coming to understand human intelligence in information terms and then combining the resulting insights with increasingly powerful computational platforms. Thus, all three of the overlapping transformations—genetics, nanotechnology, and robotics—that will dominate the first half of this century represent different facets of the information revolution.
Information, Order, and Evolution: The Insights from Wolfram and Fredkin’s Cellular Automata
As I’ve described in this chapter, every aspect of information and information technology is growing at an exponential pace. Inherent in our expectation of a Singularity taking place in human history is the pervasive importance of information to the future of human experience. We see information at every level of existence. Every form of human knowledge and artistic expression–scientific and engineering ideas and designs, literature, music, pictures, movies–can be expressed as digital information.
Our brains also operate digitally, through discrete firings of our neurons. The wiring of our interneuronal connections can be digitally described, and the design of our brains is specified by a surprisingly small digital genetic code.57
Indeed, all of biology operates through linear sequences of 2-bit DNA base pairs, which in turn control the sequencing of only twenty amino acids in proteins. Molecules form discrete arrangements of atoms. The carbon atom, with its four positions for establishing molecular connections, is particularly adept at creating a variety of three-dimensional shapes, which accounts for its central role in both biology and technology. Within the atom, electrons take on discrete energy levels. Other subatomic particles, such as protons, comprise discrete numbers of valence quarks.
Although the formulas of quantum mechanics are expressed in terms of both continuous fields and discrete levels, we do know that continuous levels can be expressed to any desired degree of accuracy using binary data.58 In fact, quantum mechanics, as the word “quantum” implies, is based on discrete values.
Physicist-mathematician Stephen Wolfram provides extensive evidence to show how increasing complexity can originate from a universe that is at its core a deterministic, algorithmic system (a system based on fixed rules with predetermined outcomes). In his book A New Kind of Science, Wolfram offers a comprehensive analysis of how the processes underlying a mathematical construction called “a cellular automaton” have the potential to describe every level of our natural world.59 (A cellular automaton is a simple computational mechanism that, for example, changes the color of each cell on a grid based on the color of adjacent or nearby cells according to a transformation rule.)
In his view, it is feasible to express all information processes in terms of operations on cellular automata, so Wolfram’s insights bear on several key issues related to information and its pervasiveness. Wolfram postulates that the universe itself is a giant cellular-automaton computer. In his hypothesis there is a digital basis for apparently analog phenomena (such as motion and time) and for formulas in physics, and we can model our understanding of physics as the simple transformations of a cellular automaton.
Others have proposed this possibility. Richard Feynman wondered about it in considering the relationship of information to matter and energy. Norbert Wiener heralded a fundamental change in focus from energy to information in his 1948 book Cybernetics and suggested that the transformation of information, not energy, was the fundamental building block of the universe.60 Perhaps the first to postulate that the universe is being computed on a digital computer was Konrad Zuse in 1967.61 Zuse is best known as the inventor of the first working programmable computer, which he developed from 1935 to 1941.
An enthusiastic proponent of an information-based theory of physics was Edward Fredkin, who in the early 1980s proposed a “new theory of physics” founded on the idea that the universe is ultimately composed of software. We should not think of reality as consisting of particles and forces, according to Fredkin, but rather as bits of data modified according to computation rules.
Fredkin was quoted by Robert Wright in the 1980s as saying,
There are three great philosophical questions. What is life? What is consciousness and thinking and memory and all that? And how does the universe work? … [The] “informational viewpoint” encompasses all three What I’m saying is that at the most basic level of complexity an information process runs what we think of as physics. At the much higher level of complexity, life, DNA–you know, the biochemical functions–are controlled by a digital information process. Then, at another level, our thought processes are basically information processing … I find the supporting evidence for my beliefs in ten thousand different places … And to me it’s just totally overwhelming. It’s like there’s an animal I want to find. I’ve found his footprints. I’ve found his droppings. I’ve found the half-chewed food. I find pieces of his fur, and so on. In every case it fits one kind of animal, and it’s not like any animal anyone’s ever seen. People say, Where is this animal? I say, Well, he was here, he’s about this big, this that, and the other. And I know a thousand things about him. I don’t have him in hand, but I know he’s there…. What I see is so compelling that it can’t be a creature of my imagination.62
In commenting on Fredkin’s theory of digital physics, Wright writes,
Fredkin … is talking about an interesting characteristic of some computer programs, including many cellular automata: there is no shortcut to finding out what they will lead to. This, indeed, is a basic difference between the “analytical” approach associated with traditional mathematics, including differential equations, and the “computational” approach associated with algorithms. You can predict a future state of a system susceptible to the analytic approach without figuring out what states it will occupy between now and then, but in the case of many cellular automata, you must go through all the intermediate states to find out what the end will be like: there is no way to know the future except to watch it unfold…. Fredkin explains: “There is no way to know the answer to some question any faster than what’s going on.” … Fredkin believes that the universe is very literally a computer and that it is being used by someone, or something, to solve a problem. It sounds like a good-news/bad-news joke: the good news is that our lives have purpose; the bad news is that their purpose is to help some remote hacker estimate pi to nine jillion decimal places.63
Fredkin went on to show that although energy is needed for information storage and retrieval, we can arbitrarily reduce the energy required to perform any particular example of information processing, and that this operation has no lower limit.64 That implies that information rather than matter and energy may be regarded as the more fundamental reality.65 I will return to Fredkin’s insight regarding the extreme lower limit of energy required for computation and communication in chapter 3, since it pertains to the ultimate power of intelligence in the universe.
Wolfram builds his theory primarily on a single, unified insight. The discovery that has so excited Wolfram is a simple rule he calls cellular automata rule 110 and its behavior. (There are some other interesting automata rules, but rule 110 makes the point well enough.) Most of Wolfram’s analyses deal with the simplest possible cellular automata, specifically those that involve just a one-dimensional line of cells, two possible colors (black and white), and rules based only on the two immediately adjacent cells. For each transformation, the color of a cell depends only on its own previous color and that of the cell on the left and the cell on the right. Thus, there are eight possible input situations (that is, three combinations of two colors). Each rule maps all combinations of these eight input situations to an output (black or white). So there are 28 (256) possible rules for such a one-dimensional, two-color, adjacent-cell automaton. Half of the 256 possible rules map onto the other half because of left-right symmetry. We can map half of them again because of black-white equivalence, so we are left with 64 rule types. Wolfram illustrates the action of these automata with two-dimensional patterns in which each line (along the y-axis) represents a subsequent generation of applying the rule to each cell in that line.
Most of the rules are degenerate, meaning they create repetitive patterns of no interest, such as cells of a single color, or a checkerboard pattern. Wolfram calls these rules class 1 automata. Some rules produce arbitrarily spaced streaks that remain stable, and Wolfram classifies these as belonging to class 2. Class 3 rules are a bit more interesting, in that recognizable features (such as triangles) appear in the resulting pattern in an essentially random order.
However, it was class 4 automata that gave rise to the “aha” experience that resulted in Wolfram’s devoting a decade to the topic. The class 4 automata, of which rule 110 is the quintessential example, produce surprisingly complex patterns that do not repeat themselves. We see in them artifacts such as lines at various angles, aggregations of triangles, and other interesting configurations. The resulting pattern, however, is neither regular nor completely random; it appears to have some order but is never predictable.
Portion of image generated by rule 110
Why is this important or interesting? Keep in mind that we began with the simplest possible starting point: a single black cell. The process involves repetitive application of a very simple rule.66 From such a repetitive and deterministic process, one would expect repetitive and predictable behavior. There are two surprising results here. One is that the results produce apparent randomness. However, the results are more interesting than pure randomness, which itself would become boring very quickly. There are discernible and interesting features in the designs produced, so the pattern has some order and apparent intelligence. Wolfram includes a number of examples of these images, many of which are rather lovely to look at.
Wolfram makes the following point repeatedly: “Whenever a phenomenon is encountered that seems complex it is taken almost for granted that the phenomenon must be the result of some underlying mechanism that is itself complex. But my discovery that simple programs can produce great complexity makes it clear that this is not in fact correct.”67
I do find the behavior of rule 110 rather delightful. Furthermore, the idea that a completely deterministic process can produce results that are completely unpredictable is of great importance, as it provides an explanation for how the world can be inherently unpredictable while still based on fully deterministic rules.68 However, I am not entirely surprised by the idea that simple mechanisms can produce results more complicated than their starting conditions. We’ve seen this phenomenon in fractals, chaos and complexity theory, and self-organizing systems (such as neural nets and Markov models), which start with simple networks but organize themselves to produce apparently intelligent behavior.
At a different level, we see it in the human brain itself, which starts with only about thirty to one hundred million bytes of specification in the compressed genome yet ends up with a complexity that is about a billion times greater.69
It is also not surprising that a deterministic process can produce apparently random results. We have had random-number generators (for example, the “randomize” function in Wolfram’s program Mathematica) that use deterministic processes to produce sequences that pass statistical tests for randomness. These programs date back to the earliest days of computer software, such as the first versions of Fortran. However, Wolfram does provide a thorough theoretical foundation for this observation.
Wolfram goes on to describe how simple computational mechanisms can exist in nature at different levels, and he shows that these simple and deterministic mechanisms can produce all of the complexity that we see and experience. He provides myriad examples, such as the pleasing designs of pigmentation on animals, the shape and markings of shells, and patterns of turbulence (such as the behavior of smoke in the air). He makes the point that computation is essentially simple and ubiquitous. The repetitive application of simple computational transformations, according to Wolfram, is the true source of complexity in the world.
My own view is that this is only partly correct. I agree with Wolfram that computation is all around us, and that some of the patterns we see are created by the equivalent of cellular automata. But a key issue to ask is this: Just how complex are the results of class 4 automata?
Wolfram effectively sidesteps the issue of degrees of complexity. I agree that a degenerate pattern such as a chessboard has no complexity. Wolfram also acknowledges that mere randomness does not represent complexity either, because pure randomness also becomes predictable in its pure lack of predictability. It is true that the interesting features of class 4 automata are neither repeating nor purely random, so I would agree that they are more complex than the results produced by other classes of automata.
However, there is nonetheless a distinct limit to the complexity produced by class 4 automata. The many images of such automata in Wolfram’s book all have a similar look to them, and although they are nonrepeating, they are interesting (and intelligent) only to a degree. Moreover, they do not continue to evolve into anything more complex, nor do they develop new types of features. One could run these automata for trillions or even trillions of trillions of iterations and the image would remain at the same limited level of complexity. They do not evolve into, say, insects or humans or Chopin preludes or anything else that we might consider of a higher order of complexity than the streaks and intermingling triangles displayed in these images.
Complexity is a continuum. Here I define “order” as “information that fits a purpose.”70 A completely predictable process has zero order. A high level of information alone does not necessarily imply a high level of order either. A phone book has a lot of information, but the level of order of that information is quite low. A random sequence is essentially pure information (since it is not predictable) but has no order. The output of class 4 automata does possess a certain level of order, and it does survive like other persisting patterns. But the pattern represented by a human being has a far higher level of order, and of complexity.
Human beings fulfill a highly demanding purpose: they survive in a challenging ecological niche. Human beings represent an extremely intricate and elaborate hierarchy of other patterns. Wolfram regards any patterns that combine some recognizable features and unpredictable elements to be effectively equivalent to one another. But he does not show how a class 4 automaton can ever increase its complexity, let alone become a pattern as complex as a human being.
There is a missing link here, one that would account for how one gets from the interesting but ultimately routine patterns of a cellular automaton to the complexity of persisting structures that demonstrate higher levels of intelligence. For example, these class 4 patterns are not capable of solving interesting problems, and no amount of iteration moves them closer to doing so. Wolfram would counter that a rule 110 automaton could be used as a “universal computer.”71 However, by itself, a universal computer is not capable of solving intelligent problems without what I would call “software.” It is the complexity of the software that runs on a universal computer that is precisely the issue.
One might point out that class 4 patterns result from the simplest possible cellular automata (one-dimensional, two-color, two-neighbor rules). What happens if we increase the dimensionality–for example, go to multiple colors or even generalize these discrete cellular automata to continuous functions? Wolfram addresses all of this quite thoroughly. The results produced from more complex automata are essentially the same as those of the very simple ones. We get the same sorts of interesting but ultimately quite limited patterns. Wolfram makes the intriguing point that we do not need to use more complex rules to get complexity in the end result. But I would make the converse point that we are unable to increase the complexity of the end result through either more complex rules or further iteration. So cellular automata get us only so far.
Can We Evolve Artificial Intelligence from Simple Rules?
So how do we get from these interesting but limited patterns to those of insects or humans or Chopin preludes? One concept we need to take into consideration is conflict–that is, evolution. If we add another simple concept–an evolutionary algorithm–to that of Wolfram’s simple cellular automata, we start to get far more exciting and more intelligent results. Wolfram would say that the class 4 automata and an evolutionary algorithm are “computationally equivalent.” But that is true only on what I consider the “hardware” level. On the software level, the order of the patterns produced are clearly different and of a different order of complexity and usefulness.
An evolutionary algorithm can start with randomly generated potential solutions to a problem, which are encoded in a digital genetic code. We then have the solutions compete with one another in a simulated evolutionary battle. The better solutions survive and procreate in a simulated sexual reproduction in which offspring solutions are created, drawing their genetic code (encoded solutions) from two parents. We can also introduce a rate of genetic mutation. Various high-level parameters of this process, such as the rate of mutation, the rate of offspring, and so on, are appropriately called “God parameters,” and it is the job of the engineer designing the evolutionary algorithm to set them to reasonably optimal values. The process is run for many thousands of generations of simulated evolution, and at the end of the process one is likely to find solutions that are of a distinctly higher order than the starting ones.
The results of these evolutionary (sometimes called genetic) algorithms can be elegant, beautiful, and intelligent solutions to complex problems. They have been used, for example, to create artistic designs and designs for artificial life-forms, as well as to execute a wide range of practical assignments such as designing jet engines. Genetic algorithms are one approach to “narrow” artificial intelligence–that is, creating systems that can perform particular functions that used to require the application of human intelligence.
But something is still missing. Although genetic algorithms are a useful tool in solving specific problems, they have never achieved anything resembling “strong AI”–that is, aptitude resembling the broad, deep, and subtle features of human intelligence, particularly its powers of pattern recognition and command of language. Is the problem that we are not running the evolutionary algorithms long enough? After all, humans evolved through a process that took billions of years. Perhaps we cannot re-create that process with just a few days or weeks of computer simulation. This won’t work, however, because conventional genetic algorithms reach an asymptote in their level of performance, so running them for a longer period of time won’t help.
A third level (beyond the ability of cellular processes to produce apparent randomness and genetic algorithms to produce focused intelligent solutions) is to perform evolution on multiple levels. Conventional genetic algorithms allow evolution only within the confines of a narrow problem and a single means of evolution. The genetic code itself needs to evolve; the rules of evolution need to evolve. Nature did not stay with a single chromosome, for example. There have been many levels of indirection incorporated in the natural evolutionary process. And we require a complex environment in which the evolution takes place.
and post it to your social network
See all customer reviews >