A hard-hitting book about the most important topic of this century and possibly beyond the issue of whether our species can survive. I wish it was science fiction but I know it's not.” Jaan Tallinn, co-founder of Skype
“The compelling story of humanity's most critical challenge. A Silent Spring for the twenty-first century.” Michael Vassar, former President, Singularity Institute
“Barrat's book is excellently written and deeply researched. It does a great job of communicating to general readers the danger of mistakes in AI design and implementation.” Bill Hibbard, author of Super-Intelligent Machines
“An important and disturbing book.” Huw Price, co-founder, Cambridge University Center for the Study of Existential Risk
“Our Final Invention is a thrilling detective story, and also the best book yet written on the most important problem of the twenty-first century.” Luke Muehlhauser, Executive Director, Machine Intelligence Research Institute
“Enthusiasts dominate observers of progress in artificial intelligence; the minority who disagree are alarmed, articulate and perhaps growing in numbers, and Barrat delivers a thoughtful account of their worries.” Kirkus Reviews
“Science fiction has long explored the implications of humanlike machines (think of Asimov's I, Robot), but Barrat's thoughtful treatment adds a dose of reality.” Science News
“This book makes an important case that without extraordinary care in our planning, powerful ‘thinking' machines present at least as many risks as benefits. … Our Final Invention makes an excellent read for technophiles as well as readers wishing to get a glimpse of the near future as colored by rapidly improving technological competence.” New York Journal of Books
“A dark new book by James Barrat, Our Final Invention: Artificial Intelligence and the End of the Human Era, lays out a strong case for why we should be at least a little worried.” NewYorker.com
“You can skip coffee this week Our Final Invention will keep you wide-awake.” Singularity Hub
“Barrat has talked to all the significant American players in the effort to create recursively self-improving artificial general intelligence in machines. He makes a strong case that AGI with human-level intelligence will be developed in the next couple of decades. … His thoughtful case about the dangers of ASI gives even the most cheerful technological optimist much to think about.” Reason
“If you read just one book that makes you confront scary high-tech realities that we'll soon have no choice but to address, make it this one.” The Washington Post
The same broad territory—discussion of various aspects of the Singularity—is covered here as by Nick Bostrom (below) but from the perspective of a documentary filmmaker, rather than a philosopher and academic. Barrat has interviewed key thinkers in the field, including Ray Kurzweil and Arthur C. Clarke, and summarizes his interpretation of their concerns very accessibly.
Cars aren't out to kill us, but that may be a side effect of building cars, writes documentary filmmaker Barrat in this oddly disturbing warning that progress in computers might spell our extinction. Computers already perform essential tasks in our national infrastructure and daily lives, including several beyond the capacity of the smartest individual--e.g., playing chess or competing against humans on Jeopardy. While dazzling, these accomplishments are too specialized for the artificial intelligence the author and the many philosophers, scientists and entrepreneurs he interviews have in mind. Within decades, computers will operate at the speed of a human brain and become rational, allowing them to learn, rewrite their own programs to learn better, solve problems better, make decisions and perhaps create more computers like themselves. Having reached this level, they have achieved artificial general intelligence. Inevitably, working on their own without human input, they will exceed human intelligence by factors of 100 and eventually thousands, achieving artificial superintelligence. Many experts assert that the first ASI machine that humans invent will be our last invention due to the fact that it will leave man's brainpower in the dust. Whether or not designers build friendliness or empathy into these machines (no one is doing that now), no ASI computer is likely to defer to our interests any more than humans deferred to, say, mice, bison or even indigenous tribes as they spread across the world. As researchers on climate change know, warnings of future disasters are a hard sell. Enthusiasts dominate observers of progress in artificial intelligence; the minority who disagree are alarmed, articulate and perhaps growing in numbers, and Barrat delivers a thoughtful account of their worries.