The Formula: How Algorithms Solve All Our Problems--and Create More
288
The Formula: How Algorithms Solve All Our Problems--and Create More
288Hardcover
-
PICK UP IN STORECheck Availability at Nearby Stores
Available within 2 business hours
Related collections and offers
Overview
Algorithms exert an extraordinary level of influence on our everyday lives - from dating websites and financial trading floors, through to online retailing and internet searches - Google's search algorithm is now a more closely guarded commercial secret than the recipe for Coca-Cola. Algorithms follow a series of instructions to solve a problem and will include a strategy to produce the best outcome possible from the options and permutations available. Used by scientists for many years and applied in a very specialized way they are now increasingly employed to process the vast amounts of data being generated, in investment banks, in the movie industry where they are used to predict success or failure at the box office and by social scientists and policy makers.
What if everything in life could be reduced to a simple formula? What if numbers were able to tell us which partners we were best matched with – not just in terms of attractiveness, but for a long-term committed marriage? Or if they could say which films would be the biggest hits at the box office, and what changes could be made to those films to make them even more successful? Or even who is likely to commit certain crimes, and when? This may sound like the world of science fiction, but in fact it is just the tip of the iceberg in a world that is increasingly ruled by complex algorithms and neural networks.
In The Formula, Luke Dormehl takes readers inside the world of numbers, asking how we came to believe in the all-conquering power of algorithms; introducing the mathematicians, artificial intelligence experts and Silicon Valley entrepreneurs who are shaping this brave new world, and ultimately asking how we survive in an era where numbers can sometimes seem to create as many problems as they solve.
Product Details
| ISBN-13: | 9780399170539 |
|---|---|
| Publisher: | Penguin Publishing Group |
| Publication date: | 11/04/2014 |
| Pages: | 288 |
| Product dimensions: | 5.80(w) x 8.30(h) x 1.10(d) |
| Age Range: | 18 Years |
About the Author
Read an Excerpt
Acknowledgments
Writing a book is almost always a solitary activity, but I was fortunate enough to be surrounded by a group of people whose love and/or support made The Formula a pleasure to work on. Thanks first of all to Clara, Tim and Celia Lunt, as well as members of my family. I could not have completed this project without the invaluable aid of Ed Faulkner, while it would never have got through the door in the first place were it not for my agent Maggie Hanbury, Henry de Rougemont, Simon Garfield and Jake Lingwood. Many thanks to Marian Lizzi, my U.S. editor. Appreciative nods also go in the direction of all those who spent countless hours speaking with me as part of my research (a full list of their names is printed on page 243), in addition to my FastCo.Labs editor Chris Dannen, Cult of Mac’s Leander Kahney, the excellent Nicole Martinelli, Karl French, tech comms guru Alice Bonasio-Atkinson, Tim Matts, Alex Millington, Michael Grothaus, Tom Atkinson, Simon Callow, and my brothers-from-other-mothers, Andre and Nathan Trantraal. All helped this book along in one way or another. All I can take full credit for are the (hopefully few) mistakes.
An Explanation of the Title, and Other Cyberbole
At their root, algorithms are little more than a series of step-by-step instructions, usually carried out by a computer. However, if their description is straightforward, their inner workings and impact on our lives are anything but.
Algorithms sort, filter and select the information that is presented to us on a daily basis. They are responsible for the search results shown to us by Google, the information about our friends that is highlighted by Facebook, and the type of products Amazon predicts we will be most likely to buy. Increasingly, they will also be responsible for what movies, music and other entertainment look like, which people we are partnered with in predictive relationships, and even the ways in which laws are enforced and police operate. An algorithm can scan through your metadata and recommend that you will likely make a hardworking employee, just as one could accuse you of a crime, or determine that you are unfit to drive a car. In the process, algorithms are profoundly changing the way that we view (to quote Douglas Adams) life, the universe and everything.
One of my favorite observations about technology is the one often attributed to the cultural theorist Paul Virilio: “The invention of the ship was also the invention of the shipwreck.” One could, of course, turn this around and say that the inventor of the shipwreck was also the person that invented the ship. Algorithms have had their fair share of shipwrecks (which I will discuss during the course of this book), but they also perform incredibly useful functions: allowing us to navigate through the 2.5 quintillion bytes of data that are generated each day (a million times more information than the human brain is capable of holding) and draw actionable conclusions from it.
As with the old adage about how to carve an elephant statue (you chip away everything that isn’t an elephant), I will start out by explaining what this book is not. It is not, for one thing, a computer science textbook about algorithms. There are far better books (and, indeed, far more qualified writers) to achieve this task.
Neither is it a history of the algorithm as a concept. While I considered attempting such a thing, I was put off by both the sheer scale of the project and the fact that its end result—while no doubt fascinating under the stewardship of the right author—would be not entirely dissimilar to the textbook I also shied away from. By this I do not mean that a history book and a textbook are necessarily the same thing, but rather that a history of a once-niche mathematical concept would likely appeal only to those mathematicians or computer scientists already familiar with it.
Instead, I want to tell the story of the myriad ways (some subtle, others less so) that algorithms affect all of our lives: from the entertainment we enjoy to the way we think about human relationships. What do scoring hot dates, shooting Hollywood turkeys, bagging your own poo, and cutting opportunities for lawyers’ fees have in common? This is a book about the algorithmization of life as we know it.
In my day job, writing about a field known as the “digital humanities” for Fast Company, I’m constantly considering the implications of “algorithmic” culture and the idea (not always a misguided one) that no matter what the problem, it can be solved with the right algorithm.
A typical illustration of what I mean can be seen in Bill Tancer’s 2009 book Click: What We Do Online and Why It Matters. Tancer—described in at least one place as “the world’s preeminent expert on online [behavior]”—begins his book by describing a radio interview he listened to in the car one day. Being interviewed was a British psychologist referring to a mathematical formula he had developed to determine the most objectively depressing week of the year. After much work, he had discovered that this was the third week of January, a feat brought about by the convergence of failed New Year’s resolutions, credit-card debt accumulated over the holiday season, and the usual dismal weather patterns. Tancer notes that he remained unconvinced: a perspective that was later backed up when the formula was severely criticized for its lack of scientific rigor. However, his lack of conviction has nothing to do with the suggestion that a reductive formula could possibly provide answers on a topic as complex and multifaceted as depression, but rather because he believes that he had come up with a better formula.1
In other words, his problem wasn’t with the existence of the sum, but rather with its working.
This book was spurred by years of hearing similar observations, all claiming that there is no problem technology cannot reduce to its most formulaic level and thereby determine objective answers in response to. This thinking is the reason “The Formula” is in upper case rather than simply existing as a catchall for the various technical processes I describe. It implies an ideological element, and that ideology is evidenced by the more expansive view I take of algorithms and their associated technological apparatus: conceiving of them as the embodiment of a particular form of techno-rationality, symptomatic of a type of social ordering built around the promise of objectivity. In this way I use The Formula much as the late American political scientist and communications theorist Harold Lasswell used the word “technique”: referring, in Lasswell’s words, to “the ensemble of practices by which one uses available resources to achieve values.” It is about both the application and the scope of application, as well as the existence of objective truths lurking just beneath the surface—to be teased out with the right data-mining tools.
Writers on technology tend, with a few notable exceptions, to be overwhelmingly utopian in their outlook. To them, all progress is positive. As a result, there is a tendency among technology writers to christen each new invention as the totemic figurehead of its own “era”—something that has led to the disdainful term “cyberbole.” While this book could very well join the number of volumes about algorithms and big data already lining the shelves, what I am interested in goes back much further than simply the birth of the Internet or the age of the personal computer.
Writing during the first half of the 1960s, the French sociologist (and Christian anarchist!) Jacques Ellul described a creature known as the Technical Man, an individual “fascinated by results, by the immediate consequences of setting standardized devices into motion . . . committed to the never-ending search for ‘the one best way’ to achieve any designated objective.” This objective could occasionally be clouded (or else speeded up) by a naive enthusiasm for the means of getting there: not by anything so unquantifiable as ethical concerns, but rather by an enthusiasm for the ingenuity, elegance and “spectacular effectiveness” of man’s ability to dream up solutions.
As Ellul’s observation proves, this approach is not therefore a new one, and the founders of Google and the heads of the various high-tech companies I discuss are not the first people to display what the late American sociologist Lewis Mumford called the “will-to-order”—meaning the desire to make formulaic sense of the world. Writing in the 1930s, long before the birth of the modern computer, Mumford noted that automation was simultaneously for “enlarging the mechanical or sensory capacities of the human body” and “for reducing to a measurable order and regularity the processes of life.” To make sense of a big picture, we reduce it, he suggested. To take an abstract concept such as human intelligence and turn it into something quantifiable, we abstract it further, stripping away complexity and assigning it a seemingly arbitrary number, which becomes a person’s IQ.
What is new is the scale that this idea is now being enacted upon, to the point that it is difficult to think of a field of work or leisure that is not subject to algorithmization and The Formula. This book is about how we reached this point, and how the age of the algorithm impacts and shapes subjects as varied as human creativity, human relationships (and, more specifically, romantic relationships), notions of identity and matters of law.
Algorithms are very good at providing us with answers in all of these cases.
The real question is whether they give the answers we want.
CHAPTER 1
The Quantified Selves
Larry Smarr weighed close to 200 pounds when he arrived in La Jolla, California, in 2000. The photograph from his driver’s license at the time depicts an overweight 51-year-old with a soft, round face and the fleshy ripple of a double chin. Although he regularly tested his mind as a leading physicist and expert in supercomputing, Smarr had not exercised his body properly in years. He regularly drank Coke and enjoyed chowing down on deep-fried, sugar-coated pastries.1 Moving to the West Coast to run a new institute at the University of California called the California Institute for Telecommunications and Information Technology, he was suddenly confronted with a feeling he hadn’t experienced in years: a sense of deep inadequacy. “I looked around and saw all these thin, fit people,” he remembers. “They were running, bicycling, looking beautiful. I realized how different I was.”
Smarr’s next question was one shared by scientists and philosophers alike: Why? He visited his local bookshop and bought “about a zillion” diet books. None of them was deemed satisfactory to a man whose brain could handle the labyrinthine details of supernovas and complex star formations, but for whom healthy eating and regular exercise seemed, ironically enough, like complex astrophysics. “They all seemed so arbitrary,” he says. It wasn’t until he discovered a book called The Zone, written by biochemist Barry Sears, that Smarr found what it was that he was looking for. Sears treated the body as a coupled nonlinear system in which feedback mechanisms like the glucose-insulin and immune systems interact with one another. That was an approach that Smarr could relate to. Inspired, he started measuring his weight, climbing naked onto a pair of scales each day and writing down the number that appeared in front of him. Next, he hired a personal trainer and began keeping track of the amount of exercise he participated in on a daily basis. After that it was on to diet—breaking food down into its “elemental units” of protein, fat, carbohydrates, fiber, sugar and salt, and modifying his diet to remove those inputs that proved detrimental to well-being. “Think of it like being an engineer, reverse-engineering the subsystems of a car,” Smarr says. “From that you can derive that you need a certain level of petrol in order to run, and that if you put water in your gas tank you will tear apart the car. We don’t think that way about our bodies, but that is the way we ought to think.”
It didn’t take long until Smarr turned to more complex forms of technology to help him lose weight. He purchased and began wearing Polar WearLink heart-rate straps, FitBits, BodyMedia, and other pieces of wearable tech that use algorithms to convert body metrics into data. Wanting to check his progress, Smarr started paying for blood tests at a private laboratory and then—in a quest for yet more numbers to pore over—began FedEx-ing off his stool for regular analysis. “I didn’t have a biology or medical background, so I had to teach myself,” Smarr says of his process.
One of the numbers he wound up fixating on related to complex reactive proteins, which act as a direct measure of inflammation in the body. In a normal human body this number should be less than one. In Smarr’s case it was five. Over time it rose to 10, then 15. As a scientist, he had discovered a paradox. “How was it that I had reduced all of the things that normally drive inflammation in terms of my food supply, but the numbers were growing and growing with this chronic inflammation?” he muses. “It didn’t make any sense.”
At that point, Smarr decided to visit his doctor to present the findings. The appointment didn’t go as planned.
“Do you have any symptoms?” the doctor asked.
“No,” Smarr answered. “I feel fine.”
“Well, why are you bothering me, then?”
“Well, I’ve got these great graphs of my personal data.”
“Why on earth are you doing that?” came the response.
The doctor told Smarr that his data was too “academic” and had no use for clinical practice. “Come back when there’s something actually wrong with you, rather than just anomalies in your charts,” the doctor said.
Several weeks later, Smarr felt a severe pain in the left side of his abdomen. He went back to the doctor’s and was diagnosed with diverticulitis, a disease caused by acute inflammation. It was the perfect illustration of Smarr’s problem: doctors would deal only in clinical symptoms, unwilling to delve into the data that might actually prove preventative. Having learned an important lesson, Smarr decided to take over his own health tracking.
“People have been brainwashed into thinking that they have no responsibility for the state of their bodies,” he says. “I did the calculation of the ratio of two 20-minute doctor visits per year, compared to the total number of minutes in the year, and it turns out to be one in 10,000. If you think that someone is going to be able to tell you what’s wrong with you and fix the problem in one 10,000th of the time that you have available to do the same, I’d say that’s the definition of insanity. It just doesn’t make any sense.”
In the aftermath of the doctor’s visit, Smarr began obsessively tracking any and all symptoms he noticed and linking each of these to fluctuations in his body data. He also upped the amount of information he was looking at, and started using complex data-mining algorithms to sift through it looking for irregularities. Another high number he zeroed in on referred to lactoferrin, an antibacterial agent shed by white blood cells when they are in attack mode, a bit like a canister of tear gas being dispersed into a crowd of people. This number was meant to be less than seven. It was 200 when Smarr first checked it, and by May 2011 had risen to 900. Searching through scientific literature, Smarr diagnosed himself as having a chronic autoimmune disorder, which he later narrowed down to something called Crohn’s disease. “I was entirely led there by the biomarkers,” he notes.
In this sense, Smarr is the epitome of algorithmic living. He tracks 150 variables on a constant basis, walks 7,000 steps each day, and has access to millions of separate data points about himself. As such, both his body and his daily life are divided, mathematized and codified in a way that means that he can follow what is happening inside him in terms of pure numbers. “As our technological ability to ‘read out’ the state of our body’s main subsystems improves, keeping track of changes in our key biochemical markers over time will become routine, and deviations from the norm will more easily reveal early signals of disease development,” Smarr argues in an essay entitled “Towards Digitally Enabled Genomic Medicine: A 10-Year Detective Story of Quantifying My Body.”2
Using his access to the University of California’s supercomputers, Smarr is currently working on creating a distributed planetary computer composed of a billion processors that—he claims within ten years—will allow scientists to create working algorithmic models of the human body. What, after all, is the body if not a complex system capable of tweaks and modifications?
As journalist Mark Bowden observed in a 2012 article for the Atlantic, “If past thinkers leaned heavily on the steam engine as an all-purpose analogy—e.g., contents under pressure will explode (think Marx’s ideas on revolution or Freud’s about repressed desire)—today we prefer our metaphors to be electronic.”
And when it comes to symbolic representations, many people (Smarr included) prefer formulas to metaphors.3
Self-Knowledge Through Numbers
Fitting an “n = 1” study in which just one person is the subject, Larry Smarr’s story is exceptional. Not everyone is an expert in supercomputing and not everyone has the ability, nor the resources (his regimen costs between $5,000 and $10,000 each year) to capture huge amounts of personal data, or to make sense of it in the event that they do.
But Smarr is not alone. As a data junkie, he is a valued member of the so-called Quantified Self movement: an ever-expanding group of similar individuals who enthusiastically take part in a form of self-tracking, somatic surveillance. Founded by Wired magazine editors Gary Wolf and Kevin Kelly in the mid-2000s, the Quantified Self movement casts its aspirations in bold philosophical terms, promising devotees “self-knowledge through numbers.”4 Taking the Positivist view of verification and empiricism, and combining this with a liberal dose of technological determinism, the Quantified Self movement begs the existential question of what kind of self can possibly exist that is unable to be number-crunched using the right algorithms?
If Socrates concluded that the unexamined life was not worth living, then a 21st-century update might suggest the same of the unquantified life. As with René Descartes’ famous statement, cogito ergo sum (“I think, therefore I am”)—I measure, therefore I exist.
In a sense, “Selfers” take Descartes’ ideas regarding the self to an even more granular level. Descartes imagined that consciousness could not be divided into pieces in the way that the body can, since it was not corporeal in form. Selfers believe that a person can be summarized effectively so long as the correct technology is used and the right data gathered. Inputs might be food consumed or the quality of surrounding air, while states can be measured through mood, arousal and blood oxygen levels, and performance in terms of mental and physical well-being.
“I like the idea that someone, somewhere is collecting all of this data,” says Kevin Conboy, the creator of a quantified sex app, Bedpost, which I will return to later on in this book. “I have this sort of philosophical hope that these numbers exist somewhere, and that maybe when I die I’ll get to see them. The idea that computer code can give you an insight into your real life is a very powerful one.”
A typical Quantified Self devotee (if there is such a thing) is “Michael.” Every night, Michael goes to bed wearing a headband sensor. He does this early, because this is when the sensor informs him that his sleep cycles are likely to be at their deepest and most restorative. When Michael wakes up he looks at the data for evidence of how well he slept. Then he gets up, does some push-ups and meditates for a while, before turning on his computer and loading a writing exercise called “750 Words” that asks him to write down the first 750 words that come to mind.5 When he has done this, text-analysis algorithms scour through the entry and pull up revealing stats about Michael’s mood, mind-set and current preoccupations—some of which he may not even be consciously aware he is worrying about. After this, he is finally ready to get moving (using a FitBit to monitor his steps, of course). If he doesn’t carry out these steps, he says, “I’m off for the rest of the day.”6
Robo-cize the World
While QS’s reliance on cutting-edge technology, social networking and freedom-through-surveillance might seem quintessentially modern—very much a creation of post-9/11 America—the roots of what can be described as “body-hacking” go back a number of years. The 1980s brought about the rise of the “robo-cized” athletes who used Nautilus, Stairmaster and other pieces of high-tech gym equipment to sculpt and hone their bodies to physical perfection. That same decade saw the advent of the portable technology known as the Sony Walkman (a nascent vision of Google Glass to come), which transformed public spaces into a controllable private experience.7 Building on this paradigm, the 1990s was home to MIT’s Wearable Computing Group, who took issue with what they considered to be the premature usage of the term “personal computer” and insisted that:
A person’s computer should be worn, much as eyeglasses or clothing are worn, and interact with the user based on the context of the situation. With heads-up displays, unobtrusive input devices, personal wireless local area networks, and a host of other context sensing and communication tools, the wearable computer can act as an intelligent assistant, whether it be through a Remembrance Agent, augmented reality, or intellectual collectives.8
There appear to be few limits to what today’s Quantified Selfers can measure. The beauty of the movement (if one can refer to it in such aesthetic terms) is the mass customization that it makes possible. By quantifying the self, a person can find apparently rigorous answers to questions as broad or specific as how many minutes of sleep are lost each night per unit of alcohol consumed, how consistent their golf swing is, or whether or not they should stay in their current job. Consider, for example, the story of a young female member of the Quantified Self movement, referred to only as “Angela.”
Angela was working in what she considered to be her dream job, when she downloaded an app that “pinged” her multiple times each day, asking her to rate her mood each time. As patterns started to emerge in the data, Angela realized that her “mood score” showed that she wasn’t very happy at work, after all. When she discovered this, she handed in her notice and quit.
“The one commonality that I see among people in the Quantified Self movement is that they have questions only the data can answer,” says 43-year-old Selfer Vincent Dean Boyce. “These questions may be very simplistic at first, but they very quickly become more complex. A person might be interested in knowing how many miles they’ve run. Technology makes that very easy to do. A more advanced question, however, would be not only how many miles a person has run, but how many other people have run the same amount? That’s where the data and algorithms come in. It’s about a quest for knowledge, a quest for a deeper understanding not only of ourselves, but also of the world we live in.”
Boyce has always been interested in quantification. As a New York University student enrolled in the Interactive Telecommunications Program, he once attached some sensors, micro-controllers and an accelerometer to a model rocket and launched it off the ground. “What was interesting,” he says, “is that I was able to make a self-contained component that could be sent somewhere, that could gather information, and that I could then retrieve and learn something about.” After analyzing the rocket’s data, Boyce had his “Eureka!” moment. A lifelong skateboarder and surfer, he decided to attach similar sensors to his trusty skateboard and surfboard to measure the mechanical movements made by each. He also broke the surrounding environment down into hundreds of quantifiable variables, ranging from weather and time of day to (in the case of surfing) tidal changes and wave height. “From a Quantified Self standpoint,” Boyce notes, “I can . . . think about where it was that I surfed from a geospatial type of framework, or what equipment I was using, what the conditions were like . . . [It] represents me doing something in space and time.”
In this way, Selfers return to the romantic image of the rugged individualists of the American frontier: an image regularly drawn upon by Silicon Valley industrialists and their followers. The man who tracks his data is no different from the one who carves out his own area of land to live on, who draws his own water, generates his own power, and grows his own food. In a world in which user data and personal information is gathered and shared in unprecedented quantities, self-tracking represents an attempt to take back some measure of control. Like Google Maps, it puts the individual back at the center of his or her universe. “My belief is that this will one day become the norm,” Boyce says of the Quantified Self. “It will become a commodity, with its own sense of social currency.”
Shopping Is Creating
One of the chapters in Douglas Coupland’s debut novel Generation X, written at the birth of the networked computer age, is titled “Shopping Is Not Creating.”9 It is a wonderfully pithy observation about 1960s activism sold off as 1990s commercialism, from an author whose fiction books Microserfs and JPod perfectly lampoon techno-optimism at the turn of the millennium. It is also no longer true. Every time a person shops online (or in a supermarket using a loyalty card) their identity is slightly altered, being created and curated in such a way that is almost imperceptible.
This isn’t just limited to shopping, of course. The same thing happens whenever you open a new web-browsing window and surf the Internet. Somewhere on a database far away, your movements have been identified and logged. Your IP address is recorded and “cookies” are installed on your machine, enabling you to be targeted more effectively with personalized advertisements and offers. Search regularly for news on a particular sport and you will begin to spot adverts related to it wherever you look—like the murderer played by Robert Walker in Hitchcock’s Strangers on a Train, who sees constant reminders of the woman he killed. Mention the words “Cape Town” in an e-mail, for instance, and watch the flood of “Cheap flights to South Africa” messages flood in.
It was the American philosopher and psychologist William James who observed, in volume one of his 1890 text The Principles of Psychology, that “a man’s self is the sum total of all that he [can] call his, not only his body and his psychic powers, but his clothes and his house, his wife and children, his ancestors and friends, his reputation and works, his lands, and yacht and bank account.”10 This counts for double in the age of algorithms and The Formula. Based on a person’s location, the sites that they visit and spend time on, and the keywords that they use to search, statistical inferences are made about gender, race, social class, interests and disposable income on a constant basis. Visiting Perez Hilton suggests a very different thing from Gizmodo, while buying airline tickets says something different from buying video games. To all intents and purposes, when combined, these become the algorithmic self: identity and identification shifted to an entirely digital (and therefore measurable) plane.
Your Pleasure Is Our Business
Identity is big business in the age of The Formula. The ability to track user movements across different websites and servers has led to the rise of a massive industry of web analytics firms. These companies make it their mission not only to amass large amounts of information about individuals, but also to use proprietary algorithms to make sense of that data.
One of the largest companies working in this area is called Quantcast. Headquartered in downtown San Francisco—but with additional offices in New York, Dublin, London, Detroit, Atlanta, Chicago and Los Angeles—Quantcast ranks among the top five companies in the world in terms of measuring audiences, having raised in excess of $53.2 million in venture capital funding since it was founded in 2006. Its business revolves around finding a formula that best describes specific users and then advising companies on how to best capitalize on this. “You move away from the human hypothesis of advertising,” explains cofounder Konrad Feldman, “where someone theorizes what the ideal audience for a product would be and where you might be able to find these people—to actually measuring an advertiser’s campaign, looking at what’s actually working, and then reverse-engineering the characteristics of an audience by analyzing massive quantities of data.”
Before starting Quantcast, English-born University College London graduate Feldman founded another business in which he used algorithms to detect money laundering for some of the world’s leading banks. “We looked through the billions of transactions these banks deal with every month to find suspicious activity,” he says. It was looking at fraud that made Feldman aware of the power of algorithms’ ability to sort through masses of data for patterns that could be acted upon. “It could represent anything that people were interested in,” he says excitedly. “Finances were interesting data, but it only related to what people spend money on. The Internet, on the other hand, has information about interests and changes in trends on the macro and micro level, all in a single data format.” He was hooked.
“Historically, measurement was done in retrospect, at the aggregate level,” Feldman says of the advertising industry. “That’s what people understood: the aggregate characteristics of an audience.” When Feldman first moved to the United States, he was baffled by the amount of advertising on television, which often represented 20 minutes out of every hour. It was a scattergun approach, rather like spraying machine-gun bullets into a river and hoping to hit individual fish. Whatever was caught was more or less done so by luck. Of course, a television channel can’t change it for every viewer, Feldman explains. The Internet, however, was different. Much like the customized user recommendations on Amazon, Quantcast’s algorithmically generated insights meant that online shopkeepers could redecorate the shop front for each new customer. In this way, audiences are able to be divided into demographics, psychographics, interests, lifestyles and other granular categories. “Yep, we’re almost psychic when it comes to reading behavior patterns and interpreting data,” brag Quantcast’s promotional materials. “We know before they do. We know before you do. We can tell you not only where your customers are going, but how they’re going to get there, so we can actually influence their paths.”
Quantcast’s way of thinking is rapidly becoming the norm, both online and off. A Nashville-based start-up called Facedeals promises shops the opportunity to equip themselves with facial recognition-enabled cameras. Once installed, these cameras allow retailers to scan customers and link them to their Facebook profiles, then target them with personalized offers and services based upon the “likes” they have expressed online. In late 2013, UK supermarket giant Tesco announced similar plans to install video screens at its checkouts around the country, using inbuilt cameras equipped with custom algorithms to work out the age and gender of individual shoppers. Like loyalty cards on steroids, these would then allow customers to be shown tailored advertisements, which can be altered over time, depending on both the date and time of day, along with any extra insights gained from monitoring purchases. “It is time for a step-change in advertising,” said Simon Sugar, chief executive of Amscreen, who developed the OptimEyes technology behind the screens. “Brands deserve to know not just an estimation of how many eyeballs are viewing their adverts, but who they are, too.”11
The Wave Theory
This notion of appealing to users based on their individual surfing habits taps—ironically enough—into the so-called wave theory of futurist Alvin Toffler.12 In his 1980 book The Third Wave, Toffler described the way in which technology develops in waves, with each successive wave sweeping aside older societies and cultures.13 There have been three such waves to date, Toffler claimed. The first was agricultural in nature, replacing the hunter-gatherer cultures and centering on human labor. The second arrived with the Industrial Revolution, was built around large-scale machinery, and brought with it the various “masses” that proliferated in the years since: mass production, mass distribution, mass consumption, mass education, mass media, mass recreation, mass entertainment and weapons of mass destruction. The Third Wave, then, was the Information Age, ushering in a glorious era of “demassification” under which individual freedoms could finally be exercised outside the heaving constraints of mass society. Demassification would, Toffler argued, be “the deepest social upheaval and creative restructuring of all time,” responsible for the “building [of] a remarkable new civilization from the ground up.” And it was all built on personalization.
Please Hold to Be Connected to Our Algorithm
It is well known that not every call-center agent is equipped to handle every type of call that comes in. The larger the company, the less likely it is that any one person will be able to deal with every single inquiry, which is the reason customers are typically routed to different departments in which agents are trained to have different skills and knowledge bases. A straightforward example might be the global company whose call centers regularly receive calls in several different languages. Both callers and agents may speak one or more of several possible languages, but not necessarily all of them. When the French-speaking customer phones up, they may be advised to press “1” on their keypad, while the English-speaking customer might be instructed to press “2.” They are then routed through to the person best suited to deal with their call.
But what if—instead of simply redirecting customers to different call-center agents based upon language or specialist knowledge—an algorithm could be used to determine particular qualities of the person calling in: based upon speech patterns, the particular words they used, and even details as seemingly trivial as whether they said “um” or “err”—and then utilize these insights to put them through to the agent best suited for dealing with their emotional needs?
Chicago’s Mattersight Corporation does exactly that. Based on custom algorithms, Mattersight calls its business “predictive behavioral routing.” By dividing both callers and agents into different personality types, it can make business both faster and more satisfactory to all involved. “Each individual customer has different expectations and behaviors,” Mattersight notes in promotional materials. “Similarly, each individual employee has different strengths and weaknesses handling different types of calls. As a result, the success of a given customer interaction is often determined by which employee handles that interaction and how well their competencies and behavioral characteristics align with each specific customer’s needs.”
The man behind Mattersight’s behavioral models is a clinical psychologist named Dr. Taibi Kahler. Kahler is the creator of a type of psychological behavioral profiling called Process Communication. Back in the early 1970s, Kahler interned at a private psychiatric hospital. While he was there, he created something called a “Miniscript” based on his observations about patients in distress. The work wound up winning him the 1977 Eric Berne Memorial Scientific Award. What Kahler noticed was that certain predictable signs precede particular incidents of distress, and that these distress signs are linked to specific speech patterns. These, in turn, led to him developing profiles on the six different personality types he saw recurring. The personality types are as follows:
Although everyone has all six personality types to a greater or lesser degree, people will respond best to individuals who reflect their own primary personality type. If people’s communication needs are not met by being given the kind of positive “feedback” they require (a feelings-oriented person being asked cold hard facts, for example) they go into distress, which can be diffused only if the person on the other end of the conversation is able to adequately pick up on the warning signals and respond appropriately.
In a call-center environment this knowledge results in an extraordinary qualitative change, according to Mattersight. A person patched through to an individual with a similar personality type to their own will have an average conversation length of five minutes, with a 92 percent problem-resolution rate. A caller paired up to a conflicting personality type, on the other hand, will see their call length double to ten minutes—while the problem-resolution rate tumbles to 47 percent.
Process Communication isn’t only being used by Mattersight, however. In the past, Kahler has helped NASA develop algorithms to aid with the selection of its astronauts, since his model can accurately predict the personality types that won’t crack under the high-pressure atmosphere of space travel. (“Persisters”—who strive for perfection and encourage others to reach their peak performance—prove to be the best personality fit.) Kahler’s company, Kahler Communications, also has a number of ongoing projects designed to help organizations come up with data-driven and algorithmic solutions to questions related to personality.
“From our perspective this is the key to diversity,” says Robert Wert, a former attorney who was employed as the COO of Kahler Communications when I had the opportunity to speak with him. “If all cultures are made up of the same building blocks, all of whom have the same type of interactions both positive and negative, then the real diversity is in personality type. It’s not in ethnicity, it’s not in gender, it’s not in anything else. I see this as the great equalizer. If you can walk into a room and immediately start speaking to someone who’s of a different background to you, and you can identify the same traits in them that you’ve dealt with for the rest of your life, that person is no longer the Other.”
The Lake Wobegon Strategy
Founded in 2011, Gild is a recruitment company that serves some of the tech industry’s biggest and best-known players. Currently focused on automating the discovery of talented programmers, Gild’s mission statement is to apply The Formula to the notoriously unreliable hiring process. To do this, the company uses algorithms to analyze individuals on tens of thousands (soon hundreds of thousands) of different metrics and data points—mining them for insights in what Gild refers to as “broad predictive modeling.”
The success stories the company trots out are impressive. A typical one tells of 26-year-old college dropout Jade Dominguez, who lived off an increasing line of credit-card debt in South Pasadena, California, while teaching himself computer programming.14 After being “discovered” by Gild’s algorithm, he now works as a programmer at the company that found him. His story is hardly unique, either. “These are people whose CVs you wouldn’t look twice at, but who our algorithm predicts would be perfect for the job,” says Vivienne Ming, Gild’s chief scientist. “For some of our customers, that is exactly what they’re looking for. These are companies that are flooded with résumés. They don’t need us to find people; they need us to find different people.”
The first time I spoke with Ming, it was May 2013, and she was sitting in the back of a taxicab on her way to San Francisco International Airport. A tall, striking woman with silver-blue eyes and strawberry-blond hair, Ming is a theoretical neuroscientist with a Carnegie Mellon University pedigree. Effortlessly assured, her geeky engineering side is evidenced by the fact that she wears a prerelease Google Glass headset. In addition to her neuroscience background, Ming’s Twitter profile describes her as an “intrepid entrepreneur, undesirable superhero [and] very sleepy mother.”
Ming is deeply invested in Gild’s utopian vision of turning the workplace into the kind of meritocracy she believes it should be. “This is the way things ought to work, right?” she says, rhetorically. “The person making the hiring decisions really should have an accurate picture of who I am—not just a snap judgment made because I look a certain way. But believe me, the way that people look is a huge influence on hiring.”
Table of Contents
Acknowledgments ix
An Explanation of the Title, and Other Cyberbole 1
1 The Quantified Selves 7
2 The Match & the Spark 61
3 Do Algorithms Dream of Electric Laws? 106
4 The Machine That Made Art 161
Conclusion: Predicting the Future 208
A Note on Author Interviews 243
Notes 245
Index 267
What People are Saying About This
“The clash between humanists and technologists, between brain power and machine power, is an ancient battle. In his lucidly written account of how this clash has played out in past years and how it will unfold in the future, Luke Dormehl is a tour guide with the breadth of a scholar, the sagacity of a judge, and the clear eye of a good journalist. This important book deserves to be read, and digested, by all who wrestle with, and enjoy — or worry about — a world transformed by digital technology.”
—Ken Auletta, author of Googled
“This information-rich narrative is fascinating for experts and laymen alike. A great resource for anyone seeking to understand the intersection of technology and humanity in the 21st century.”
—Publishers Weekly
“This is exactly the type of book we need to be reading as society considers the computerized control of nearly all the systems that affect our lives.”
—Chris Dannen, Fast Company
“A perfect combination of journalism and scholarship ... An essential text for understanding the shimmering boundary between human beings and the machines they create.”
—Stephen Ramsay, author of Reading Machines