|Product dimensions:||6.40(w) x 9.10(h) x 1.20(d)|
|Age Range:||18 Years|
About the Author
Read an Excerpt
Of all the data in recorded human history, 90 percent has been created in the last two years. However, the need to use and interpret such Big Data has been around for much longer. In fact a the earliest examples of using data to track and control businesses date back 7,000 years, when Mesopotamians used rudimentary accounting to record the growth of crops and herds. Accounting principles continued to improve, and in 1663, John Graunt recorded and examined all information about mortality rolls in London. He wanted to gain an understanding of and build a warning system for the ongoing bubonic plague. In the first recorded example of statistical data analysis a he gathered his findings in the book Natural and Political Observations
Made upon the Bills of Mortality, which provides great insights into the causes of death in the seventeenth century. Because of his work,
Graunt can be considered the father of statistics.
The nineteenth century witnessed the start of the information age.
Modern data was first gathered in 1887, when Herman Hollerith invented a computing machine that could read holes punched into paper cards to organize census data.
THE TWENTIETH CENTURY
In 1937, during Franklin D. Roosevelt administration, the United States created the first major data project to keep track of contributions by more than three million employers and 26 million employees under the new Social Security Act. IBM was awarded the contract to develop a punch card-reading machine for this immense bookkeeping task.
The British developed the first data-processing machine in 1943
to decipher Nazi codes during World War II. The device, named Colossus a searched for patterns in intercepted messages at a rate of 5,000
characters per second. This reduced the time required to perform the task from weeks to merely hours. It was a huge step forward.
In 1952 the U.S. National Security Agency (NSA) was created and, within 10 years, it had contracts with more than 12,000 cryptologists.
They were confronted with information overload during the Cold War, as they started collecting and processing intelligence signals automatically.
In 1965, the U.S. Government decided to build the first data center to store its more than 742 million tax returns and 175 million sets of fingerprints. Employees transferred all those records onto magnetic computer tape that was stored in a single location. The project was later dropped out of fear of “Big Brother,” but it represented the beginning of the electronic data storage era.
Then, in 1989, British computer scientist Tim Berners-Lee developed what eventually became the World Wide Web. He wanted to facilitate the sharing of information through a “hypertext” system.
Little could he know at that moment the impact his invention would have on everyone.
Beginning in the 1990s, data was created at an amazing rate as more and more devices were connected to the Internet. In 1995, the first supercomputer was built; it performed as much work in a second than a calculator operated by a single person could do in 30,000
THE TWENTY-FIRST CENTURY
In 2005, Roger Mougalas of O’Reilly Media coined the term “Big
Data,” a year after the company created the term Web 2.0. He used the term to refer to a large set of data that is almost impossible to manage and process using traditional business intelligence tools.
In that same year, Yahoo! created Hadoop on top of Google’s MapReduce. Its goal was to index the entire World Wide Web;
nowadays, many organizations around the world use the open-source
Hadoop to crunch massive data sets.
As more and more social network sites appeared and Web 2.0 took flight, more and more data was created daily. Innovative startups slowly mined this vast amount of data and governments also began Big Data projects. In 2009, the Indian government decided to take an iris scan a fingerprint, and photograph of all of its 1.2 billion inhabitants. All this data is stored in the largest biometric database in the world.
By 2010, when Eric Schmidt, the Executive Chairman of Google a spoke at the Techonomy forum in Lake Tahoe, California, he put the information revolution in perspective by stating that “every two days now we create as much information as we did from the dawn of civilization up until 2003. . . . That’s something like five exabytes of data. . . .”
In 2011 the well-received McKinsey report on “Big Data: The Next
Frontier for Innovation, Competition, and Productivity,” concluded that by 2018, the United States would face a shortage of 140,000 to
190,000 data scientists, as well as 1.5 million data managers. The job Big Data Scientist is therefore often coined the sexiest job of the twenty-first century.
In the past few years, there has been a massive increase in the number of Big Data startup companies. All are trying to help organizations manage and understand this explosion of Big Data. As more companies are slowly adopting Big Data, just as with the Internet in
1993, the Big Data revolution is still ahead of us, so a lot will change in the coming years.
In fact, the amount of data is growing at such an explosive rate that we have gone past the decimal system. Today, U.S. agencies, such as NSA and the FBI, are talking about yottabytes when calculating the size of their files. In the (near) future, we will be talking about brontobytes regarding sensor data. Therefore, new terms have been created to describe the amount of data that is expected to be created in coming years.
Big Data will completely change organizations and societies around the world. It is expected that the amount of data currently available will double every two years worldwide. So, let’s take a closer look at what Big Data exactly is.
Table of Contents
1 The History of Big Data 1
The Twentieth Century 1
The Twenty-First Century 2
2 What Is Big Data? 5
Explanation of the 7Vs 5
Eight Realities of Big Data You Should Already Know 12
The Impact of Big Data on Society 19
3 Big Data Trends 23
On-the-Go Big Data 25
Big Real-Time Data 31
The Internet of Things 36
The Quantified-Self 43
Big Social Data 46
Public Big Data 50
4 Big Data Technologies 58
Hadoop HDFS and MapReduce 59
Open-Source Tools 60
Big Data Tools and Types of Analysis 62
5 Big Data Privacy, Ethics, and Security 71
Big Data Privacy 73
Big Data Ethics 80
Big Data Security 84
6 Big Data in Your Organization 93
Key Characteristics of Information-Centric
Some Generic Big Data Uses 98
Big Data and Return on Investment (ROI) 104
Big Data on the Balance Sheet 106
Internal Big Data Opportunities 107
Big Data Roadmap 123
Big Data Competencies Required 127
Small and Medium Enterprises (SMEs) Can Achieve
Remarkable Results with Big Data 141
Governance for the Accuracy of Big Data 143
7 Big Data by Industry 149
Agriculture Industry 149
Automotive Industry 153
Consumer Goods 157
Education Industry 161
Energy Industry 165
Financial Services Industry 169
Gaming Industry 174
Healthcare Industry 178
Legal Profession 183
Manufacturing Industry 185
Not-for-Profit Sector 189
Media and Entertainment Industry 194
Oil and Gas Industry 198
Public Sector 201
Retail Industry 207
Telecom Industry 211
Transportation Industry 216
Travel and Leisure Industry 220
8 The Future of Big Data 225
About the Author 278