Big Data: are we ready?

Beyond the hype, what Big Data really means for humanity

As writer Kenneth Cukier explains in a Ted Talk, there is currently lot of hype around the term “Big Data”, an important tool by which society is going to make advances in a range of fields.

When you have a large body of data, you do not just understand "more", you understand better, you see new things that were previously not apparent. When you have big data, you can fundamentally do things that you couldn’t do when you only had smaller amounts.

In the past, what did data look like physically? It looked like an inscription on a rock disk. We still make data, but now it is easier to store, copy, share, process. We can now use this information in ways that we never imagined possible when we first collected the data. In other words, the data has gone from something that is stationary to something that is fluid and dynamic. There is now a liquidity to information. Previously, there was something informational, but never put it into a data format: to know the location of somebody you would have to follow that person around and note it down. Currently, we know that there is a database entry of where most people have been at all times, as your cellphone records it. Location has been “data-fied”.

So what is the value of big data? It finds its most impressive use in Machine Learning, which is a branch of AI, which is a branch of computer science. Instead of instructing the computer what to do, we throw data at the problem and tell the computer to work it out, understand it by seeing its origins.

In the 1950s, a computer scientist called Arthur Samuel gave the rules of chess to a computer, plus a sub-program for strategy. He could still outplay the computer. Then he let the computer play against itself, thus giving it increased data to work on its prediction. He played it again, and loses. He had created a computer that surpassed his ability in a task that he taught it.

To provide a current-day example, automated vehicles are empowered to be smart enough to drive by supplying them with large amounts of data in order to let the vehicle figure out itself how to maneuver. Other examples include search engines, Amazon’s personalization rhythm and computer translation. In medicine, researchers have looked at cancerous biopsies, asking the computer to identify by looking at data and survival rates to determine if the cells are cancerous or not. By throwing the data at the machine, the machine could work out the 12 telltale signs. Amazingly, 3 of those were unknown to doctors, having been previously unidentified as telltale signs.

Some claim that big data and machine learning are the only way this planet is going to deal with global challenges: food supply chains, medical care supply chains, energy infrastructure optimization, predictive maintenance, accident prevention.

What are the potential downsides of this technology? It has been ventured that fundamental rights could be infringed upon by certain uses of big data, including criminal profiling i.e. using algorithmic data as seen in the movie Minority Report.

While it is acceptable for geo-locatable data to be used as a sign for potential crimes, there are other more personal data sets that could be “useful”: sleep patterns, school performance, web-surfing habits, Fitbits to show when a pulse spikes to indicate aggressive thoughts, to predict criminal behavior.

"This brings to light some potential challenges for humans. Privacy was the challenge in the small data era; in the big data era, the challenge will be safeguarding human free-will, moral choice, human volition, human agency."

There is another issue: big data and algorithms are going to challenge white-collar professional knowledge work in the 21st century in the same way that machinery challenged blue collar workers in the 20th century. Not does one person lose their job, but an entire fleet of trained people and their positions as stakeholders in society will be disrupted. We like to think that technology creates jobs, but there are some categories of jobs that are eliminated and never come back. Therefore we will needs to take big data and adjust it for our human needs; we need to learn how to handle the data we collect, not only on government but corporate level.

Big Data is going to transform how we work, how we live, how we think. In the past, we have looked at Information Technology, we have always focused on “Technology” as it appears as a physical thing. In fact we are now casting our gaze towards “Information”, less apparent, but in some ways more important. Humanity has always been on a quest to understand our place in it, now we can finally learn from the information it can collect.

Source: Kenneth Cukier, Ted Talks