Today global enterprises are collecting, storing and working with
unimaginable amounts of data and information. According to experts, global
organizations and enterprises now gather in excess of 2.5 quintillion
bytes of data everyday.
The raw data originates from sources such as mobile devices,
electronic transactions, social media conversations, electronic sensors,
remote cameras, RFID readers and the like.
This tsunami of raw information is currently being referred to as
“Big Data.” IT professionals around the globe are now being challenged to
manage, organize and analyze these vast data sets strategically.
Big Data refers to a very large data set , no conventional database
management tool or information management can really work. Indeed, we
procreate about 2.5 trillion bytes of data every day.
This information come from everywhere : messages we send or videos
we publish , weather information, GPS signals, transaction records of
online purchases and much more.
This data is named Big Data or massive data volumes. The giants of
the Web, the first of which Yahoo (but also Facebook and Google), were the
very first to deploy this technology.
However, no specific or universal definition can be given to Big
Data. Being a polymorphic complex object, its definition varies between
communities who are interested in as a user or service provider.
The database marketplace is defined by the following
Storage for data : this is primarily hardware, and even though Big
Data uses less expensive hardware, it uses a lot of it. I suggest this
will go upwards as we deploy more and more supercomputing platforms. In
particular, if we start seeing lots of changes into solid state drives for
data nodes. We just will get more for our money in this market
Servers for databases : this is the high-end servers and the
licensing fees with the supporting consulting. This probably will see
changes due to the impact of open source licensing.
Business intelligence : this is the marketplace for traditional data
warehousing. This segment will also see a lot of changes. The more
traditional OLAP solutions will certainly be replaced by supercomputing
platforms. But the number might very well increase as former BI solutions
migrate to become the backbone technology for many global
Advanced analytics : this is a moneymaker. Absolutely this small
market segment will increase. There will particularly be a lot more
dollars spent in terms of consulting and training.
Data integration : another sure winner. There's a lot of stranded
data to be rescued and these are tough jobs with a lot of challenges.
There will be a lot of new software tools and a lot of small niche
companies emerging in this space.
Text analytics : another small segment which may come with some
interesting surprises. There are a handful of very specialty companies
here, but any one of them could bring forward a remarkable solution with
Challenges of Security and Privacy
All of us understand that technology in and of itself is amoral.
It's the people who use it that determine its morality. There's an
impressive and tremendous list of positive uses for Big Data. But there
are just as many possible immoral, bad, and criminal uses for it.
Big Data can be very destructive in the wrong hands. Right now,
today, we have to defend against an organized criminal effort, using
supercomputing platforms to conduct financial crimes.Entire industries are
coming under threat from these criminal attacks.
Let me give you just one small example: Big Data technology is being
used by organized crime to now run a cyber scam. The criminals use the
platform to identify victims, normally elderly, and their relationship to
new relatives who are traveling to foreign countries. They then make a
call, and impersonate foreign officials asking for immediate payment to
post bail, or to pay for urgent medical care.
The collection of Big Data gives them enough data to make the con
work. The issue of security is on the top list of engineering and
operational challenges to be dealt with in any Big Data
Big Data may be the arms race of the 21st century. In 2009 the US
military stood up an entire command structure called Cyber Command. The
issue is again the collection of vast amounts of data that can be used to
attack the economy, infrastructure, and personnel of the
It is proposed that this will be the battlefield of the next war,
and there may be no shots fired to bring down the enemy. Today there are
real threats of using cyber blackmail to bend an enemy to your
One example of high concern in the United States is the electrical
grid system. It is not as tightly secured as it should be, and it could be
penetrated and controlled by those with hostile intent. We can expect
ever-increasing amounts of engineering, resources, and money to be spent
on cyber warfare.
Is Big Data the end of privacy as we know it?
The answer is fundamentally yes. Each of us in the modern world
leaves an ever-increasing digital footprint, which is detailed and
complete. This footprint is only going to get larger and even more
connected. There is a number of companies that make revenue by tracking
every click, and every second you spend on the Internet.
The number of companies, government agencies, and research
organizations that track and use the telephony data from mobile phones is
growing rapidly. They track every movement of a switched on mobile phone,
and store all this collection into a Big Data solution.
Now the correlation of tracking data to psychosymmetrics is the
heart of recommendation engines, such as those used by Netflix and
These recommendations offer value, and they do make our lives easier
in a very increasingly complicated world. But we are giving up a lot of
what is traditionally considered privacy.
The Age of Big Data is here, and these are truly revolutionary times
if both business and technology professionals continue to work together
and deliver on the promise.