The Big Facts about Big Data

You'd have to have been living in the Big Brother house or in the middle of a jungle to be unaware of the emergence of Big Data as a top discussion topic. Like cloud computing a few years previously, it's generated a lot of marketing hype and, just like cloud computing, organisations have been hard-pushed to define it.

There's no standard definition of the subject, at least not one that has been widely adopted as the NIST definition of cloud has been. Research firm IDC's is probably as viable as any. It defines Big Data as a new generation of technologies and architectures, designed to economically extract value from very large volumes of a wide variety of data by enabling high-velocity capture, discovery, and/or analysis.

Go to any conference and the Big Data sessions will be spilling over as managers are reluctant to miss out on the emerging trend. No-one is in any doubt that future enterprises will be looking at adopting some Big Data techniques as they attempt to analyse the vast volumes of information at their disposal to derive business value.

On the other hand, as with any new trend, there are also large numbers of people sceptical about Big Data's effectiveness. Indeed, when a technology is hyped so much, there is going to be a certain amount of cynicism.

There are two contrasting views on the importance of Big Data. But what both groups have in common is a belief in a number of myths about Big Data and shaky belief in some of the fundamentals.

The myths

Myth 1: It's not about sheer volume

The name Big Data is itself a misnomer as if the sheer size of the database is the problem. But that's not the only factor at play. According Alan Priestley, strategic marketing director for Intel in Europe, the Middle East and Africa (EMEA). There are other elements too: notably, the variety of data and the speed with which the data needs to be delivered. Companies also need to be aware whether the data is accurate in the first place.

Myth 2: It's not just about social media

Much of the debate around Big Data has centred on the effect that social media data has on organisations. It's easy to see why it's assumed that this is the case: most of the media attention has focused on the stereotypical business looking to get up-to-date information on its customers. Naturally enough these days, this means looking at social media engagements: the likes of Twitter, Facebook, Instagram et al.

But, as Priestley points out, far more common for businesses is machine-generated data: network logs, data centre logs and other information.

"Airlines can also leverage the power of Big Data. For example, they can take the data from an aircraft travelling through the sky and analyse it to try to predict any problems. In the past, it would have examined the engine after a certain number of hours or after a failure. You don't want to have a failure but if you're examining after a failure, that's too late," he says.

"By using Big Data in this way, they can track vibrations in an engine. It can examine the data produced and if it looks to be divergent from the norm, it can send an alert so that engine can be examined."

The other example of successful handling of Big Data described by Priestley is BMW. The car manufacturer has a number of its cars connected to the internet via 3G. By using Big Data and associated analytics it can track those cars and contact the owner.

But there are a multitude of other examples: credit card companies checking for fraudulent transactions in real time, ensuring that a purchase made on the other side of the world is legitimate and doing so in a matter of seconds.

Intel itself is major user of Big Data techniques. The company uses Big Data to check the effectiveness of its wafer-making plants, meaning that it can reduce wastage and cut costs considerably.