What Is Big Data?

Is Big Data yet another buzz term? Steve Cassidy ponders what it really means for the IT industry...

And SEO is making matters worse, not better. Once a term is being researched by enough people, the SEO guys start to ask for subject material that hits the sweet spot when simpler, more naf searches are typed into Google. I can't claim to be immune from the effect, because that's what drove the title of this article but I am at least talking directly about the topic, rather than trying to spice up an uninspiring topic with a hot search phrase, in search of ratings or clicks.

As will be clear if you get the Super Crunchers book, the process of data mining pre-dates the whole Big Data concept by many years. Give yourself a relational database, and the process of construction includes a certain intellectual perspective on the information you're collecting.

This is all that very entertaining stuff (I am being ironic) such as Backus-Naur forms, Codd & Date scores, and the meaty and difficult comparisons between relational database competitors. Is SQL Server better at representing your data, or should you choose Oracle, or MySQL? Certainly if you are a hard-hitting entrepreneur then the abstract intellectual view doesn't look like the easy way to the right answer. The key revelation when looking at this issue is that any work that is poorly aligned to the data structure you're stuck with, is going to cost you money.

Superbly efficient data models are needed to store lots of data rapidly, but the better the job done with that, the more expensive the queries that cut across the data in unexpected slices become.

Advertisement - Article continues below

If your lips are moving (mine were, writing that) then here's a usable but inaccurate metaphor. If you are trying to get in or out of a Formula 1 race day, you can join in with the efficient group transport and arrive by bus. But, if some crisis arises, it's just not going to work to attempt to hijack the carefully scheduled and timed bus system the only way back out again is in a helicopter, and they don't come cheap.

Databases are like that or rather, evolved ones are. A mature relational database has been honed and tuned and tweaked to the nth degree, so that (for instance) 5,000 checkout tills in every Tesco in the country can all go "beep" within a single second: That's efficient design. But, when Tesco want to work out how many Baked Beans are sold before 11am, the resulting query has to climb into a computational helicopter. Hourglass icons animate; fans spool up; temporary disks get full; LEDs blink furiously.

Featured Resources

Application security fallacies and realities

Web application attacks are the most common vulnerability, so what is the truth about application security?

Download now

Your first step researching Managed File Transfer

Advice and expertise on researching the right MFT solution for your business

Download now

The KPIs you should be measuring

How MSPs can measure performance and evaluate their relationships with clients

Download now

Life in the digital workspace

A guide to technology and the changing concept of workspace

Download now

Most Popular

operating systems

17 Windows 10 problems - and how to fix them

4 Nov 2019
Business strategy

The pros and cons of net neutrality

4 Nov 2019
Domain Name System (DNS)

Microsoft embraces DNS over HTTPS to secure the web

19 Nov 2019
social media

Can Wikipedia founder's social network really challenge Facebook?

19 Nov 2019