IT Pro is supported by its audience. When you purchase through links on our site, we may earn an affiliate commission. Learn more

What Is Big Data?

Is Big Data yet another buzz term? Steve Cassidy ponders what it really means for the IT industry...

And SEO is making matters worse, not better. Once a term is being researched by enough people, the SEO guys start to ask for subject material that hits the sweet spot when simpler, more naf searches are typed into Google. I can't claim to be immune from the effect, because that's what drove the title of this article but I am at least talking directly about the topic, rather than trying to spice up an uninspiring topic with a hot search phrase, in search of ratings or clicks.

As will be clear if you get the Super Crunchers book, the process of data mining pre-dates the whole Big Data concept by many years. Give yourself a relational database, and the process of construction includes a certain intellectual perspective on the information you're collecting.

This is all that very entertaining stuff (I am being ironic) such as Backus-Naur forms, Codd & Date scores, and the meaty and difficult comparisons between relational database competitors. Is SQL Server better at representing your data, or should you choose Oracle, or MySQL? Certainly if you are a hard-hitting entrepreneur then the abstract intellectual view doesn't look like the easy way to the right answer. The key revelation when looking at this issue is that any work that is poorly aligned to the data structure you're stuck with, is going to cost you money.

Superbly efficient data models are needed to store lots of data rapidly, but the better the job done with that, the more expensive the queries that cut across the data in unexpected slices become.

If your lips are moving (mine were, writing that) then here's a usable but inaccurate metaphor. If you are trying to get in or out of a Formula 1 race day, you can join in with the efficient group transport and arrive by bus. But, if some crisis arises, it's just not going to work to attempt to hijack the carefully scheduled and timed bus system the only way back out again is in a helicopter, and they don't come cheap.

Databases are like that or rather, evolved ones are. A mature relational database has been honed and tuned and tweaked to the nth degree, so that (for instance) 5,000 checkout tills in every Tesco in the country can all go "beep" within a single second: That's efficient design. But, when Tesco want to work out how many Baked Beans are sold before 11am, the resulting query has to climb into a computational helicopter. Hourglass icons animate; fans spool up; temporary disks get full; LEDs blink furiously.

Featured Resources

The Total Economic Impact™ Of Turbonomic Application Resource Management for IBM Cloud® Paks

Business benefits and cost savings enabled by IBM Turbonomic Application Resource Management

Free Download

The Total Economic Impact™ of IBM Watson Assistant

Cost savings and business benefits enabled by Watson Assistant

Free Download

The field guide to application modernisation

Moving forward with your enterprise application portfolio

Free Download

AI for customer service

Discover the industry-leading AI platform that customers and employees want to use

Free Download

Most Popular

Why convenience is the biggest threat to your security
Sponsored

Why convenience is the biggest threat to your security

8 Aug 2022
How to boot Windows 11 in Safe Mode
Microsoft Windows

How to boot Windows 11 in Safe Mode

29 Jul 2022
The benefits of a hardware update for SMBs
Sponsored

The benefits of a hardware update for SMBs

2 Aug 2022