What Is Big Data?

Is Big Data yet another buzz term? Steve Cassidy ponders what it really means for the IT industry...

And SEO is making matters worse, not better. Once a term is being researched by enough people, the SEO guys start to ask for subject material that hits the sweet spot when simpler, more naf searches are typed into Google. I can't claim to be immune from the effect, because that's what drove the title of this article but I am at least talking directly about the topic, rather than trying to spice up an uninspiring topic with a hot search phrase, in search of ratings or clicks.

As will be clear if you get the Super Crunchers book, the process of data mining pre-dates the whole Big Data concept by many years. Give yourself a relational database, and the process of construction includes a certain intellectual perspective on the information you're collecting.

This is all that very entertaining stuff (I am being ironic) such as Backus-Naur forms, Codd & Date scores, and the meaty and difficult comparisons between relational database competitors. Is SQL Server better at representing your data, or should you choose Oracle, or MySQL? Certainly if you are a hard-hitting entrepreneur then the abstract intellectual view doesn't look like the easy way to the right answer. The key revelation when looking at this issue is that any work that is poorly aligned to the data structure you're stuck with, is going to cost you money.

Superbly efficient data models are needed to store lots of data rapidly, but the better the job done with that, the more expensive the queries that cut across the data in unexpected slices become.

If your lips are moving (mine were, writing that) then here's a usable but inaccurate metaphor. If you are trying to get in or out of a Formula 1 race day, you can join in with the efficient group transport and arrive by bus. But, if some crisis arises, it's just not going to work to attempt to hijack the carefully scheduled and timed bus system the only way back out again is in a helicopter, and they don't come cheap.

Databases are like that or rather, evolved ones are. A mature relational database has been honed and tuned and tweaked to the nth degree, so that (for instance) 5,000 checkout tills in every Tesco in the country can all go "beep" within a single second: That's efficient design. But, when Tesco want to work out how many Baked Beans are sold before 11am, the resulting query has to climb into a computational helicopter. Hourglass icons animate; fans spool up; temporary disks get full; LEDs blink furiously.

Featured Resources

Choosing a collaboration platform

Eight questions every IT leader should ask

Download now

Performance benchmark: PostgreSQL/ MongoDB

Helping developers choose a database

Download now

Customer service vs. customer experience

Three-step guide to modern customer experience

Download now

Taking a proactive approach to cyber security

A complete guide to penetration testing

Download now

Most Popular

REvil threatens to release Apple’s hardware schematics

REvil threatens to release Apple’s hardware schematics

21 Apr 2021
How to find RAM speed, size and type

How to find RAM speed, size and type

8 Apr 2021
Samsung Galaxy S21 Ultra review: Ultra in every sense of the word
Mobile Phones

Samsung Galaxy S21 Ultra review: Ultra in every sense of the word

22 Apr 2021