Why startups won't lead the Big Data revolution

Incumbent companies may move more slowly, but they've got all the data

Sorry startups with dreams of data-led disruption, but incumbent companies are in a better position to benefit from the Big Data revolution than you are.

That's according to Kenneth Cukier, data editor at The Economist and co-author of Big Data, speaking at Comptel's Nexterday North conference in Helsinki today.

Startups benefitted from their agility during the early expansion of the web, but larger, legacy companies have a leg up when it comes to Big Data, for one reason: they have the data already.

"In a world of Big Data, what it takes to succeed is access to the data," said Cukier. "Who has the data? Big companies."

However, that head start doesn't guarantee success. Cukier pointed to airlines, which had stacks of data about their customers and the flights they were buying, using it to adjust prices on the fly meaning shopping many months before your travel didn't mean you necessarily got the best price.

Oren Etzioni, the founder of Farecast, realised that he'd paid more than fellow passengers on a flight, and as a computer scientist, wrote an algorithm to figure out the best time to purchase a ticket, Cukier explained. The eventually hoovered up data on every single American flight for a year, some 70 billion data points  before it was bought by Microsoft for $100 million and all but killed off.

The idea has been picked up by others, helping consumers buy flights at lower prices the opposite of what airlines were hoping to do with their data. "Never for a second did the airlines think for a second that their own information could be used to undercut their own business," Cukier noted.  

Of course, for incumbent companies to make use of the data they already hold requires repurposing it for other uses something data protection regulators aren't always keen on. That balance between privacy and the potential of Big Data to bring value and solve problems needs to be solved for such companies to benefit from the "raw natural resource" that's sitting on their systems, he said.

"Who owns all this data anyway," he asked. "The entity who bothered to collect it or the individual from whom it was taken?" That needs to be clearly decided, with accountability and transparency to ensure data isn't abused just because Uber knows your smartphone battery is low doesn't mean it should bump up prices, something that's technically possible but the company has widely chosen not to implement.

"We need to accept Big Data because it leads to benefits, but it can't be a new alchemy, a black box or a religion it's only a representation of reality," Cukier said. "We need to accept Big Data, but it can't come at the expense of our values."

Featured Resources

Virtual desktops and apps for dummies

An easy guide to virtual desktop infrastructure, end-user computing, and more

Download now

The total economic impact of optimising and managing your hybrid multi-cloud

Cost savings and business benefits of accelerating the cloud journey

Download now

A buyer’s guide for cloud-based phone solutions

Finding the right phone system for your modern business

Download now

What’s next for the education sector?

A new learning experience

Download now

Most Popular

KPMG offers staff 'four-day fortnight' in hybrid work plans
flexible working

KPMG offers staff 'four-day fortnight' in hybrid work plans

6 May 2021
Hackers use open source Microsoft dev platform to deliver trojans
Security

Hackers use open source Microsoft dev platform to deliver trojans

14 May 2021
How to move Windows 10 from your old hard drive to SSD
operating systems

How to move Windows 10 from your old hard drive to SSD

30 Apr 2021