Dropbox plans SMR deployment to transform its Magic Pocket infrastructure

SMR will be deployed in a quarter of its in-house Magic Pocket infrastructure by 2019

Dropbox has announced plans to deploy shingled magnetic recording (SMR) technology on a massive scale in a bid to transform its in-house cloud infrastructure.

The file hosting platform said deploying SMR drives on its custom-built Magic Pocket infrastructure at exabyte scale will increase storage density, reduce its data centre footprint and lead to significant cost savings, without sacrificing performance.

Dropbox says it is the first company to deploy SMR hard drive technology on such a scale. 

"Creating our own storage infrastructure was a huge technological challenge, but it's already paid dividends for our customers and our business," said Quentin Clark, Dropbox's senior vice president of engineering, product, and design.

"As more teams adopt Dropbox, SMR technology will help us scale our infrastructure by providing greater flexibility, efficiency, and cost savings. We're also excited to make this technology open-source so other companies can benefit from it."

SMR, a hard drive technology that allows tracks on circular disks to be layered on top of one another, will be deployed on a quarter of the Magic Pocket infrastructure by 2019, according to Dropbox, with plans to open source the test software created in this process underway in the coming months.

Magic Pocket is the name of Dropbox's custom-built infrastructure project that was rolled out after the file sharing company decided to migrate away from Amazon Web Services (AWS). The company initially built a prototype as a proof of concept in 2013, before managing to serve 90% of its data from in-house infrastructure in October 2015.

In what Dropbox describes as a "significant undertaking", SMR technology was chosen for its ability to expand disk capacity from 8TB to 14TB while maintaining performance and reliability. Drives were sourced from third-parties, before the company designed a bespoke hardware ecosystem around it, also creating new software to ensure compatibility with Magic Pocket architecture in the process.

"SMR HDDs offer greater bit density and better cost structure ($/GB), decreasing the total cost of ownership on denser hardware," the Magic Pocket and hardware engineering teams explained. "Our goal is to build the highest density Storage servers, and SMR currently provides the highest capacity, ahead of the traditional storage alternative, PMR.

"This new storage design now gives us the ability to work with future iterations of disk technologies. In the very immediate future we plan to focus on density designs and more efficient ways to handle large traffic volumes.

"With the total number of drives pushing the physical limit of this form factor our designs have to take into consideration potential failures from having that much data on a system while improving the efficacy of compute on the system."

Towards the end of the year, the file hosting service says its infrastructure will span 29 facilities across 12 countries, with Dropbox projecting huge cost-saving and increased storage density benefits if SMR deployment is deemed a success.

Image credit: Dropbox

Featured Resources

Five lessons learned from the pivot to a distributed workforce

Delivering continuity and scale with a remote work strategy

Download now

Connected experiences in a digital transformation

Enable businesses to meet the demands of the future

Download now

Simplify to secure

Reduce complexity by integrating your security ecosystem

Download now

Enhance the safety and security of your people, assets and operations

Enable a true vision of security with an engineered solution based on hyperconverged and storage platforms

Download now

Recommended

NordLocker encryption heads to the cloud
encryption

NordLocker encryption heads to the cloud

21 Aug 2020
Fivetran is now worth $1.2 billion thanks to new funding
big data

Fivetran is now worth $1.2 billion thanks to new funding

30 Jun 2020
Best NAS drives 2020
network attached storage (NAS)

Best NAS drives 2020

19 Feb 2020

Most Popular

Accenture ploughs $3 billion into cloud migration support group
digital transformation

Accenture ploughs $3 billion into cloud migration support group

17 Sep 2020
Google takes on Zoom with launch of Meet hardware
video conferencing

Google takes on Zoom with launch of Meet hardware

16 Sep 2020
16 ways to speed up your laptop
Laptops

16 ways to speed up your laptop

16 Sep 2020