Introducing Amazon Sneakernet aka AWS Import/Export

Posted by on Jul 8, 2011 | No Comments

With gigabit connections available on the intranets of many businesses, it is easy to forget that it can still be faster to transfer large datastores by physically moving a copy on disk from one location to another on a disk. Back when I started in IT, walking files from one part of the building to another was frequently faster because there was a high probability the person on the receiving end wouldn’t know what to do with what you sent. I still regularly ship video files in a FedEx envelope any time the total upload time from my location is longer than the delivery time, but until today, uploading massive data collections to your AWS account required a combination of patience and a wicked-fast Internet connection.

AWS Import/Export addresses large data migrations by offering you a means of shipping a physical drive to an AWS facility and using the internal Amazon network to more quickly load your data into an EBS volume. Current capacity limits are set at 16 TB for a device and a weight of 50 pounds. Amazon covers return shipping, which is likely due to their existing economy of scale from Amazon.com orders. For all but the enterprise customers, shipping your data may make sense for anything over 1 TB, because most small and medium sized businesses top out at 20-30 megabit connections out of their office, with an even lower actual limit due to having a backplane that caps you at something less. If you can achieve a true 10 megabits coming out of your upload infrastructure, you would be looking at about ten days in a best case scenario to get your data into your AWS volume. If there’s a network hiccup, your time frame may double or triple.

For many projects, a week of data upload is a serious blocking issue. Longer is even worse. AWS Import/Export eliminates the possibility of network hiccups or an connection bottleneck by putting your data at the source. Pricing for AWS Import/Export is a reasonable $80-99.00 per device, depending on the region you are transferring data into. The rate for ingesting data from your device is $2.49-2.99 per hour. Considering the trade-off of potential developer and engineering downtime, those rates are a steal.

While I haven’t tried AWS Import/Export, I applaud Amazon for making the feature available. I know there have been many occasions where I could have benefited from loading data in more quickly, both into my AWS infrastructure and into other data centers. AWS Import/Export should keep Amazon competitive with Azure and RackSpace Cloud in providing a robust set of tools for building projects with larger datasets.