High-Throughput with Azure Blob Storage

I am happy to announce that High-Throughput Block Blob (HTBB) is globally enabled in Azure Blob Storage. HTBB provides significantly improved and instantaneous write throughput when ingesting larger block blobs, up to the storage account limits for a single blob. We have also removed the guesswork in naming your objects, enabling you to focus on building the most scalable applications and not worry about the vagaries of cloud storage.

HTBB demo of 12.5GB/s single blob throughput at Microsoft Ignite

I demonstrated the significantly improved write performance at Microsoft Ignite 2018. The demo application orchestrated the upload of 50,000 32MiB (1,600,000 MiB) block blobs from RAM using Put Block operations to a single blob. When all blocks were uploaded, it sent the block list to create the blob using the Put Block List operation. It orchestrated the upload using four D64v3 worker virtual machines (VMs), each VM writing 25 percent of the block blobs. The total time to upload the object took around 120 seconds which is about 12.5GB/s. Check out the demo in the video below to learn more.

GB+ throughput using a single virtual machine

To illustrate the possible performance using just a single VM, I created a D32v3 VM running Linux in US West2. I stored the files to upload on a local RAM disk to reduce local storage performance affecting the results. I then created the files using the head command with input from /dev/urandom to fill them with random data. Finally I used AzCopy v10 (v10.0.4) to upload the files to a standard storage account in the same region. I ran each iteration 5 times and averaged the time to upload in the table below.

HTBB everywhere

HTBB is active on all your existing storage accounts, and does not require opt-in. It also comes without any extra cost. HTBB doesn’t introduce any new APIs and is automatically active when using Put Block or Put Blob operations over a certain size. The following table lists the minimum required Put Blob or Put Block size to activate HTBB.

Azure Tools and Services supporting HTBB

There is a broad set of tools and services that already support HTBB, including:

Conclusion

We’re excited about the throughput improvements and application simplifications High-Throughput Block Blob brings to Azure Blob Storage! It is now available in all Azure regions and automatically active on your existing storage accounts at no extra cost. We look forward to hearing your feedback. To learn more about Blob Storage, please visit our product page.