Azure Storage – 5 Areas to Consider Before Diving In

by Ciprian Moroșanu

Storing your data into the cloud is nowadays a no-brainer. For many businesses, Azure offers a wide range of features for a small price. Let’s find out how to easily take advantage of the benefits of Azure Storage.

In this article:

  1. Reliability
  2. Data Security
  3. Limits
  4. Integration
  5. Pricing

 

Long gone are the days in which you had to add disks to your virtual machines, set backup procedures, or implement interfaces for distributed access. Cloud came to the rescue by offering better and better storage solutions for your data.

Due to the fact that the subject may become intimidating large, I’m going to restrict it to the persistence of files in Azure cloud. In my experience, the most frequent types are pdfs, images, and documents. Microsoft offers a nice flowchart for picking up the proper data store.

According to it, the two candidates are Blob Storage (BS) and Azure Files (AF). BS is the default storage location, with better flexibility on performance and a lower cost. We are using it for new implementations. AF is aimed towards file structures and brings support for SMB protocol. This means you can attach it to a VM and it behaves as a drive. It supports the usual disk operations and it benefits from the cloud features. In my experience, I’ve found it useful for “lift-and-shift” scenarios.

Let’s go further into the top features you should know about.

1. Reliability

Simplicity is the prerequisite for reliability. – Edsger Dijkstra

Azure storage makes things as simple as they can be. The last thing you want to end up with is losing your customer data. Add the risk of high unavailability and accidental deletion and you have a recipe for putting the business in danger. Azure Storage handles all of the concerns with minimal configuration.

Redundancy

You are more likely to be struck by lightning than lose your data.

Hardware failures, network or power outages, and massive natural disasters are events that can happen. Azure offers three main options that increase the durability of your data:

Availability goes hand in hand with redundancy. In this regard, the guaranteed SLA is 99.9%, meaning 8.76 h/year. It may look like a lot, but I’ve experienced almost no storage unavailability so far.

Backup

Hope for the best, prepare for the worst.

We are humans and humans do mistakes, be it directly such as accidental deletion or indirectly by introducing bugs that alternate the behavior. It’s useful to know BS is more advanced than AF when it comes to the options it provides:

  • Versioning means that a new version of a blob is stored each time that blob is modified or deleted;
  • Soft deletion is like versioning with a retention period;
  • Snapshots are point in time copies of the blob.

At the moment, AF supports neither versioning nor soft deletion. The way to go is a combination of snapshots and Recovery Service Vault. RSV automates the management of the snapshots according to a configurable schedule. MS offers an overview of this solution and a step-by-step guide of the setup.

2. Data Security

Prepare the umbrella before it rains. Malay Proverb

Before any sensitive files reach storage, engineers consider the following:

  • Communication is done via secure channels.
    By default, the flag “Require Secure Transfer” is enabled, which means that all of the requests to the storage account must be made over HTTPS.
  • Data follows the compliance requirements of your organization.
    Azure Storage is encrypted and decrypted transparently using 256-bit AES encryption, similar to BitLocker encryption on Windows. It even supports customer-managed keys.
  • Access is controlled.

– Every request must be authorized and there are plenty of options to achieve it. The easiest and most frequent way is by providing a Shared Key in the Authorization header.

– One may also want to support anonymous access to some of your resources, such as public images. This is supported only for Blob Storage and it can be as granular as necessary, from containers to blobs.

– The ultimate access control is by generating a shared access signature (SAS). It supports expiration date, whitelisted IPs, different permissions and it can be used in the URL for blobs.

3. Limits

Cloud is the limit.

In the old days, when you were running out of space, you had to install a bigger disk, and then migrate the data. After a while, it usually happened to do the upgrade again. This operation was time consuming and redundant. In the cloud, you can adjust the available storage space by just moving a slider. The limitations of the Azure storage don’t affect the majority of the use cases. Although the cloud boundaries expand over time, it’s better to be aware of the current state before choosing between BS and AF. They are within the limits of the storage account, which got even higher within v2.

Blob Storage* Azure Files**
Max blob/file size 4.75 TB 4 TB
Max capacity 5 PB 100 TB
Max IOPS 50K 100K
Max ingress/egress Up to 50 Gbps / 50 Gbps *** Up to 4 Gbps / 6 Gbps

 

* Limits may be less due to the type of data, configuration, and region. Please check the detailed documents if you expect to hit the boundaries.
** Premium file storage scales with the provisioned share size.
*** Not strictly specified, capped by account.

When it comes to latency, also referred to as response time, the numbers vary depending on conditions. Usually, the premium tiers offer superior performance due to SSDs. Excluding network transfer, we are recording in our projects averages under 10ms in both BS and AF, with fewer spikes for premium AF.

Documentation with all of its details and exceptions may look intimidating. Numbers vary in real life depending on multiple factors. However, we’ve always found them good enough for our scenarios. There were times we couldn’t rely on the promise, so we ended up putting cloud storage to the test… and it passed.

4. Integration

Talk is cheap. Show me the code. Linus Torvalds

The integration is easy, as you’d expect. There are plenty of examples of how to manage your Azure storage resources using your favorite programming language.

Here is a .NET code snippet that downloads an image from BS and retrieves it as an action result.

public async Task<FileStreamResult> OnGet()
{
    var accountName = "***username***";
    var accountKey = "***key***";
    var blobUri = "***blobUrl***";

    var blobClient = new BlobClient(
        new Uri(blobUri), 
        new StorageSharedKeyCredential(accountName, accountKey));

    BlobDownloadInfo blob = await blobClient.DownloadAsync();

    return new FileStreamResult(blob.Content, blob.ContentType);
}

 

Although it’s straightforward, let me explain a bit the steps. First of all, install the Azure.Storage.Blob NuGet package. Get the access key from the Azure portal and pass them to the BlobClient instance along with the Url to the blob. Finally, call the DownloadAsync for the Content and ContentType.

 

You may find useful to keep in mind the following BS object model.

Azure Files supports SMB protocol, which means that a file share can be mounted to a VM. Afterwards, it works as any local file system. You can even restore a previous file version from a shared snapshot.

5. Pricing

All things are worth their price. German proverb

The cost is mostly influenced by three metrics:

  • stored data volume;
  • quantity and type of operations performed;
  • redundancy and backup options.

Azure Files is usually pricier than Blob Storage for non-Premium tier. Standard performance tiers can optimize costs by choosing one of the access tier levels: archive, cool, and hot. Use archive tier if you plan to access the files rarely; this way, the cost for the data volume is lower but the operations become pricier. For the hot access tier, it is the other way around. For example, storing 1TB in BS can cost between a few under 10 euros for Standard performance, Archive access, LRS redundancy, and a bit more than 220 euros for Premium performance, ZRS redundancy.

By knowing your data, its usage, and the redundancy requirements, you will be able to choose the optimum solution in relation to the costs. Azure offers a calculator so that you can simulate an upcoming setup. The configuration doesn’t have to be perfect upfront because most of the settings can be adjusted live, based on the actual production insights.

Summary

Azure Storage proved to be a reliable, secure, and cost-efficient solution for us. It offers a lot of features by default and many more with minimal configuration. The Azure team is working hard to make it even better. Although one may be intimidated by the number of options it provides, you don’t have to be. Give it a try, start small, and learn as you go.

About Ciprian Moroșanu

A graduate of Computer Science BA and Software Engineering MA, Ciprian has over 10 years of experience in developing .NET applications. During this period, he has acquired extensive business knowledge in the domains of payments, healthcare and insurance, and has also taken the role of Scrum Master in his team for the past 5 years, making sure that Scrum is enacted in the team so that they deliver the best results with the available resources.

Share this article