Azure Storage: How to keep data 'in the clouds'

With the exponential increase in data generated within companies and the increasingly massive adoption of cloud solutions for their storage and subsequent use, it is important to have a clear idea of what, for your business, is the best way to move data “into the clouds”. Microsoft Azure, the Redmond company's cloud computing platform, one of the leaders in the sector, provides a large number of options for storing your data in the cloud and in this article we will give a general overview of all the types available, how they work, their cost and how to start optimizing it with best practices, to make the most of the Azure Storage offer.

What you'll find in this article

  • Azure Storage: a brief introduction
  • Azure Storage: types of storage and what they do
  • Azure Storage Pricing: account types, costs, and optimization
  • Azure Storage: replication and data access
Azure Storage: How to keep data 'in the clouds'

Azure Storage: a brief introduction

Modern businesses are now fully focused on using the cloud for their operations, and this includes not only the adoption of cloud computing through application migration, but also a transition from disk storage to cloud storage.

With the increase in data generated by big data, analytics and the Internet of Things (IoT), companies need effective solutions to store and manage large amounts of information that would once have required significant time, labor and money to create specific on-premise infrastructures for its storage and use.

The competition between cloud service providers to win customers is leading to an unprecedented degree of innovation, and services are becoming easier to use, more robust, scalable and available, and cheaper day in and day out.

In this scenario, Microsoft has certainly not stood idly by with folded arms, but has invested in its Azure Storage, which has thus become one of the leaders in the sector. This cloud service of the Microsoft Azure platform offers storage solutions with practically unlimited capacity combined with the quality and reliability of the digital ecosystems of the Redmond house.

Highly scalable, robust and economical, Microsoft's offer for storing its data supports a variety of clients, such as .NET, Ruby and Java for REST, and access from multiple Windows and Linux operating systems, as well as the possibility of exploiting their functionality even on mobile devices such as tablets and smartphones.

But how does it work exactly? What options does it provide? Let's find out together in the next sections.

Did you know that we help our customers manage their Azure tenants?

We have created the Infrastructure & Security team, focused on the Azure cloud, to better respond to the needs of our customers who involve us in technical and strategic decisions. In addition to configuring and managing the tenant, we also take care of:

  • optimization of resource costs
  • implementation of scaling and high availability procedures
  • creation of application deployments through DevOps pipelines
  • monitoring
  • and, above all, security!

With Dev4Side, you have a reliable partner that supports you across the entire Microsoft application ecosystem.

Azure Storage: types of storage and what they do

One of the most important steps in choosing the type of Microsoft Azure storage is to define what you want to store, how to do it, and what options and functionality are needed to achieve this goal.

There are five types of storage available in Microsoft Azure, divided into two groups.

The first group includes the types of storage accessible through REST APIs and are intended to solve scalability and communication problems between the type of storage used and the applications that access the files. They are respectively:

  • Queue Storage
  • Table Storage
  • Blob Storage

The other group, on the other hand, is used to extend the capabilities of the Microsoft Azure virtual machine environment and for exclusive access from VMs and includes:

  • Azure Files (Although the name may suggest that this is storage for unstructured files, in reality this is not the case. 'File Storage' is an Azure service used to create file shares that work like a file server in the cloud)
  • Disk Storage

To better understand how they work, let's now look at them one by one in a little more detail and try to understand what their main characteristics are.

Queue Storage

Queue Storage is designed to connect the components of your application. It allows you to build flexible applications with independent components that are based on asynchronous queued messaging. This is a convenient service for more advanced users but we will try to give a general overview of how it works.

Let's say we have an on-premise application that interacts with a server somewhere in the cloud. This server could be offline, which means you can't send messages to it anymore and if we tried, it would normally result in an error.

However, all of this can be avoided simply by using a mediator that collects messages while one of the communication partners is offline. With Azure Queues, you have a third element that connects the two components and acts as both a buffer and a mediator.

Azure Queue Storage is essentially a service for storing a large number of messages that can be accessed from anywhere through authenticated HTTP or HTTPS requests. A single queue can hold messages of up to 64 KB.

Azure supports two types of queue mechanisms:

  1. Storage queues. As part of the Azure storage infrastructure, they offer a simple REST-based interface for GET/PUT/PEEK operations with reliable and persistent messaging within and between services.
  2. Service Bus queues. They are part of a larger Azure messaging infrastructure that supports both queuing and more advanced integration models.

Azure Tables (Table Storage)

Azure Table Storage is used to store large amounts of structured data in tables, such as structured NoSQL data. NoSQL is a type of non-relational database that works like a key/value store (unlike Microsoft SQL Server and Azure SQL, which are relational databases). Azure tables have a schema-free design and can be used to store user data for web applications and other metadata.

This type of storage is highly scalable and accessible at a low price and represents a less expensive alternative to Microsoft Azure SQL, if you are comfortable with the associated limitations and have no requirements for the SQL structure and architecture. However, note that the cost increases as the number of file accesses increases.

This database is managed by Microsoft and administrators won't have to worry about the underlying infrastructure in the cloud. So it is not necessary to distribute or install anything, but a first configuration is sufficient to be able to start using it immediately.

Microsoft Azure Blob Storage

Blob stands for Binary Large Object and is a type of data storage for unstructured data (files) such as images, audio, video, backup, etc. It is used to store particularly large files and also for data lakes with high workloads.

Blob Storage is one of the most widely used types of Azure storage, and the data within it is organized in containers similar to a folder structure.

There are three types of Blob Storage: Block Blob, Append Blob, and Page Blob. Let's see them in more detail:

  • Block Blob: Ideal for file storage, with a capacity of 4.77 TB per file. When you store a file as a block blob, it comes into storage in small parts called blocks. Only after the upload is complete, the file/blob is merged into a single piece. With this architecture, the file cannot be modified without a complete new upload. This is the easiest and most affordable way to store files in Azure. You can access the files, but you can't modify them within the repository without completely rewriting them.
  • Append Blobs: Ideal for storing logs or metadata, it can be updated constantly. As we have already said, it is not possible to modify a Block Blob without reloading it. However, there are times when you need to do a lot of input/output operations, such as in databases. Append Blobs were created precisely for this purpose: they are structured so that the user can add a new block at the end of the blob.
  • Page Blobs: Designed for disk storage, Page Blobs are the foundation of the Microsoft Azure virtual machine environment. They were specifically designed with disk limitations in mind: each Page Blob must be a multiple of 512 bytes. The architecture of Page Blobs allows you to write data to each part of a blob. The purpose of Page Blobs is simple: when a VM or local machine operates with local storage (HDD or SSD with disk partitions inside), it operates on various parts of the device, without a linear structure. Actually, if you run any type of disk on a VM in Microsoft Azure, you use Page Blobs.

However, that's not all for blobs. In addition to the three types listed above, there are also access levels to consider.

There are three levels of access to Microsoft Azure Blob Storage, and each offers features to choose from based on the lifecycle of the data stored in the Azure cloud. When selecting Azure Blob Storage storage levels, it is important to first define how often the data will be accessed and for how long it should be kept, since choosing the correct level makes it possible to greatly optimize storage costs:

  • The level called 'Hot' is for the data that you need frequently. It's expensive to store but cheap to access.
  • The 'Cool' level is for data that you need less frequently. It's less expensive to store files than the Hot tier, but it's a little more expensive to access them.
  • The 'Cold' (or Archive) level is for archives. It is extremely cheap to store files there, but very expensive to access them, and access can be very slow due to the low priority of this level.

Azure Files

Although the name may suggest that it is storage for unstructured files, in reality this is not the case. 'File Storage' is an Azure service used to create file shares that work like a file server in the cloud.

So, although the name may suggest storage for unorganized or random files, the service is designed to be a structured solution for sharing and accessing files through the creation of a shared folder.

In this shared folder you can save your files just like you could on a computer's hard drive and you can open them and use them from virtual machines that are in the Azure cloud or from local computers that have permission to access this folder.

Azure Files therefore works like a file server in the cloud that allows you to have your documents accessible from multiple places. It supports two protocols, SMB (Server Message Block) and NFS (Network File System), which are standard methods for sharing files across networks, making it easy for different platforms and operating systems to access and share files stored in Azure.

Azure Disk Storage

Azure Disk Storage is a persistent block-level storage service designed to provide persistent, high-performance storage for virtual machines and applications running on Azure. It's useful when you need to store data that needs to be accessible quickly and reliably, such as operating system disks or application data critical to your business.

These disks can be configured as a Premium SSD (solid state drive) for high performance or as a Standard HDD (hard disk drive) for general purposes, including backup and data storage.

There are two main types of disks in Azure Disk Storage:

  1. Managed Disks: These disks are fully managed by Microsoft. When you create a VM in Azure, you can connect managed disks directly to the VM without having to manually create a storage account. These disks are ideal if you want Microsoft to manage the underlying infrastructure.
  2. Unmanaged Disks: These disks require manual management of the underlying storage infrastructure as part of a storage account. You must then manually create a storage account and manage the disks as page blobs.

Azure Storage: replication and data access

Now that we have finished choosing the type of storage that suits our needs, one last concern remains: in the IT world, data security is among the most essential and fundamental things.

When you store data in Microsoft Azure, regardless of the type of storage, it is stored somewhere in Microsoft data centers and, to ensure its availability and preservation, Azure Storage creates and stores copies of the data in multiple locations with the objective of providing redundancy to protect your information from hardware failures, power or network interruptions.

By default, Azure Storage replicates data three times within a primary region. In addition, Azure offers the following options that you can configure for replication within the selected primary region:

  • Locally Redundant Storage (LRS): synchronously replicates data across three disks within a data center in the primary region. It offers a moderate level of availability at a lower cost.
  • Zone Redundant Storage (ZRS): synchronously replicate data between three Azure Availability Zones in the primary region. It provides a higher level of resilience at a higher cost.

But suppose the occurrence of a catastrophic scenario that could tear down the entire primary region (unlikely but still an possibility). If you want to cover every possibility, there are two other types of replication available in Azure Storage:

  • Geo-redundant storage (GRS): stores three other copies of the data in an associated Azure region. Your data is then stored a total of six times, with three secure copies in another region hundreds of kilometers away in case of catastrophic scenarios. Copies in the other region will become available for reading in case those stored in the primary region become inaccessible.
  • Geo-redundant storage with read access (RA-GRS): same as GRS, but allows you to read data from both Azure regions.

Azure Storage Pricing: account types, costs, and optimization

Azure Storage uses a pay-as-you-go service model that fits your business, whether you're storing just a few hundred GB or trillions of objects.

The service provides different types of storage accounts, each with its own list of features and pricing models. It's important to understand the differences between these accounts before you start using Azure Storage for your applications and make the right choice for your company's needs. The main options are:

  • Generic account v2: A type of basic storage account that supports blobs, files, queues, and tables. This type of account is sufficient for most use cases.
  • BlockBlobStorage Account: A premium high-performance account focused on optimizing block blobs and append blobs. This type of account is recommended for high transaction frequency scenarios, smaller objects, and other scenarios that require consistently low latency.
  • FileStorage Account: A storage account that supports only Azure Files, but with improved performance for large scale enterprise applications.

Azure storage is priced based on capacity, the number of storage transactions, and the amount of data transferred. Azure Storage rates are comprised of the following three key elements:

  1. Bandwidth: The data transfer rate in the storage account area. Managed services can be placed in the same location, providing free bandwidth between computing services and storage services. You only pay for the use of your access bandwidth when you access the storage service outside your location.
  2. Transactions: The number of requests made on your storage account. REST requests are generated for each storage service (blob, table, and queue) and are considered billable.
  3. Total capacity: The sum of the data in persistent storage. Azure calculates the total capacity of stored blobs, entities, messages, apps, and metadata to determine total capacity.

Other factors that could affect service costs are the geographical region of the account, the specified data usage pattern (GPv2, Blob, etc.), the number of copies and the type of replication chosen, and the total amount of outgoing data transferred.

For all those companies that want to take advantage of the Azure storage services offer, it is therefore important to follow some guidelines and implement the best usage strategies in order to maximize the cost/efficiency ratio. Below we offer you a small list of 6 tips to begin to understand how to do:

  1. Understand your Azure storage needs: choose the right type of account, the right options for redundancy and evaluate what you want to do with your data. Do we just want to keep them, or do we want to be able to access them and modify them and add items on a regular basis? All these questions allow us to understand what is the best solution for our needs.
  2. Optimize archive and compute size to minimize waste: Having almost unlimited storage and computing capacity is convenient, but this doesn't have to be an invitation to laziness or carelessness, so let's take time to precisely identify the storage and computing resources needed to support applications and workloads without exceeding.
  3. Careful cleaning: Terminate computing resources that are no longer needed and delete unconnected volumes (Azure managed disks) once the related computing services have been deleted through the Azure dashboard.
  4. Check the status of the virtual hard disks: Within your Azure Storage Account, we are looking for 'Virtual Hard Drive'. We will find the disk properties in the Overview tab. If the state of a disk is' unlocked 'and its location status is' available ', it means that the resource is not connected to any volume and we can safely delete it.
  5. The right load for the right disc: maintaining less I/O-intensive workloads on Standard disks instead of Premium or Ultra Disks allows you to drastically reduce the cost of transactions.
  6. Reserved Capacity: Once we have a rough idea of the amount of storage needed, we take advantage of Azure Storage's reserved capacity options to benefit from discounts related to commitments to reserve a number of resources (for a certain number of months or years) offered by Microsoft.

Conclusions

With their variety of options, able to adapt to the needs of companies of all types and sizes, Azure Storage storage spaces are a safe and reliable choice for all those companies that want to take their data “to the clouds”.

The underlying infrastructure, fully managed by Microsoft, then frees organizations from the burden of having to spend precious time and money on the purchase, management and maintenance of on-premise infrastructure and allows their IT departments and data engineers to focus on more important tasks, significantly increasing their operational efficiency.

Over the years, Azure has become one of the leading platforms in the cloud computing sector for many valid reasons, and we can safely say that its storage services are among them. All that remains is to invite you to touch its potential and let the storage offer of the Microsoft cloud computing platform speak for itself.

FAQ on Azure Storage

What is Azure Storage?

Azure Storage is Microsoft's cloud storage solution that provides highly available, secure, and scalable storage services for a wide range of data types, including files, blobs, queues, and tables.

What are the main types of Azure Storage?

Azure Storage offers several storage types, including Blob Storage for unstructured data, File Storage for managed file shares, Queue Storage for message queuing, and Table Storage for NoSQL data.

How does Azure Blob Storage work?

Azure Blob Storage is designed to store large amounts of unstructured data such as images, videos, and documents. It organizes data in containers and allows for massive scalability and data accessibility over HTTP or HTTPS.

What is the benefit of using Azure File Storage?

Azure File Storage provides fully managed file shares in the cloud that are accessible via the SMB protocol. This service is ideal for applications that require shared file access and can integrate seamlessly with existing on-premises environments.

How can Azure Queue Storage be utilized?

Azure Queue Storage is used for storing large numbers of messages that can be accessed from anywhere via authenticated HTTP or HTTPS requests. It is commonly used to build scalable and resilient distributed systems.

What makes Azure Table Storage unique?

Azure Table Storage is a NoSQL key-value store that is ideal for storing structured, non-relational data. It allows for fast access to large volumes of data and can scale to meet the needs of applications that require high throughput.

How does Azure Storage ensure data security?

Azure Storage secures data through encryption at rest and in transit. It also offers advanced security features such as role-based access control (RBAC), virtual network integration, and private endpoints to protect data from unauthorized access.

What are the pricing models for Azure Storage?

Azure Storage offers flexible pricing models, including pay-as-you-go, reserved capacity, and storage tiers like Hot, Cool, and Archive. Each model is designed to optimize cost based on data access patterns and retention needs.

How does Azure Storage support data redundancy?

Azure Storage provides multiple redundancy options such as Locally Redundant Storage (LRS), Zone-Redundant Storage (ZRS), Geo-Redundant Storage (GRS), and Read-Access Geo-Redundant Storage (RA-GRS) to ensure data durability and availability.

Can Azure Storage be integrated with other Azure services?

Yes, Azure Storage integrates seamlessly with various Azure services, such as Azure Virtual Machines, Azure Functions, and Azure Data Lake, enabling a cohesive cloud environment for building and deploying applications.

Find out why to choose the team

Infra & Sec

The Infra & Security team focuses on the management and evolution of our customers' Microsoft Azure tenants. Besides configuring and managing these tenants, the team is responsible for creating application deployments through DevOps pipelines. It also monitors and manages all security aspects of the tenants and supports Security Operations Centers (SOC).