AWS Shared Storage: What You Need to Know to Choose the Right Solution
2022-05-09 10:58:27If you thought that cloud adoption has been on the rise, you were right.
Recent data proves that cloud infrastructure services continue to be in high demand. According to analyst Canalys, only in Q4 2021 worldwide cloud investment topped US$50 billion, hitting a new milestone and bringing full-year spending to $191.7 billion.
Unsurprisingly, the big three dominate the market, with Amazon Web Services (AWS) alone accounting for a share of 33%.
AWS has been a popular choice among our clients at Tenesys. As a result, we often get asked about different aspects of migrating, developing, and managing assets and services in the Amazon cloud.
That’s why today I would like to discuss the issue of shared storage solutions in AWS.
But first things first, so let’s start with a crucial question.
Why cloud storage matters
In a nutshell, it’s about the ease of scaling, which is a critical aspect of application architecture.
AWS gives users a range of choices, such as Elastic Kubernetes Service (EKS), Elastic Container Service (ECS), and Auto Scaling Group (ASG). They all allow you to scale applications and adjust resources to the workloads you wish to run in the cloud.
EKS, ECS, and ASG should be sufficient If your application consists of stateless microservices. Of course, you still need to consider issues such as a shared database for microservices or content caching. However, this is where solutions such as CloudFront, RDS, or Elastic Cache can help.
The final thing you need is efficient, scalable, and highly available storage for your files. The obvious choice that usually comes to mind for this task is Amazon Simple Storage Service (AWS S3). It is an object storage service offering scalability, data availability, security, and performance.
But what should you do if your application doesn’t support object storage and only allows for saving files in folders created at the operating system level?
And how can you ensure data sharing between microservices or individual EC2 instances in such a situation?
AWS shared storage solutions to the rescue
AWS responds to this challenge with four different solutions:
Elastic Files System (EFS) is an equivalent of NFS, a network file sharing protocol known from Linux systems.
FSx is, similarly to its predecessor, a network file sharing system based on the NetApp ONTAP, OpenZFS, Windows File Server, and Lustre file systems.
FSx for Lustre is a network file system using the Lustre cluster file system to enable sharing petabytes of data at a speed of 100GB/s.
Elastic Block Storage (EBS Multi-attach) is Amazon’s idea to simultaneously plug a block device into multiple virtual machines.
Read on to find out which of these AWS shared storage solutions will be most efficient and affordable for your project. I will overview their features and performance to give you a better idea of which one to choose.
Delving into AWS share storage solutions
#1: Elastic Files System (EFS)
Amazon Elastic File System is a network file system service characterised by flexibility and ease of use, either with other AWS cloud services or on-premises resources.
The service can scale on-demand storage for petabytes of data, automatically scaling data space up and down as you add or remove files. This feature eliminates the need to constantly monitor and manage capacity to match the volume of stored data.
Amazon EFS has a simple interface that facilitates a quick and easy creation and configuration of file system resources. Thanks to full integration with CloudWatch, users no longer have to deploy, install patches, maintain and monitor complex file system configurations.
Amazon EFS supports Network File System version 4 protocol (NFSv4.1 and NFSv4.0), so it works seamlessly with most applications, tools, and operating systems. Services such as Amazon EC2, Amazon ECS, and AWS Lambda can concurrently use the Amazon EFS file system resource, thus providing a common data source.
#2: FSx and FSx for Lustre
Amazon FSx is a shared file storage service enabling a wide range of uses for applications based on the most popular operating systems – Linux, Windows, and macOS.
Amazon FSx ensures ultra-low latency–under a millisecond–and high throughput to meet the performance requirements of the most demanding processes.
FSx combines the capabilities of commonly used file systems with the ease of AWS interfaces. As a result, it makes it easy to migrate legacy applications to the AWS cloud and easier to build new applications that require high-performance, shared data space.
As part of the service, you can use file systems such as NetApp ONTAP, OpenZFS, Windows File Server, and Lustre.
#3: EBS Multi-attach
Unlike the other two AWS shared storage solutions, EBS Multi-Attach is an Elastic Block Storage service.
This means that it enables you to connect a single block device–Provisioned IOPS SSD (io1/io2)–to multiple virtual machines located in the same Availability Zone (AZ).
You can add multiple multi-attach volumes to each instance or group of instances. Each instance with an attached volume has full rights to read and write files on the shared volume.
Multi-Attach helps you achieve high application availability in Linux-based application clusters requiring parallel write operations.
All in all, AWS brings a fair share of shared storage solutions for you to choose from. Which one you decide to pick will of course depend on your specific project requirements.
Get in touch if you’d like to discuss shared storage or any other issue related to your cloud computing needs. Our team of friendly IT experts will be happy to support you.
Sources:
Canalys – Global cloud services spend exceeds US$50 billion in Q4 2021
https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html
https://docs.aws.amazon.com/efs/latest/ug/whatisefs.html
https://aws.amazon.com/fsx/
About the author: Łukasz Ratajczyk is the head of technology at Tenesys. He has been in this industry for many years and he has extensive experience in IT. He does a lot of things but never says much about them.