AWS re:Invent 2022 Storage Service Updates for EFS, S3, Failback, and Backup Announced

AWS re:Invent 2022 opened with a slew of product announcements impacting AI/ML, compute, analytics, containers, databases, and more. This post focuses on updates to some of AWS storage services.

Amazon EFS Elastic Throughput

Amazon EFS Elastic Throughput is a new throughput mode for Amazon EFS designed to provide applications with as much throughput as they need with pay-as-you-use pricing. Throughput mode enables customers to simplify running workloads and applications on AWS by providing shared file storage that doesn’t need provisioning or capacity management.

Elastic Throughput is designed specifically for dynamic and unpredictable workloads with performance requirements that are difficult to forecast. When enabled, Elastic Throughput on an Amazon EFS file system actively manages a file system performance and prevents over-paying for idle resources to ensure performance for your applications.

Enabling Elastic Throughput removes the need to specify or provision throughput capacity since Amazon EFS automatically delivers the throughput performance the application needs. At the same time, customers only pay for the amount of data read or written.

Amazon EFS is built to provide serverless, fully elastic file storage to allow sharing file data for cloud-based applications without thinking about provisioning or managing storage capacity and performance. With Elastic Throughput, Amazon EFS extends its simplicity and elasticity to performance, so customers can run an even broader range of file workloads. Amazon EFS is well suited to support a broad spectrum of use cases, including analytics and data science, machine learning, CI/CD tools, content management, web serving, and SaaS applications.

Amazon EFS Elastic Throughput is available in all Regions supporting EFS except for the AWS China Regions.

Failover Controls for Amazon S3 Multi-Region Access Points

Amazon S3 Multi-Region Access Points provide a global endpoint that spans S3 buckets in multiple AWS Regions. With S3 Multi-Region Access Points, multi-region applications can be built with the same simple architecture used in a single Region. This new feature uses AWS Global Accelerator to monitor network congestion and connectivity and routes traffic to the closest copy of the data. If connectivity between a client and a bucket in a particular Region is lost, the Multi-Region Access Point will automatically route all traffic to the closest bucket (synchronized via S3 Replication) in another Region.

Failover Controls for Multi-Region Access Points

Failover controls let users shift S3 data access request traffic routed through an Amazon S3 Multi-Region Access Point to an alternate AWS Region within minutes to test and build highly available applications for business continuity.

The existing Multi-Region Access Point model treats all of the Regions as active and can send traffic to any of them. The model introduced at AWS re:Invent allows users to designate Regions as either active or passive. Buckets in active Regions receive traffic (GET, PUT, and other requests) from the Multi-Region Access Point; buckets in passive Regions don’t. Amazon S3 Cross-Region Replication operates regardless of the active or passive status of a Region concerning a particular Multi-Region Access Point.

Jeff Barr listed a few things for customers to remember when using the new Failover Control feature.

Active/Passive – There must be at least one active Region at all times.

CLI & API Access – You can initiate a failover programmatically by calling SubmitMultiRegionAccessPointRoutes. You can retrieve the current set of routes by calling GetMultiRegionAccessPointRoutes. The endpoints for these APIs are available in the US East (N. Virginia), US West (Oregon), Asia Pacific (Sydney, Tokyo), and Europe (Ireland) Regions.

Pricing – There is no extra charge for this feature beyond using the new APIs, billed as standard S3 GET and PUT requests. For S3 Multi-Region Access Point usage prices, see the Data transfer tab of the Amazon S3 Pricing page.

Regions – This feature is available in all AWS Regions where Multi-Region Access Points are currently available.

New for AWS Backup – Protect and Restore Your CloudFormation Stacks

To define the data protection policy of an application, it’s essential to look at its components and find which ones store data that need protection. Those are the stateful components of your application, such as databases and file systems. Other components don’t store data but need to be restored in case of issues. These are stateless components, such as containers and their network configurations.

When managing applications using infrastructure as code (IaC), there is a single repository describing all these components. It would be great if that information could be used to help protect applications. AWS Backup now supports attaching an AWS CloudFormation stack to data protection policies.

When using CloudFormation as a resource, all stateful components supported by AWS Backup are backed up around the same time. The backup also includes the stateless resources in the stack, such as AWS Identity and Access Management (IAM) roles and Amazon Virtual Private Cloud (Amazon VPC) security groups.

Now there’s a single recovery point that can be used to recover the application stack or the individual resources. In case of recovery, there’s no need to mix automated tools with custom scripts and manual activities to recover and put the whole application stack back together. As you modernize and update an application managed with CloudFormation, AWS Backup automatically keeps track of changes and updates the data protection policies for you.

CloudFormation support for AWS Backup also helps prove compliance with data protection policies. Monitoring application resources in AWS Backup Audit Manager is a feature of AWS Backup providing auditing and reporting on the compliance of data protection policies. AWS Backup Vault Lock can also be used to manage the immutability of backups as required by the company’s compliance obligations.

Availability and Pricing

AWS Backup support for CloudFormation stacks is available today using the console, AWS Command Line Interface (CLI), and AWS SDKs in all AWS Regions where AWS Backup is offered. There is no additional cost for the stateless resources backed up and restored by AWS Backup. You only pay for stateful resources such as databases, storage volumes, or file systems. For more information, see AWS Backup pricing.

Amazon Redshift Supported in AWS Backup

Amazon Redshift allows customers to analyze data in the cloud at any scale. Amazon Redshift offers native data protection capabilities to protect data using automatic and manual snapshots. This works great by itself, but when using other AWS services, it is necessary to configure more than one tool to manage your data protection policies.

To make this easier, AWS has added support for Amazon Redshift in AWS Backup. AWS Backup allows customers to define a central backup policy to manage the data protection of applications and also protect Amazon Redshift clusters. This provides a consistent experience when managing data protection across all supported services.

In a multi-account setup, the centralized policies in AWS Backup provide the option to define data protection policies across all accounts within the AWS Organizations. To meet regulatory compliance needs, AWS Backup now includes Amazon Redshift in its auditor-ready reports. There is also the option to use AWS Backup Vault Lock to have immutable backups and prevent malicious or inadvertent changes.

Availability and Pricing

Amazon Redshift support in AWS Backup is available today in the AWS Regions where both AWS Backup and Amazon Redshift are offered, with the exception of the Regions based in China. Use this capability via the AWS Management Console, AWS Command Line Interface (CLI), and AWS SDKs.

There is no additional cost for using AWS Backup compared to the native snapshot capability of Amazon Redshift. Overall costs depend on the amount of storage and retention you need. For more information, see AWS Backup pricing.

Automated in-AWS Failback for AWS Elastic Disaster Recovery

When enabled, AWS Elastic Disaster Recovery (DRS) maintains a constant replication posture for the operating systems, applications, and databases. AWS has announced that DRS now supports in-AWS failback, adding to the existing support for non-disruptive recovery drills and on-premises failback.

Because testing and drills are disruptive and time-consuming, they are often overlooked. Add automation and simplification to the mix, which encourages frequent drills at scale to better prepare for disaster. With in-AWS Failback, these tests can be used on-premises or AWS. Implementing non-disruptive recovery drills gives confidence the recovery time objectives (RTOs,) and recovery point objectives (RPOs) will be met in the event a recovery or failback is initiated.

The automated support in this new service provides a simplified and expedited experience to fail back Amazon Elastic Compute Cloud (Amazon EC2) instances to the original Region, and both failover and failback processes (for on-premises or in-AWS recovery) can be conveniently started from the AWS Management Console.

Failover vs. Failback

Failover is switching the running application to another Availability Zone or even a different Region, should outages or issues threaten the application’s availability. Failback is the process of returning the application to the original on-premises location or Region. For failovers to another Availability Zone, customers who are agnostic to the zone may continue running the application in its new zone indefinitely if so required. In this case, they will reverse the recovery replication so the recovered instance is protected for future recovery. However, suppose the failover was to a different Region. In that case, it’s likely customers would want to eventually fail back and return to the original Region when the issues that caused the failover have been resolved.

The below images illustrate architectures for in-AWS applications protected by DRS. The architecture in the image below is for cross-Availability Zone scenarios.

The architecture diagram below is for cross-Region scenarios.

Learn more about in-AWS failback with Elastic Disaster Recovery

As previously mentioned, three new APIs are also available for customers who want to customize the granular steps involved. The documentation for these can be found using the links below.

The new in-AWS failback support is available in all Regions where AWS Elastic Disaster Recovery is available. Learn more about AWS Elastic Disaster Recovery in the User Guide.

Engage with StorageReview

1 year ago

Harold Fritts

I have been in the tech industry since IBM created Selectric. My background, though, is writing. So I decided to get out of the pre-sales biz and return to my roots, doing a bit of writing but still being involved in technology.