Easily build complex reports
Monitoring and efficiency metrics
Custom cost allocation tags
Network cost visibility
Organizational cost hierarchies
Budgeting and budget alerts
Discover active resources
Consumption-based insights
Alerts for unexpected charges
Automated AWS cost savings
Discover cost savings
Unified view of AWS discounts
COGS and business metrics
Model savings plans
Collaborate on cost initiatives
Create and manage your teams
Automate cloud infrastructure
Cloud cost issue tracking
Detect cost spikes
by Emily Dunenfeld
Contents
Two months have passed since the AWS re:Invent announcement of a new EFS (Elastic File System) storage class. EFS Archive is the third EFS storage class, which comes at an even lower price than EFS Infrequent Access. However, just as with EFS Infrequent Access vs EFS Standard, there is a tradeoff between availability and expense.
In one scenario, EFS Archive is 49% less compared to EFS Infrequent Access and 97% less than EFS Standard. In that scenario, we found the breakeven point of EFS Infrequent Access vs EFS Archive. Then, we found even further savings on monthly storage and tiering costs by moving files directly from EFS Standard to EFS Archive.
Note—the costs below apply to Regional File Systems (Multi-AZ) with the default (and recommended) EFS Elastic Throughput mode in the US East region. See the file system section for more information.
EFS is a scalable elastic file storage system, where workloads are scaled up and down automatically as files are added and removed. It stands apart from other file systems by its ease of use—there is no need to provision or manage storage capacity and no deployment, patching, or maintenance is required.
Amazon recommends EFS for a wide range of use cases, including containerized and serverless applications, big data analytics, development and testing, and database backups. Another EFS re:Invent announcement of replication failback and increased IOPS expanded the use cases to include IOPS-demanding workloads, such as machine learning training, fine-tuning, and inference.
There are three storage classes, one high-performance (EFS Standard), and two cost-optimized (EFS Infrequent Access and EFS Archive). When selecting a storage class, consider performance, data access, and cost requirements.
EFS Infrequent Access and EFS Archive have a minimum billable file size of 128 KiB, so files underneath that size will be charged for 128 KiB of storage.
Standard is the high-speed, low-latency option for regularly accessed or modified data workloads. Standard storage is $0.30 per GB-month. Reads are $0.03 per GB transferred, and writes are $0.06 per GB transferred.
Providing the same features, durability, throughput, and IOPS scalability as Standard, the Infrequent Access class is ideal for workloads where the “sub-millisecond latencies” of Standard are not needed. Use this class for data that is accessed a few times a quarter.
As a part of the new announcement of Archive, the storage price for Infrequent Access was decreased by 36%. Infrequent Access storage is now $0.016 per GB-month. For reads, you are charged an additional $0.01 per GB transferred.
Just like Infrequent Access, Archive provides the same features, durability, throughput, and IOPS scalability as Standard. Archive is recommended for workloads where data is accessed a few times a year. Archive storage is just $0.008 a month; however, you are charged an additional $0.03 per GB transferred for reads. Also, there is a minimum storage duration of 90 days for this storage class, meaning if you delete a file before 90 days, you will still need to pay up to day 90.
A comparison of EFS storage classes
Each class contains the same scalability, durability of 99.999999999%, and Availability SLA of 99.99%.
To fully make use of these classes, use Lifecycle management to set lifecycle policies to transition files into lower-cost storage classes after periods of no use. By default, files not accessed for 30 days will be moved into Infrequent Access. Likewise, files that haven’t been accessed for 90 days are moved into the Archive class.
You can also use lifecycle policies to transition files back to Standard. By default, files stay in the low-cost storage classes once they are transitioned to them; however, you can set a lifecycle policy to move files to Standard “On first access.”
Keep in mind that data tiering charges occur per GB transferred, so in certain cases, it may be more cost-effective to set lifecycle policies to transition directly into Archive.
There are two file systems to choose from, with availability and cost being the tradeoff.
With this file system, files are stored across at least three Availability Zones (AZ) within an AWS region. It is recommended for use cases where high availability and durability are needed. The prices we showed in the last section are for the Regional file system.
One Zone stores files in just one AZ within an AWS region. It is recommended for files where availability and durability are less important, since in the case of your AZ being damaged or destroyed, your files could be lost.
One Zone is available for Standard and Infrequent Access at a much lower price. For Standard storage, the price is nearly half off—$0.16 per GB-month. Infrequent Access is $0.0133 per GB-month.
A research team at a large university stores their files with EFS. The project involves simulations and modeling, which has a large output of files, such as simulation results, model outputs, and log files. The amount is roughly 7.5TB.
While the research is active, there are thousands of read and write operations happening daily as researchers across the country review results and adjust models. Let’s assume 37TB is transferred for read operations and 28TB is transferred for write operations in a month. Standard is the best storage class for this portion of the research since files are accessed often, and need to be immediately accessible.
To calculate the cost, consider charges for storage, reads, and writes.
The total monthly cost during active research with Standard is $5,160.96.
Immediately after the research is completed, other than around 50GB of key files, such as the ones summarizing the work, the files are not updated and very rarely accessed. By default, files not accessed for 30 days will be moved to the Infrequent Access class, then after another 60 days, they will be moved to Archive. However, you can configure lifecycle policies or manually move files sooner to save you money.
Let’s calculate the breakeven point for GB transferred during reads of the 7.45TB of files you are considering moving to the Infrequent Access or Archive classes.
In this scenario, if your monthly data read exceeds 211.58TB of transferred data it is most cost-effective to use the Standard class. For read volumes surpassing 2.98TB of data transferred a month, the Infrequent Access class is the cost-optimized choice. If your monthly data transfer for reads is less than 2.98TB a month, use the Archive class.
We now know that if their monthly data read is less than 2.98TB a month to use the Archive storage class. Let’s see just how much they can save with Archive when they only read 20GB a month. The costs would be:
Archive is 97% less expensive than Standard and 49% less expensive than Infrequent Access in this scenario. The last thing to note is the tiering cost per GB transferred. From Standard to Infrequent Access, the cost is $0.01 per GB transferred. From Standard or Infrequent Access to Archive, the cost is $0.03 per GB transferred.
So, for situations like these, we recommend calculating the breakeven point for Infrequent Access vs Archive, and if your monthly data transfer for reads is less than the breakeven point, configure lifecycle policies or manually move files directly from Standard to Archive. This way you will save on both monthly storage charges and tiering costs.
With three distinct storage classes available, each tailored to specific use cases, users face a clear decision-making process. As the affordability of storage classes increases, so does the compromise on availability. Similarly, the choice between file systems leads to a similar conclusion, where One Zone emerges as the preferred option for a budget-friendly, albeit less available, file system.
For use cases not requiring high availability, opting for a cost-optimized storage class can result in substantial cost reductions. If you can predict your monthly data read, you can easily calculate which storage class will be the most cost-effective. Then, if it meets your use case, with further configuration, you can move files into cost-optimized storage classes sooner, saving you money on storage costs. You can also move files directly into Archive from Standard, saving on storage costs and paying twice for data tiering costs.
MongoDB Atlas is the cost-effective choice for production workloads where high-availability is a requirement.
Grafana is a strong competitor to the monitoring and observability features of Datadog for a fraction of the price.
AWS is implementing a policy update that will no longer allow Reserved Instances and Savings Plans to be shared across end customers.