Easily build complex reports
Monitoring and efficiency metrics
Custom cost allocation tags
Network cost visibility
Organizational cost hierarchies
Budgeting and budget alerts
Discover active resources
Consumption-based insights
Alerts for unexpected charges
Automated AWS cost savings
Discover cost savings
Unified view of AWS discounts
COGS and business metrics
Model savings plans
Collaborate on cost initiatives
Create and manage your teams
Automate cloud infrastructure
Cloud cost issue tracking
Detect cost spikes
by Vantage Team
Contents
TL;DR: Amazon S3 offers a service called S3 Intelligent-Tiering that has the potential to save you money by shifting S3 objects between various S3 storage classes on your behalf. However, there are accompanying fees and use cases where this will actually cost you more money. This post helps break down the mechanics of how S3 Intelligent-Tiering works in simple terms.
Amazon Simple Storage Service (S3) is one of the most popular services that AWS provides and the third highest customer spend on our AWS Cost Leaderboard. AWS has made it dead simple for you to push files (which AWS refers to as objects) up to the cloud into namespaces (which AWS refers to as buckets) that allow you to pay for only what you use. Despite its popularity, S3 is one of the most complicated services from a pricing perspective as it charges you on a variety of dimensions including monitoring costs, the HTTP requests for accessing the service, as well as, the underlying storage utilized. This blog post covers the topic of S3 Intelligent-Tiering which has potentially drastic cost-saving implications for S3 storage in particular, with the ability to save you over 50% depending on your objects’ access patterns.
Before diving into what S3 Intelligent-Tiering is and how it works, you should be comfortable with understanding the concept that S3 offers various different storage types that can be set on a per-object basis. If you’re familiar with S3 storage classes you can skip to the next section.
S3 storage classes cover different use cases, offering varying degrees of durability and access performance, and as a result, different pricing. By default, all objects are stored in the S3 Standard Storage class, which happens to be the second most expensive storage class behind S3 Express One Zone. All other storage classes offer varying, lower levels of durability or access performance.
This post won’t go in depth on the various S3 storage classes and how they’re priced, but there is a separate blog post where you can learn more about this written here, or you can read brief summaries in the S3 Storage Class section of the Cloud Cost Handbook.
S3 Intelligent-Tiering in simplistic terms is an automated service that S3 provides that will monitor access patterns of S3 objects and change their respective storage classes on your behalf to get you automatic cost savings. Objects uploaded or transitioned to S3 Intelligent-Tiering are automatically stored in the Frequent Access tier which has the same pricing as S3 Standard Storage. As objects are transitioned through the tiers the storage cost decreases and the access time increases.
The five storage classes that S3 Intelligent-Tiering transitions objects in between are:
Note: Archive Access and Deep Archive Access are not automatically enabled and need to be configured for objects to move to these tiers.
Now that you understand the concept of S3 storage classes and what S3 Intelligent-Tiering is, the first question is typically: “How much can this save me?” This question is answered by three key factors:
The easiest way to think about possible savings is by understanding the relative pricing between the five storage classes that S3 Intelligent-Tiering can possibly place your objects in. Below is a table of relative savings percentages to S3 Standard Storage based upon prices in us-east-1:
* Note that Frequent Access has multiple tiers within it ranging from $0.023 to $0.021 per GB month. For simplicity's sake for this blog post, we use the rate for the smallest amount of storage.
Ultimately, only you know your objects’ access patterns. If all of your objects are being regularly accessed, you’re not going to yield any savings through S3 Intelligent-Tiering. If your access patterns are incredibly seldom then over time you can save over 95% on storage costs by leveraging the Deep Archive Access tier.
The final consideration mentioned above is “How many objects you’re monitoring with S3 Intelligent-Tiering.” For how that impacts potential cost savings, the entire section below covers this.
S3 Intelligent-Tiering has monitoring fees associated with it of $0.0025 per 1,000 objects per month. Note that this fee will occur when you turn on S3 Intelligent-Tiering but it will take time for objects to transition from one storage class to another to realize savings. Ultimately, your savings rate will be impacted by the number of objects that are being monitored by S3 Intelligent-Tiering. To calculate S3 Intelligent-Tiering Monitoring Fees, you can use the following equation (with one example) below:
S3 Intelligent-Tiering Monitoring Costs per Month = # of Objects With S3 Intelligent-Tiering / 1000 Objects * $0.0025
$2.50 per month = 1,000,000 / 1000 * $0.0025
When you think of your possible net savings rate as it relates to S3 Intelligent-Tiering, you should try and gauge how much savings you think you’ll yield from having objects transition from one storage class to another. Remember to factor in the monthly fee for the number of objects you’ll have S3 Intelligent-Tiering monitor. This can be tricky as S3 Intelligent-Tiering fees are based upon the number of objects monitored whereas the potential savings are derived from the storage of the underlying objects.
If you had 1,000,000 objects that in aggregate were storing roughly 45TB at a total storage cost of about $1,000 per month and you think half of that storage will transition to the Infrequent Access tier (roughly half the cost of the Frequent Access tier), your net savings rates would be about:
Net Savings Rate = (Total Costs Before S3 Intelligent-Tiering - Total Costs After S3 Intelligent-Tiering) - Monthly Tiering Fee
≈$247.5 in monthly savings = ($1000 - $750) - $2.50
Timelines for S3 Intelligent-Tiering transitions are important to take note of as potential savings will be realized over the span of multiple months. It’s very likely for your first month of S3 Intelligent-Tiering, your S3 costs will *go up.* Below is a view of these timelines and how long it may take you to realize savings for transitioning objects from one storage class to another.
Frequent Access Tier
Objects uploaded or transitioned to S3 Intelligent-Tiering are automatically stored in the Frequent Access tier.
Infrequent Access Tier
Objects that have not been accessed for 30 consecutive days transition to the Infrequent Access tier.
Archive Instant Access Tier
Objects that have not been accessed for 90 consecutive days transition to the Archive Instant Access tier.
Additionally, you can configure the optional storage classes for even more savings.
Archive Access Tier
Objects that have not been accessed for 90 consecutive days transition to the Archive Access tier.
Deep Archive Access Tier
Objects that have not been accessed for 180 consecutive days transition to the Deep Archive Access tier.
When an object is accessed from the Infrequent Access tier or one of the archive tiers it is automatically moved back to the Frequent Access tier.
There are a few final considerations to be aware of when it comes to S3 Intelligent-Tiering to level set expectations:
MongoDB Atlas is the cost-effective choice for production workloads where high-availability is a requirement.
Grafana is a strong competitor to the monitoring and observability features of Datadog for a fraction of the price.
AWS is implementing a policy update that will no longer allow Reserved Instances and Savings Plans to be shared across end customers.