Save on CloudWatch Log Costs With the Infrequent Access Class

CloudWatch can be costly, ranking as the 7th highest of the AWS services on the AWS Cost Leaderboard. A substantial portion of these costs, 43% according to data from Vantage Cloud Cost Reports, can be attributed to logs. Even AWS recognizes that users are making sacrifices in what they are logging to cut back on costs. Gaps in logs can lead to troubleshooting and analysis issues, as well as security and compliance risks.

The introduction of a new log class, the Infrequent Access log class, provides a cost-saving alternative to the Standard log class. The Infrequent Access log class offers significant savings of 50% on ingestion costs due to a reduction in available features. In two scenarios, we found nearly 50% overall savings when ingesting, storing, and analyzing logs.

When to Use - Infrequent Access Log Class

The Infrequent Access log class contains a subset of Standard log class capabilities (organized by category):

• Log Collection and Storage:
  • Managed ingestion
  • Storage
• Metric Collection and Insights:
  • Insights queries (most commands)
  • Cross-account observability
• Data Protection:
  • Encryption with AWS KMS

The Standard log class contains many more advanced capabilities (see next section), which many users simply don’t need.

Therefore, this class is recommended for use cases where logs are accessed infrequently and use ad-hoc querying. Some specific examples are debug logs, web server logs, Internet of Things fleets, and compliance logs. Another important thing to note is this applies only to new workloads as you cannot change the log class after creation.

When to Use - Standard Log Class

The Standard log class is ideal for cases where logs are accessed frequently or advanced features are needed. Advanced features, organized by category, include:

• Metric Collection and Insights:
  • Contributor, container, and Lambda insights
  • Metric filtering and alerting
  • Embedded metric format (EMF)
  • CloudWatch Logs anomaly detection
  • Comparing changes to a previous time range
  • Subscription filters
• Data Protection:
  • Sensitive data protection with masking
  • Exporting to Amazon S3
• Querying:
  • Live Tail
  • GetLogEvents and FilterLogEvents API operations
  • Natural language query assist

Infrequent Access Pricing

The difference in pricing applies only to ingestion costs, which is where the 50% savings come in. The free tier includes 5GB a month of data for ingestion, storage, and logs insights queries.

Applicable Infrequent Access Class CloudWatch Logs Costs

There is also a difference in ingestion costs for Vended Logs. Vended Logs are specific AWS service logs, such as API Gateway Access logs, SageMaker logs, etc., that are automatically generated and published by AWS services for the customer. In many cases, Vended Logs are infrequently accessed, making them a great candidate for the Infrequent Access class.

Data Ingestion Price for CloudWatch Vended Logs

Note that these are the costs associated only with the Infrequent Access class. Visit the pricing page for all other CloudWatch costs, such as metrics and alarms.

Scenarios

Though the 50% savings only apply to ingestion costs, you can still save significantly with the Infrequent Access log class, even when taking into consideration other costs. Let’s go over a couple of scenarios within the capabilities of the Infrequent Access log class.

Scenario 1: Cost-Effective Compliance

A large financial company collects compliance data that, while crucial for regulatory purposes, is not accessed frequently. The primary cost considerations are related to data ingestion and storage. They ingest 1500GB monthly.

This scenario is perfectly suited for the Infrequent Access log class since they would only need to access the data in infrequent cases, such as audits, regulatory checks, or legal investigations.

Ingestion Charges

1500GB - 5GB free tier = 1495GB

1495GB * $0.25 = $373.75 (Infrequent Access)

1495GB * $0.50 = $747.50 (Standard)

Storage Charges

We can assume the data compresses to 300GB

300GB - 5GB free tier = 295GB

295GB * $0.03 = $8.85

Total

$373.75 + $8.85 = $382.60 (Infrequent Access)

$747.50 + $8.85 = $756.35 (Standard)

Infrequent Access costs $382.60 compared to the Standard class cost of $756.35, providing nearly 50% savings.

Scenario 2: Debug Logs Optimization

A software development team generates extensive debug logs for troubleshooting and code optimization purposes. These logs are necessary during development and occasional debugging sessions but are not accessed frequently in a production environment. They ingest 2000GB and analyze 200GB monthly.

Therefore, in this scenario, the Infrequent Access log class is a suitable choice as it offers a cost-effective solution for logs that are accessed infrequently.

Ingestion Charges

2000GB - 5GB free tier = 1995GB

1995GB * $0.25 = $498.75 (Infrequent Access)

1995GB * $0.50 = $997.50 (Standard)

Storage Charges

We can assume the data compresses to 400GB

400GB - 5GB free tier = 395GB

395GB * $0.03 = $11.85

Logs Insights Queries Charges

200GB - 5GB = 195GB

195GB * $0.005 = $0.975

Total

$498.75 + $11.85 + $0.975 = $511.58 (Infrequent Access)

$997.50 + $11.85 + $0.975 = $1,010.33 (Standard)

Infrequent Access costs $511.58 compared to the Standard class cost of $1,010.33, providing nearly 50% savings.

Conclusion

In scenarios where your logs only require minimal access and features, using the Infrequent Access class can save you up to 50%. By significantly reducing ingestion costs, it addresses the concerns of users who were previously either paying Standard class prices, limiting logging for certain cases, or exploring lower-cost solutions from external providers. This class is ideal for use cases where advanced features are unnecessary, specifically in situations where data is infrequently accessed or requires only ad-hoc querying.