Today, Vantage announces support for Confluent costs in the Vantage console. Vantage customers can now see their overall Confluent costs alongside their other infrastructure providers. Customers can add any number of Confluent organizations from the Integrations page, and Vantage will automatically ingest and visualize the costs accordingly.
Customers typically use Confluent alongside their primary infrastructure provider to power their real-time data streaming applications. Confluent is usage-based and needs to be scaled with the amount of event data flowing through the various clusters and topics. This can cause costs to run away as systems scale. In addition, having to log in to multiple providers’ cost dashboards (e.g., AWS, Azure, and Confluent) to perform FinOps tasks is cumbersome.
Now, customers can grant Vantage secure access to their Confluent billing data via the Confluent Costs API, accessible through a provided Confluent Service Account API key. Upon granting access, Vantage will ingest invoice line items, which are used to calculate overall costs and credits. Filters in Vantage have been expanded to allow for Cost Reports to report on Confluent costs with dimensions of a specific service, cost category, resource, and organization. By combining filters for Confluent and the infrastructure provider where the compute portion of their application is running, users are able to see the combined costs for their entire application deployment in a single view.
Filtering on Confluent and Redshift costs in one Cost Report.
Confluent is now available to all customers at the time of this blog post. Additionally, you can read more about this integration in the Confluent section of the Vantage documentation. To connect your Confluent account to Vantage, head to the Integrations page of account settings.
Frequently Asked Questions
1. What is being launched today?
Today, Vantage is announcing support for Confluent costs. Vantage users can now provide Vantage with a Confluent API key and secret, and costs will automatically begin to be ingested and visualized within Vantage from the corresponding organization. Vantage will refresh cost data from Confluent on a daily basis to ensure data is always up-to-date.
2. Who is the customer?
This feature is available to all Vantage users, including users in the free tier. You must have a Confluent account.
3. What is Confluent?
Confluent offers tools and services for Apache Kafka, a real-time data streaming system. Their platform enables its customers to easily access, store, and manage data as continuous, real-time streams.
4. How much does this cost?
There is no additional cost to Confluent support; however, Confluent costs will be included in quota tier enforcement. In the event that your Confluent costs push you over your current tier limit, you may be prompted to upgrade. To see more details on pricing, please refer to the Pricing page.
5. How does Vantage technically integrate with Confluent?
Vantage uses an API key and secret to access the Billing API. This API key must be granted the OrganizationAdmin role in order to access this API. This endpoint provides structured cost data, broken down by category (e.g., Kafka or Connect), resource (e.g. Kafka Cluster), and subcategory (e.g., Kafka Storage or Connect Capacity). All credentials are stored as encrypted.
6. Does Vantage have write access to my Confluent account?
Given the limited permission options available from Confluent, yes, Vantage needs to be granted write access, despite the fact that Vantage will never attempt to do this. All API keys generated via Confluent have revocable read-write access; however, Vantage will never write to any endpoint and only reads from the Usage and Organization endpoint (incurs no additional cost to you as a Confluent customer). This is ultimately a platform limitation of Confluent until they have more fine-grained RBAC roles for its API keys.
7. What dimensions can Confluent costs be filtered by?
Confluent reports can be filtered by the Organization, Service (e.g., Kafka), Resource, and Category (e.g., Kafka Storage).
8. What Confluent services does Vantage support?
Vantage supports the following services:
- Kafka
- Connect
- KSQL
- Audit logs
- Cluster Links
- Custom Connect
- Support Cloud
9. Are there any active resources available for Confluent?
Yes, your Confluent Kafka Clusters will be synced as active resources and available in resource reports.
10. My costs don’t match my Confluent portal — what should I do?
Please contact support@vantage.sh with relevant information from your Confluent Cloud portal, such as screenshots of your total usage, daily usage, and credits.
11. What Cost Report groupings are available for Confluent costs?
Confluent costs can be grouped by service, category, account (organization), and resource.
12. How often does Confluent data refresh in the Vantage console?
Confluent data refreshes daily in the Vantage console.
13. What happens if I remove a Confluent integration?
If you decide to remove your Confluent integration from Vantage, all associated costs will be removed from the Vantage console associated with that Confluent API key and secret.
14. Can I have multiple Confluent organization integrations?
Yes, Vantage supports multiple Confluent organization integrations. You will have to add each one as a separate integration.
15. Will Confluent costs be represented in the Overview page?
Yes, Confluent costs are represented in the Overview page and present in the “Provider Summary” widget.
16. Is there any additional documentation on this integration?
Yes. Documentation on the Confluent integration can be found in the Product Documentation.
17. I’ve just added my Confluent organization, how long will it take for my cost data to be present in the Vantage console?
Costs will be ingested and processed as soon as you add the integration. It usually takes less than 15 minutes to ingest Confluent costs. As soon as they are processed, they will be available on your “All Resources” Cost Report.
18. Will historical cost data for Confluent be available?
Yes, when following the integration setup, you will be configuring six months of historical usage data from Confluent.
19. What other cloud service providers is Vantage adding?
Vantage is planning on adding support for fly.io, PlanetScale, and Cloudflare. Please contact support@vantage.sh with other providers you’d like to see added.
20. I have a request for a cloud service provider that is not on the current roadmap — can I get it supported?
Maybe. Please write to support@vantage.sh as we are prioritizing the next set of cloud service providers.
21. Does Vantage account correctly for credits and discounts?
Yes, these can be toggled on and off from the Settings dropdown of a Cost Report.
