Today, Vantage announces support for OpenAI costs in the Vantage console. Vantage customers can now see their overall OpenAI costs alongside their other infrastructure providers. Customers can add any number of OpenAI organizations from their integrations page and Vantage will automatically ingest and visualize the various model costs accordingly.
OpenAI has taken the world by storm as the new de-facto infrastructure layer for artificial intelligence. The rate at which it is being adopted by not only hobbyists and startups but also enterprises is hard to ignore. Customers of all shapes and sizes are beginning to ask “What is our AI story?” and OpenAI is typically at the center of it. Vantage customers have increasingly asked to get visibility into OpenAI usage and treat it as an infrastructure provider like other providers such as AWS, Datadog, and other “traditional” IaaS providers. Due to OpenAI being an API-based service and following a usage-based billing model, it is very easy for developers to rack up large bills very quickly, and getting visibility into usage was difficult or impossible.
As of today, Vantage is the first FinOps provider to offer OpenAI support. Customers can now grant Vantage access to their OpenAI usage data via the “usage” API endpoint. Upon granting access, Vantage will ingest daily usage data for each OpenAI model. Filters in Vantage have been expanded to allow for Cost Reports to report on OpenAI costs with dimensions of a specific operation, model, and organization. Users can combine OpenAI costs with their other cost providers as well as receive notifications for costs and detected anomalies.
OpenAI is now available in beta to all customers at the time of this blog post. Additionally, you can read more about this integration in the OpenAI section of the Vantage documentation. To connect your OpenAI account to Vantage, head to the Integrations page of account settings.
Frequently Asked Questions
1. What is being launched today?
Today, Vantage is announcing support for OpenAI costs in beta. Vantage users can now provide Vantage with an OpenAI API key and costs will automatically begin to be ingested and visualized within Vantage from the corresponding organization. Vantage will refresh cost data from OpenAI on a daily basis to ensure data is always up-to-date.
2. Who is the customer?
This feature is available to all Vantage users, including users in the free tier. You must have an OpenAI account.
3. What is OpenAI?
OpenAI is an artificial intelligence research company. OpenAI’s research focuses on areas such as natural language processing, computer vision, and reinforcement learning. OpenAI provides AI services and tools to businesses and organizations, including language models like GPT-4, which can generate human-like text.
3. How much does this cost?
There is no additional cost to OpenAI support. However, OpenAI costs will be included in quota tier enforcement. In the event that your OpenAI costs push you over your current tier limit, you may be prompted to upgrade. To see more details on pricing, please refer to the pricing page here: https://www.vantage.sh/pricing
4. How does Vantage technically integrate with OpenAI?
Vantage uses an API key to access the “Usage” API endpoint. This endpoint is currently undocumented as of the publication date of this blog post, but it provides structured cost data broken down by operation (e.g. Chat Completion or Embedding) and model (e.g. GPT-3.5-turbo).
5. Does Vantage have access to run models in my OpenAI account?
Technically, yes, despite the fact that Vantage will never attempt to do this. All API keys generated via OpenAI have revokable read-write access, however, Vantage will never write to any endpoint and only reads from the Usage endpoint (incurs no additional cost to you as an OpenAI customer). This is ultimately a platform limitation of OpenAI for not having more fine-grained permission scoping for API keys.
6. What dimensions can OpenAI costs be filtered by?
OpenAI reports can be filtered by the organization, operation (e.g. “Chat Completion”), and the model (e.g. “GPT-3.5”).
7. What OpenAI services does Vantage support?
We support the following models:
- Language models: GPT-4, GPT-3.5
- GPT-3 Fine-tuning models: Ada, Babbage, Curie, and Davinci.
- Embedding models: Ada.
- Image models: DALL-E.
- Audio models: Whisper.
We support all models of the following operations:
- Completions
- Chat Completions
- Edits
- Images
- Embeddings
- Audio (Translations and Transcriptions)
- Fine-tunes
- Moderations
8. Why is this being released in beta?
The “Usage” endpoint is not currently publicly documented by OpenAI. While we’ve reconstructed cost data using OpenAI’s pricing information, there is currently no way to confirm that our cost data matches the data provided by OpenAI perfectly. Until OpenAI adds this support for their API, we plan to use the “beta” tag to inform customers that the cost usage data is approximate and not precise. In general, we will overestimate your usage costs because we don’t have access to your account information such as credits or free trial periods.
9. My costs don’t match my OpenAI portal - What should I do?
Please contact support@vantage.sh with relevant information from your OpenAI portal, such as screenshots of your total usage, daily usage, and credits.
10. What Cost Report groupings are available for OpenAI costs?
OpenAI costs can be grouped by operation and organization.
11. How often does OpenAI data refresh in the Vantage console?
OpenAI data refreshes daily in the Vantage console.
12. What happens if I remove an OpenAI integration?
If you decide to remove your OpenAI integration from Vantage, all associated costs will be removed from the Vantage console associated with that OpenAI API Key.
13. Can I have multiple OpenAI organization integrations?
Yes, Vantage supports multiple OpenAI organization integrations.
14. Will OpenAI costs be represented on the Overview page?
Yes, OpenAI costs are represented on the Overview page and present in the “Provider Summary” widget.
15. Is there any additional documentation on this integration?
Yes. Documentation on the OpenAI integration can be found here: https://docs.vantage.sh/connecting_open_ai/.
16. I’ve just added my OpenAI organization, how long will it take for my cost data to be present in the Vantage console?
Costs will be ingested and processed as soon as you add the integration. It usually takes less than 15 minutes to ingest OpenAI costs. As soon as they are processed they will be available on your “All Resources” Cost Report.
17. Will historical cost data for OpenAI be available?
Yes, when following the integration setup, you will be configuring 6 months of historical usage data from OpenAI.
18. What other cloud service providers is Vantage adding?
Vantage is planning on adding support for fly.io, Planetscale, and Confluent. Please contact support@vantage.sh with other providers you’d like to see added.
19. I have a request for a cloud service provider who is not on the current roadmap - can I get it supported?
Maybe. Please write to support@vantage.sh as we are prioritizing the next set of cloud service providers.
20. Are there any cost recommendations specific to OpenAI?
Not yet, although in general, it’s better to use the newer models when possible. For example, gpt-3.5-turbo is faster, 10x cheaper, and on par with the GPT-3 Davinci model.
21. Will this integration provide cost data on ChatGPT?
No, we only provide the cost data for your API usage on the operations we support (Completions, Chat Completions, Embeddings, etc.). ChatGPT is a separate product unrelated to your API keys and because it doesn’t consume any of your organization’s API usage, it doesn’t incur costs and won’t show up in a cost report.
