FinOps as Code with the Terraform CDK (CDKTF)

This is the second post in our FinOps as Code series.

In our last FinOps as Code exploration, we introduced how you can apply Infrastructure as Code (IaC) techniques to the FinOps world. We walked through several FinOps use cases, showcasing how Terraform can automate and optimize cloud financial operations. This blog will go over additional examples for FinOps as Code, using the Cloud Development Kit for Terraform—or CDKTF.

What Is the Terraform CDK (CDKTF)?

IaC developers can use CDKTF to provision infrastructure without needing to learn HCL (HashiCorp Programming Language). CDKTF supports TypeScript, Python, Java, C#, and Go. Developers may find that using CDKTF can help make infrastructure definitions more flexible. Of course, this viewpoint is based on team preference.

Under the hood, CDKTF is based on the AWS Cloud Development Kit (AWS CDK) and uses concepts from the AWS version to translate code into Terraform-compatible configurations. CDKTF is made up of constructs—or the building blocks of the application’s infrastructure. An application is made up of the App class. App contains one or more stacks, which contain resources—or the definition of each infrastructure object.

CDKTF takes the infrastructure you define and synthesizes it into JSON configuration files that are compatible with Terraform. Once deployed, Terraform uses those files to provision your infrastructure. You can use the CDKTF CLI to make deployments, and you can also use the configuration files directly.

Terraform (HCL) vs. Terraform CDK: Comparing IaC Options

Go to any DevOps- or IaC-related forum, and you’ll see that opinions are often mixed on whether to use CDKTF and traditional programming languages or Terraform with HCL. Some users highlight that HCL carries a learning curve you just have to accept:

“When I first started using Terraform, I…got immediately frustrated when I couldn’t apply some basic programming concepts in HCL. After a while…I stopped trying to force it to do something it wasn’t supposed to, and treat it like the declarative language it is.”

—Source

On the other hand, others prefer using CDKTF based on its ability to use constructs and work with higher-level abstractions:

“I am using cdk for terraform and the experience is great. The concept of constructs makes it easier to abstract and reuse things and lot better with programming language vs HCL.”

—Source

The Terraform documentation indicates that CDKTF “is not the right choice for every project” and suggests you should consider using it on a per-project basis. The table below provides a high-level aspect comparison of the two tools and is not meant to be an exhaustive comparison.

Ultimately, the choice comes down to personal preference, team preferences, or project-specific requirements. Programmers just getting started with IaC may be more attracted to CDKTF as it provides a similar experience to what they already know.

Comparing IaC Options: Traditional Terraform vs. CDKTF

FinOps as Code with CDKTF

In our last FinOps as Code article, we discussed the FinOps Principles and how FinOps as Code can help practitioners make their cost optimization strategy even more efficient. The Reports should be accessible and timely principle is particularly pertinent, in the case of CDKTF, where CDKTF provides higher levels of abstraction and can therefore promote even greater efficiency. The HashiCorp documentation suggests CDKTF is particularly useful when creating constructs for a reusable infrastructure “composed of multiple resources and convenience methods.”

The even greater modularity and reusability encouraged by CDKTF provides another method for practitioners to create reusable templates and set infrastructure specifications (e.g., specify resource size limits). All this can be done while keeping cost optimization in mind.

The Teams need to collaborate principle is also exemplified by CDKTF, where teams that prefer to use traditional programming languages vs. HCL can use the tool for creating infrastructure. Depending on a team’s preference or experience, they can instead work with CDKTF based on existing team tools and practices.

In this demo, we’ll focus on creating a reusable cloud cost reporting infrastructure. We’ll showcase how you can use CDKTF to create a Vantage cost reporting construct for managing a Vantage folder, saved filter, and Cost Report.

Step 1: Set Up Project

All demo files are also included in the FinOps as Code demo repo.

Before you use CDKTF, you will need a Vantage API token.

• Store your API token as an environment variable using export VANTAGE_API_TOKEN=an-api-token. You can also store the token in a .env file if you are using a virtual environment.

• Obtain the workspace token for your Vantage workspace using the /workspaces Vantage API endpoint. You can also export this workspace token as an environment variable.

Create a directory for your project, initialize a Python (or your language of choice) CDKTF project, and add the vantage-sh/vantage provider. We’ll also specify the local flag, for this example, to use local state storage.

mkdir vantage-cdktf-example && cd vantage-cdktf-example
cdktf init --template=python --local --providers="vantage-sh/vantage"

After answering a few configuration questions, CDKTF will create the follow directory structure. cdktf.json contains the information that describes the project. imports/ contains the constructs for the Vantage provider. main.py is where we’ll create our final stack.

vantage-cdktf-example/
├── Pipfile
├── Pipfile.lock
├── cdktf.json
├── help/
├── imports/
├── main-test.py
└── main.py

Step 2: Create the Vantage Stack Construct

Create a new file called vantage-stack.py. Open the file in your preferred code editor or integrated development environment (IDE). In this file, we are going to create a Vantage cost reporting construct that we can reuse throughout the rest of our infrastructure.

# vantage-stack.py
from constructs import Construct
from imports.vantage.cost_report import CostReport
from imports.vantage.folder import Folder
from imports.vantage.saved_filter import SavedFilter

class VantageCostStack(Construct):
    def __init__(
        self,
        scope: Construct,
        id: str,
        folder_title: str,
        filter_title: str,
        cost_report_title: str,
        filter_expression: str,
        workspace_token: str
    ):
        super().__init__(scope, id)

        vantage_folder = Folder(
            self,
            "vantage_folder",
            title=folder_title,
            workspace_token=workspace_token
        )

        saved_filter = SavedFilter(
            self,
            "vantage_filter",
            title=filter_title,
            filter=filter_expression,
            workspace_token=workspace_token
        )

        cost_report = CostReport(
            self,
            "vantage_cost_report",
            title=cost_report_title,
            folder_token=vantage_folder.token,
            saved_filter_tokens=[saved_filter.token],
            workspace_token=workspace_token
        )

The above sample performs the following actions:

• Imports the Construct base class to create the new construct. It also imports the Vantage CostReport, Folder, and SavedFilter classes from the Vantage provider.

• Creates the VantageCostStack class. The VantageCostStack class includes a number of parameters that we’ll define each time we call the class.

• Defines a set of cost resources: a folder, a saved filter, and a Cost Report. The Cost Report uses the created vantage_folder and saved_filter as inputs.

Step 3: Create Multiple Cost Reporting Stacks

In main.py, you can now call the VantageCostStack and create multiple instances of a Vantage cost reporting infrastructure.

# main.py
import os
from constructs import Construct
from cdktf import App, TerraformStack
from imports.vantage.provider import VantageProvider
from vantage-stack import VantageCostStack


class MyStack(TerraformStack):
    def __init__(self, scope: Construct, id: str):
        super().__init__(scope, id)

        VantageProvider(
            self,
            "vantage",
            api_token=os.environ["VANTAGE_API_TOKEN"]
        )

        VantageCostStack(
        self,
        "aws_cost_stack",
        folder_title="AWS Costs",
        filter_title="AWS Filter",
        cost_report_title="AWS Report",
        filter_expression="costs.provider = 'aws'",
        workspace_token=os.environ["VANTAGE_WORKSPACE_TOKEN"]
        )

        VantageCostStack(
        self,
        "snowflake_cost_stack",
        folder_title="Snowflake Costs",
        filter_title="Snowflake Filter",
        cost_report_title="Snowflake Report",
        filter_expression="costs.provider = 'snowflake'",
        workspace_token=os.environ["VANTAGE_WORKSPACE_TOKEN"]
        )


app = App()
MyStack(app, "vantage_cdktf_example")

app.synth()

The above code sample performs the following actions:

• Imports the VantageProvider. Also imports the VantageCostStack we just created.

• Instantiates two instances of the VantageCostStack. Each instance contains user-defined values for the cost stack’s parameters. Note that the filter_expression uses VQL as input. VQL is a SQL-like language used to programmatically create Vantage filters. The expression used here is a very basic VQL expression; however, you can create more advanced expressions to filter your Cost Report even further, such as by resource ID, account ID, etc.

• When run, generates the Terraform configuration via app.synth.

On the CLI, run cdktf deploy. You will be prompted to review your infrastructure changes before they are deployed. Once deployed, you should see your two folders, filters, and Cost Reports in the Vantage console.

Next Steps with Terraform CDK and FinOps as Code

This demo showed a basic example of getting up and running with FinOps as Code using CDKTF and the Vantage provider. The most important point to note here is that we were able to create a cost reporting infrastructure that we could reuse. Consider how you can extend this practice in your main infrastructure.

• Customize the VantageCostStack construct further to fit your specific requirements. Consider incorporating additional Vantage resources, such as dashboards, to align with your organization’s cost management goals.

• Instead of hard-coding cloud provider-specific details, reference outputs from provider resources as inputs for your cost stacks. For example, let’s say we created an RDS instance and assigned it to a variable called rds_instance. We can dynamically call the ARN of the created instance within our saved filter (see example below).

  
  filter_expression = (
      "costs.provider = 'aws' AND "
      "costs.service = 'Amazon Relational Database Service' AND "
      f"costs.resource_id = '{rds_instance.arn}'"
  )
  
  

  You’ll have a Cost Report built to specifically track this instance.

Conclusion

This article discussed how to incorporate FinOps as Code with CDKTF. CDKTF offers a flexible and efficient approach to managing cloud financial operations, allowing teams who are less familiar with HCL or Terraform the ability to create infrastructure with familiar programming languages.

By using the CDKTF with a number of providers—like Vantage—you can create reusable constructs for cost reporting and continue to promote a team mindset for cloud cost optimization.