Improving AI-Assisted Infrastructure as Code - Maximizing Productivity with Provider Schema Integration
Context
Writing Terraform configurations or any other form of Infrastructure as Code with AI-Assistance lacks the comfort found in other software engineering domains for a simple reason: Cloud providers iterate at relentless speeds, requiring frequent updates to providers on a weekly basis. Backwards compatibility is often an afterthought. Since LLM-models are not updated in the same manner, auto-completion can lead nowhere as resource types and parameters are often completely unknown to the LLM.
Hypothesis
Most AI extensions allow users to extend the context by including the entire workspace or manually selected files. As Terraform allows users to export the schema of a downloaded provider into a JSON file, it should be possible to add the exported schema for context when working with an updated provider, which could result in significantly better outcomes when using chat and auto-completion.
Validation
Scenario
We are going to deploy a resource group, a virtual network including a single subnet, a storage account as well as a private endpoint to connect to the storage account privately all using the latest AzureRM provider from the Terraform Registry(4.24 at the time of writing).
We are going to setup the configuration twice using the same prompts except for the inclusion of the updated schema on second trial.
Setup
Development Container
We're going to fire up a fresh dev container with Terraform and Azure CLI installed and setup Ollama locally to serve deepseek-coder-v2 as LLM both for auto-completion and chat.
First, we're prompting for the latest AzureRM provider version that the LLM is aware of:
As expected, the AzureRM provider known to the LLM is outdated by more than two years.
Install dependencies
After spawning the container, we're creating a main.tf with some bootstrapping information and run terraform init to install all dependencies.
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 4.0"
}
}
}
provider "azurerm" {
features {}
subscription_id = "<SUBSCRIPTION_ID>"
}
Initial Prompt
The following prompt will be used to generate the configuration.
@main.tf Add an Azure Resource Group called rg-ai-0 containing a virtual network called vnet-ai-0 with a single subnet called snet-pe all within germanywestcentral. Include a storage account called saai0 followed by a random string up to length of 24 characters without any special characters all lower case. The storage account should contain a single blob container called demo and the storage account may only be connected using a private endpoint that points to the subnet defined earlier. don't forget to add the necessary private dns zone in the resource group as well.
Results
First Trial
On first trial, the updated schema has not been added to the context. As expected, the resulting configuration won't deploy without manual intervention as the private endpoint is lacking a required parameter.
The issue got caught by the Terraform Language Server, but keeping auto-completion by Linting and AI should be in sync desirably.
Additionally, two deprecated parameters were used.
To make matters worse, public access has not been disabled even though that has been stated explicitly.
Second Trial
On second trial, we're repeating the same bootstrapping process, but after initialization, we'll export the providers schema to the local filesystem using the Terraform CLI terraform providers schema -json > azurerm.json.
After exporting the schema, let's use the prompt from earlier, but add some context from the latest provider:
@main.tf Add an Azure Resource Group called rg-ai-0 containing a virtual network called vnet-ai-0 with a single subnet called snet-pe all within germanywestcentral. Include a storage account called saai0 followed by a random string up to length of 24 characters without any special characters all lower case. The storage account should contain a single blob container called demo and the storage account may only be connected using a private endpoint that points to the subnet defined earlier. don't forget to add the necessary private dns zone in the resource group as well. Make the configuration use the latest provider version for azurerm. The schema has been exported into @azurerm.json.
This time, the mandatory parameter has been added automatically for the private endpoint:
The looming deprecation for the storage_account_name parameter of the storage container resource has not been taken care of by the LLM again.
Conclusion
By adding the current provider's schema to the context window of the LLM used, results for Infrastructure as Code-generation can be improved to maximize developer productivity.
This idea could potentially be extended to other providers and IaC technologies like Pulumi and automated by embedding the schema file to the context permanently.