Dependency Management in Infrastructure as Code
Intro
This is the first part in a series of two posts outlining alternatives when managing dependencies between different Infrastructure as Code (IaC) configurations. First, we will outline a vision for configuration data management before diving into options for managing dependencies between Terraform configurations. The second part will cover a Proof of Concept implementation of a centralized configuration service used to model inter-configuration dependencies.
Long-Term Vision
The Challenge of Scaling Cloud Platforms
Scaling Infrastructure as Code (IaC) or Configuration as Code (CaC) is challenging due to the heterogeneous nature of different providers and tools involved. Most importantly, all elements of a modern cloud platform are interdependent and require orchestration. This orchestration necessitates a mechanism for providing information from one tool, pipeline, or configuration output to the next.
Central Configuration Service
One way to address this challenge is by building a central configuration service that allows storing both static data and data generated from configurations serving as single point of truth for configuration data. This service can be accessed via a CLI tool or an API. By augmenting the service with change notifications (e.g., webhooks or events), you can build workflows to model dependencies.
One product implementing such a service—or at least parts of it—is Pulumi ESC. For the rest of this post, we will use Pulumi ESC as an example; however, other solutions are also valid options.
Scenario
Within this post, we focus on minimizing the problem space to make it more accessible. Given that Terraform/OpenTofu is currently the leader in cloud configuration adoption, we will trim down the issue of describing dependencies between Terraform resources. The solution should implement a reasonable separation of concerns, minimize blast radius, prevent race conditions, and enable future tool integration.
Concepts
The Terralith
Recently, I came across the term "Terralith," borrowed from monolithic application architectures. In a Terralith, resources of different domains are managed within the same Terraform configuration. This approach often leads to cluttered configurations that are hard to read and maintain, increasing the blast radius in case of misconfigurations. Additionally, separation of concerns is not possible as the whole configuration uses the same context (accounts, secrets, ...). Despite these downsides, single configurations allow for easy modeling of dependencies and are resistant to race conditions if written cautiously.
Remote States
In the early days of Terraform, the concept of remote states was state-of-the-art. Remote states provide the state file of another Terraform configuration to be used as a data source from within the primary configuration (hence the name "remote state"). While easy to implement and understand, challenges arise mainly in two areas:
- Race conditions: If the required value is not present yet in the referred state file (as it hasn't been applied yet), the configuration will fail if no error-handling is implemented.
- Separation of concerns: Within a state file, Terraform makes no distinction between secret and other values. By allowing different teams to read each others' Terraform state, maintaining security becomes difficult.
Terragrunt Dependencies
With the rise of Terraform, an ecosystem emerged around it, providing CLI-based orchestration tools like Terragrunt or Terramate (among others). Terragrunt implements a concept called dependencies that couples two or more Terraform configurations based on mapping one configuration's output values to another configuration's inputs. By checking the dependency chain each time the configuration is invoked, Terragrunt refreshes the dependencies, preventing race conditions.
Unfortunately, as both configurations share the same execution context, separation of concerns is not achieved using this solution. Additionally, scaling this solution beyond Terraform/Terragrunt is not trivial.
Pulumi ESC
While being developed mainly with Pulumi IaC in mind, Pulumi ESC can be integrated with Terraform as well. Pulumi ESC can be used as a configuration store that organizes configuration data within so-called environments in a structured data format usually exposed in YAML- or JSON-Format. Using the CLI-Tool, environments can be exposed and updated from a local shell. This allows to populate Terraform-Variables from environment data. By writing configuration output into the environment, configuration results can be persisted. While being superior in certain areas like Single Source of Truth, separation of concerns, scalability, orchestration must be implemented manually (we'll cover that area in the second part).
Summary
The following table summarizes the pros and cons of the solutions mentioned above.
| Concept | Pros | Cons |
|---|---|---|
| Terralith | - Simplifies dependency modeling. - Resistant to race conditions if executed carefully. |
- Clutters configurations, making them hard to maintain. - No separation of concerns; risks security issues due to shared context. |
| Remote States | - Enables state sharing between Terraform configs. | - Prone to race conditions without error handling. - Security risks as different teams may access each other's states, exposing sensitive data. |
| Terragrunt Dependencies | - Prevents race conditions via dependency refreshes | - Shared execution context leads to a lack of separation of concerns. - Scaling beyond Terraform/Terragrunt can be complex and non-trivial. |
| Pulumi ESC | - Provides a single source of truth for configuration data. | - Manual orchestration required, increasing complexity. - Primarily designed for Pulumi |
What all solutions are lacking is a mechanism for declaring the relationship between configurations bi-directionally, which is actual common in Software Engineering ("contract"). While for the last option, the issue can be mitigated by implementing a schema definition for environments, for the other options, the relationship is only known to the consuming configuration.
In summary, to fulfill the vision laid out throughout this post, it is necessary to solve the orchestration problem when using a central configuration service to interconnect two IaC configurations. We will delve deeper into this topic in the second part of this series.