Terraform modules are opinions

2022-Nov-12 • by David Norton

Quick comment on Terraform modules. When searching the internet for how to use a particular cloud feature with Terraform (for example, create an EKS cluster in AWS) using Terraform, you'll often run across a Terraform module that will do this. And that's fine - but you need to decide if you want to use a module, or the resource itself. Let's walk through some examples, and then we'll go through some options .

Resource

The aws_eks_cluster resource is the official offering for creating an EKS cluster, provided in the AWS provider for Terraform. It doesn't provide everything you need (for example, you'll need to give it an IAM role and place it in some VPC subnets), but it's what you get when you use the AWS console, aws CLI, or API to create a cluster.

resource "aws_eks_cluster" "app" {
  name     = "app-${var.environment}"
  role_arn = aws_iam_role.app.arn

  vpc_config {
    subnet_ids = [aws_subnet.app1.id, aws_subnet.app2.id]
  }
}

Module

On the other hand, there are a number of published modules for EKS clusters. These are further abstractions, bundling together a variety of resources. This is fine -- abstractions are an important part of software engineering. But consider how the AWS-provided terraform-aws-eks module is used:

module "eks" {
  source = "../.."

  cluster_name                    = local.name
  cluster_endpoint_private_access = true
  cluster_endpoint_public_access  = true

  cluster_addons = {
    coredns = {
      resolve_conflicts = "OVERWRITE"
    }
    kube-proxy = {}
    vpc-cni = {
      resolve_conflicts = "OVERWRITE"
    }
  }

  # Encryption key
  create_kms_key = true
  cluster_encryption_config = [{
    resources = ["secrets"]
  }]
  kms_key_deletion_window_in_days = 7
  enable_kms_key_rotation         = true

  ...
}

That is only about 30 of the 200+ lines in their example!

Why are there so many options? Well, there are a lot of options on the underlying resources, and rather than take an opinion, they've decided to surface all of the underlying options as variables on the module. In my opinion, leaking the entire abstraction eliminates the value of the abstraction in the first place!

This is not an uncommon problem. There are many instances of these modules scattered throughout the Terraform registry. Some of the modules are valuable as they express an opinionated way of doing things. Some seem... pointless.

Is this module an opinion, an example, or DRYing out my own code?

Whenever you are thinking about creating a module, you need to ask yourself if this module is intended to be an opinion (an abstraction), forcing the developer into a particular way of using a set of cloud resources? Or is it an example?

This can be good when you have organizational non-functional requirements regarding security, monitoring, etc. that you want to be implemented by default. However, it also means that any features from the underlying cloud resource must be surfaced as variables in your own module.

Another way to think about a Terraform module is that it's simply an example. In this case, you aren't really intending for others to pull your module into their own code. You are just showing how the different resources can be used together -- for example, load balancer, autoscaling group, and DNS resources. In this case, you don't have to surface all the available features of the underlying resources, you're simply showing others how to use resources together.

Finally, a Terraform module can be used internally within your own project to reduce duplication (DRY things out). This is a variant of "opinion", but you're keeping your opinion to yourself. It's analogous to refactoring your code while keeping external APIs consistent.

Maintenance

If you're publishing a Terraform module, you need to consider that people will use your module and need access to the underlying features of the cloud. In that case, you are signing up for a whole lot of updates to expose those variables and options. Your simple, clean module will not be so simple any more.

Also, the provider you are using will be updated, eventually with a major version and breaking API changes, and you need to think about keeping up with those updates.

If you're not prepared to update the module and maintain it as any other open source project would be maintained, consider publishing it as an example or a tutorial.

Conclusion

Terraform modules are opinions. If you don't have an opinion on how the resources should be used, don't publish the module.

Shameless plug: If you need help with public cloud, declarative infrastructure, or Terraform specifically -- Platformers may be able to help. Consider reaching out today!