Why Terraform Code Testing is important?

We prefer to modularise Terraform code to avoid repetition and to combine multiple resources that are used together. These modules then get consumed in other terraform configuration blocks. Testing Terraform code is very important especially when we have Terraform modules, that are being used in many places to deploy Infrastructure resources. A small mistake or typo causes failures in many consumer code blocks. We can use tools like Terratest, Inspec to test Terraform code.

What is Terratest?

Terratest is a Go library that we can use to write automated tests for our Terraform code. It provides a variety of helper functions and patterns for Terraform code.

In this quick start demo, we’ll use Terratest to write automated tests for terraform code that deploys resources on Google Cloud Platform.

Writing Terratest Test Case for GCP Storage Bucket Resource Code

Prerequisites:

• Go (requires version >=1.17)

We have following Terraform code which creates a GCS bucket. It has a variable block, resource block and output block. I have copied this configuration into a file called main.tf in terratest-gcp folder.

variable "bucket_name" {
  type = string
}

resource "google_storage_bucket" "bucket" {
  name     = var.bucket_name
  location = "EU"
  project  = "devops-counsel-demo"
}


output "bucket_name" {
  value = google_storage_bucket.bucket.name
}

Now we are going to add tests using Terratest for above tf code.

Create a folder called ‘test’ inside terratest-gcp directory and copy below Go code into a file called terraform_gcp_demo_test.go.

package test

import (
        "fmt"
        "testing"
        "strings"
        "github.com/gruntwork-io/terratest/modules/random"
        "github.com/gruntwork-io/terratest/modules/gcp"
        "github.com/gruntwork-io/terratest/modules/terraform"
)

func TestGCSExample(t *testing.T) {
        expectedBucketName := fmt.Sprintf("test-tf-gcs-bucket-%s", strings.ToLower(random.UniqueId()))
        terraformOptions := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
                TerraformDir: "../",
                Vars: map[string]interface{}{
                        "bucket_name": expectedBucketName,
                },
        })
        defer terraform.Destroy(t, terraformOptions)
        terraform.InitAndApply(t, terraformOptions)
        gcp.AssertStorageBucketExists(t, expectedBucketName)
}

The above code will do following things

• It tests code inside “../” parent directory which is terratest-gcp(code inside main.tf)
• It generates a random suffix for GCS bucket.
• Then it passes that bucket name with random suffix as a variable to terraform.Options since our main.tf accepts bucket name as a parameter.
• Then it initializes and apply Terraform code.
• Then it checks whether bucket is available in GCP or not using Terratest gcp module.
• Then it removes GCS bucket as a last step and print out test results.

File structure should look like below inside terratest-gcp directory.

cloudshell:~/terratest-gcp$ tree 
.
├── main.tf
└── test
    └── terraform_gcp_demo_test.go

1 directory, 2 files

Now we are going to resolve go modules by using following commands.

cloudshell:~/terratest-gcp$ cd test
cloudshell:~/terratest-gcp/test$ go mod init "devops-counsel-demo"
go: creating new go.mod: module devops-counsel-demo
go: to add module requirements and sums:
        go mod tidy
cloudshell:~/terratest-gcp/test$ go mod tidy
[...]

Now we are going to run test.

(output has been redacted to highlight important messages)

cloudshell:~/terratest-gcp/test$ go test -v
=== RUN   TestGCSExample
[...]
TestGCSExample 2022-07-22T17:54:27Z logger.go:66: Terraform has been successfully initialized!
[...]
TestGCSExample 2022-07-22T17:54:27Z logger.go:66: Running command terraform with args [apply -input=false -auto-approve -var bucket_name=test-tf-gcs-bucket-copsmb -lock=false]
[...]
TestGCSExample 2022-07-22T17:54:31Z logger.go:66: Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
TestGCSExample 2022-07-22T17:54:31Z logger.go:66: Outputs:
TestGCSExample 2022-07-22T17:54:31Z logger.go:66: bucket_name = "test-tf-gcs-bucket-copsmb"
TestGCSExample 2022-07-22T17:54:31Z storage.go:190: Finding bucket test-tf-gcs-bucket-copsmb
TestGCSExample 2022-07-22T17:54:31Z retry.go:91: terraform [destroy -auto-approve -input=false -var bucket_name=test-tf-gcs-bucket-copsmb -lock=false]
TestGCSExample 2022-07-22T17:54:31Z logger.go:66: Running command terraform with args [destroy -auto-approve -input=false -var bucket_name=test-tf-gcs-bucket-copsmb -lock=false]
[...] 
TestGCSExample 2022-07-22T17:54:33Z logger.go:66: Plan: 0 to add, 0 to change, 1 to destroy.
TestGCSExample 2022-07-22T17:54:33Z logger.go:66: Changes to Outputs:
TestGCSExample 2022-07-22T17:54:33Z logger.go:66:   - bucket_name = "test-tf-gcs-bucket-copsmb" -> null
[...]
TestGCSExample 2022-07-22T17:54:35Z logger.go:66: Destroy complete! Resources: 1 destroyed
--- PASS: TestGCSExample (8.35s)
PASS
ok      devops-counsel-demo     8.364s

In the above log you can all Terraform stages and test results.

At present, Terratest has very few helper functions to run resource assertions against GCP APIs. As a workaround, we can compare Terraform output values with expected values. In the next example we will use that method.

Writing Terratest Test Case for GCP BigQuery Dataset Resource Code

Following Terraform code creates a GCP BigQuery dataset. It has a variable block, resource block and output block. I have copied this configuration into a file called main.tf in terratest-bigquery folder.

variable "dataset_name" {
  type = string
}

resource "google_bigquery_dataset" "dataset" {
  dataset_id = var.dataset_name
  location   = "US"
  project    = "devops-counsel-demo"
}

output "dataset_name" {
  value = google_bigquery_dataset.dataset.dataset_id
}

Now we are going to add tests using Terratest for above tf code.

Create a folder called ‘test’ inside terratest-bigquery directory and copy below Go code into a file called terraform_bigquery_demo_test.go.

package test

import (
        "fmt"
        "testing"
        "strings"
        "github.com/gruntwork-io/terratest/modules/random"
        "github.com/gruntwork-io/terratest/modules/terraform"
        "github.com/stretchr/testify/assert"
)

func TestBigQueryExample(t *testing.T) {
        expectedDatasetName := fmt.Sprintf("test_tf_dataset_%s", strings.ToLower(random.UniqueId()))
        terraformOptions := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
                TerraformDir: "../",
                Vars: map[string]interface{}{
                        "dataset_name": expectedDatasetName,
                },
        })
        defer terraform.Destroy(t, terraformOptions)
        terraform.InitAndApply(t, terraformOptions)
        outputDatasetName := terraform.Output(t, terraformOptions, "dataset_name")
        assert.Equal(t, expectedDatasetName, outputDatasetName)
}

When I’m writing this article, Terratest does not have helper function for BigQuery. Above test case compares Terraform output “dataset_name” value with the dataset name that we supplied with random suffix as a variable value.

File structure should look like below inside terratest-bigquery directory.

cloudshell:~/terratest-bigquery$ tree
.
├── main.tf
└── test
    └── terraform_bq_demo_test.go

1 directory, 2 files

Now we are going to resolve go modules by using following commands.

cd test
go mod init "bigquery-demo"
go mod tidy

Now we are going to run test on Terraform code that we added to main.tf file.(output has been redacted to highlight important messages)

cloudshell:~/terratest-bigquery/test$ go test -v
=== RUN   TestBigQueryExample
[...]
TestBigQueryExample 2022-07-22T18:56:59Z logger.go:66: Running command terraform with args [apply -input=false -auto-approve -var dataset_name=test_tf_dataset_febqdi -lock=false]
[...]
TestBigQueryExample 2022-07-22T18:57:05Z logger.go:66: Plan: 1 to add, 0 to change, 0 to destroy.
[...]
TestBigQueryExample 2022-07-22T18:57:06Z logger.go:66: Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
TestBigQueryExample 2022-07-22T18:57:06Z logger.go:66: Outputs:
TestBigQueryExample 2022-07-22T18:57:06Z logger.go:66: dataset_name = "test_tf_dataset_febqdi"
TestBigQueryExample 2022-07-22T18:57:06Z logger.go:66: "test_tf_dataset_febqdi"
[...]
TestBigQueryExample 2022-07-22T18:57:06Z logger.go:66: Running command terraform with args [destroy -auto-approve -input=false -var dataset_name=test_tf_dataset_febqdi -lock=false]
TestBigQueryExample 2022-07-22T18:57:08Z logger.go:66: Plan: 0 to add, 0 to change, 1 to destroy.
[...]
TestBigQueryExample 2022-07-22T18:57:09Z logger.go:66: google_bigquery_dataset.dataset: Destroying... [id=projects/devops-counsel-demo/datasets/test_tf_dataset_febqdi]
[...]
TestBigQueryExample 2022-07-22T18:57:09Z logger.go:66: Destroy complete! Resources: 1 destroyed.
TestBigQueryExample 2022-07-22T18:57:09Z logger.go:66: 
--- PASS: TestBigQueryExample (11.68s)
PASS
ok      bigquery-demo   11.692s

In the above log you can see all Terraform stages and test results.

Conclusion

In this quick start demo we have tested terraform code written for deploying GCP provider’s resources. You can add these Terratest tests to CI pipelines to run tests against Terraform code before releasing it to production use. You can find more code examples here for other infrastructure platforms.

More on Terraform.

Terraform: for_each and count meta-arguments