Rotate Service Account Key using Terraform and Secret Manager

In this quick demo, we will create a service account and key file and store the key into secret manager and rotate the key. We are going to use Terraform code to do all these tasks.

What is Service Account?

GCP IAM offers two types of accounts for authentication and authorization, they are user accounts, service accounts.
User account is specific to an individuals to sign with user own credentials.
A service account is a non-human account used by applications, automated tasks, VMs to get identified by other applications to grant access. Generally service accounts have tokens or key files to authenticate against API endpoints. GCP IAM Service account have a key files with public/private RSA key pairs to authenticate against Google APIs. Service accounts can also be impersonated by other service accounts or normal users. Every GCP service account is identified by a email address.

Service account key is similar to user account’s password. If we dont manage these keys carefully they become security risk. There are best practises to reduce security risk. Some of them are as follows.

  1. Rotate keys regularly
  2. Do not share keys between users
  3. Do not store keys in source code repos
  4. Do not store keys on file systems.
  5. Do not embed keys in program binaries.
  6. Use secret manager to store keys and use IAM to control access to the key secret.

How to Rotate Service Account Key with Terraform and Secret Manager?

Copy following Terraform code to a .tf file.

resource "google_service_account" "demo_sa" {
  account_id = "demo-service-account"
  project    = "devops-counsel-demo"

resource "time_rotating" "sa_key_rotation" {
  rotation_days = 5

resource "google_service_account_key" "demo_sa_key" {
  service_account_id =
  public_key_type    = "TYPE_X509_PEM_FILE"
  keepers = {
    rotation_time = time_rotating.sa_key_rotation.rotation_rfc3339

resource "google_secret_manager_secret" "demo_secret" {
  project   = "devops-counsel-demo"
  secret_id = "demo_service_account_key_secret"
  replication {
    automatic = true

resource "google_secret_manager_secret_version" "key_secret_version" {
  secret      =
  secret_data = base64decode(google_service_account_key.demo_sa_key.private_key)

The above creates a service account called “demo-service-account” and then create a key file and copy key file content to secret manager. “time_rotating” resource block helps to rotate the key whenever Terraform code is applied after “keepers.rotation_time condition” condition is met(after 5 days from key creation date)

Run “terraform init” -> “terraform plan” -> “terraform apply” to apply above Terraform code. See the below apply output.

cloudshell:~/SA$ terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # google_secret_manager_secret.demo_secret will be created
  + resource "google_secret_manager_secret" "demo_secret" {
      + create_time = (known after apply)
      + expire_time = (known after apply)
      + id          = (known after apply)
      + name        = (known after apply)
      + project     = "devops-counsel-demo"
      + secret_id   = "demo_service_account_key_secret"

      + replication {
          + automatic = true

  # google_secret_manager_secret_version.key_secret_version will be created
  + resource "google_secret_manager_secret_version" "key_secret_version" {
      + create_time  = (known after apply)
      + destroy_time = (known after apply)
      + enabled      = true
      + id           = (known after apply)
      + name         = (known after apply)
      + secret       = (known after apply)
      + secret_data  = (sensitive value)

  # google_service_account.demo_sa will be created
  + resource "google_service_account" "demo_sa" {
      + account_id = "demo-service-account"
      + disabled   = false
      + email      = (known after apply)
      + id         = (known after apply)
      + name       = (known after apply)
      + project    = "devops-counsel-demo"
      + unique_id  = (known after apply)

   # google_service_account_key.demo_sa_key will be created
  + resource "google_service_account_key" "demo_sa_key" {
      + id                 = (known after apply)
      + keepers            = (known after apply)
      + key_algorithm      = "KEY_ALG_RSA_2048"
      + name               = (known after apply)
      + private_key        = (sensitive value)
      + private_key_type   = "TYPE_GOOGLE_CREDENTIALS_FILE"
      + public_key         = (known after apply)
      + public_key_type    = "TYPE_X509_PEM_FILE"
      + service_account_id = (known after apply)
      + valid_after        = (known after apply)
      + valid_before       = (known after apply)

  # time_rotating.sa_key_rotation will be created
  + resource "time_rotating" "sa_key_rotation" {
      + day              = (known after apply)
      + hour             = (known after apply)
      + id               = (known after apply)
      + minute           = (known after apply)
      + month            = (known after apply)
      + rfc3339          = (known after apply)
      + rotation_days    = 1
      + rotation_rfc3339 = (known after apply)
      + second           = (known after apply)
      + unix             = (known after apply)
      + year             = (known after apply)

Plan: 5 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

time_rotating.sa_key_rotation: Creating...
time_rotating.sa_key_rotation: Creation complete after 0s [id=2022-08-10T20:55:25Z]
google_service_account.demo_sa: Creating...
google_secret_manager_secret.demo_secret: Creating...
google_secret_manager_secret.demo_secret: Creation complete after 0s [id=projects/devops-counsel-demo/secrets/demo_service_account_key_secret]
google_service_account.demo_sa: Creation complete after 1s [id=projects/devops-counsel-demo/serviceAccounts/]
google_service_account_key.demo_sa_key: Creating...
google_service_account_key.demo_sa_key: Creation complete after 1s [id=projects/devops-counsel-demo/serviceAccounts/]
google_secret_manager_secret_version.key_secret_version: Creating...
google_secret_manager_secret_version.key_secret_version: Creation complete after 1s [id=projects/400560814631/secrets/demo_service_account_key_secret/versions/1]

Apply complete! Resources: 5 added, 0 changed, 0 destroyed.

Now the service account has been created and key file has been added to secret manger. you can find the key file secret in secret manger console.

The best practice is to run this Terraform code with a cron schedule inside a pipeline so that it rotates key file when ever rotation time condition(rotation_time = time_rotating.sa_key_rotation.rotation_rfc3339) is met.

Use below Terraform resource block to download the key file locally.

resource "local_file" "sa_json_file" {
  content  = base64decode(google_service_account_key.demo_sa_key.private_key)
  filename = "${path.module}/demo_key.json"



In this quick demo, we used Terraform to create service key file and coped it to secret manager and used time_rotating resoure block for adding key rotation time for key file. For more options for time_rotating block read Terraform official documentation.

For more on Terraform:

Terraform: for_each and count meta-arguments


  1. How do you prevent the existing key from being deleted immediately so that it doesn’t create a race condition between the creation of the new key and when the new key can be effectively distributed/updated?

    1. Hi, I don’t think there will be any race condition in this scenario. Have you faced this issue? curious to know. I intentionally delayed key creation using below configuration, to make sure that key deleted first from secret manager, sleep 30 secs to creating new key and updating secret manger version with new key. it went through well.

      resource “time_sleep” “wait_30_seconds” {
      create_duration = “30s”


      resource “google_service_account_key” “demo_sa_key” {
      service_account_id =
      public_key_type = “TYPE_X509_PEM_FILE”
      keepers = {
      rotation_time = time_rotating.sa_key_rotation.rotation_rfc3339
      depends_on = [time_sleep.wait_30_seconds]

  2. The race condition isn’t in your specific scenario. It’s if the svc account is being used outside of the project. For ingress authentication, for example. With this terraform the old key is expired or deleted immediately, so any in flight tokens are automatically expired. Right?

Leave a ReplyCancel reply

Exit mobile version