News HashiCorp to join IBM to accelerate multi-cloud automation. Read the blog →
Terraform
HCP Terraform

Manage variable sets in HCP Terraform

  • 16min
  • |
  • HCP Terraform
  • Terraform

HCP Terraform variable sets let you reuse variables in an efficient and centralized way, so you can set variable values once and use them in multiple workspaces. When you update the values, the changes automatically affect all associated workspaces, making it easier to update credentials and infrastructure settings. HCP Terraform also lets you override variables defined in variable sets on a per-workspace basis, which lets you modify the workspace's configuration without affecting other workspaces that use the variable set.

In this tutorial, you will manage multiple HCP Terraform variable sets. You will define variable sets for your AWS credentials and DynamoDB configuration. You will also review variable precedence between duplicated variables across variable sets, and between variables in variable sets and workspace-specific variables.

Prerequisites

This tutorial assumes that you are familiar with HCP Terraform and the standard Terraform workflow. If you are new to Terraform, complete the Get Started tutorials first. If you are new to HCP Terraform, complete the HCP Terraform Get Started tutorials first.

For this tutorial, you will need:

Clone example configuration

First, clone the example repository. This repository contains example configuration to create AWS DynamoDB tables in two environments.

$ git clone https://github.com/hashicorp/learn-terraform-variable-sets.git

Now, change into the repository directory.

$ cd learn-terraform-variable-sets

This repository contains two subdirectories: 

$ tree.├── README.md├── dev│   ├── main.tf│   ├── variables.tf│   └── versions.tf└── staging    ├── main.tf    ├── variables.tf    └── versions.tf

The dev and staging directories each contain Terraform configuration that defines a DynamoDB table and configures it using input variables. The configuration uses the random_pet resource to ensure a unique name for the table, and configures the DynamoDB table using the db_read_capacity and db_write_capacity input variables.

Initialize configuration

Navigate to the dev directory.

$ cd dev

Open versions.tf in your code editor, and replace <ORGANIZATION_NAME> in the cloud block with your own HCP Terraform organization name.

dev/versions.tf

terraform {   cloud {    organization = "<ORGANIZATION_NAME>"     workspaces {      name = "learn-terraform-variable-sets-dev"    }  }###

Notice that this configuration uses the learn-terraform-variable-sets-dev workspace.

Initialize the configuration, which will also create the workspace in HCP Terraform.

$ terraform init Initializing HCP Terraform... Initializing provider plugins...Reusing previous version of hashicorp/random from the dependency lock file- Reusing previous version of hashicorp/aws from the dependency lock file- Installing hashicorp/random v3.1.0...- Installed hashicorp/random v3.1.0 (signed by HashiCorp)- Installing hashicorp/aws v3.63.0...- Installed hashicorp/aws v3.63.0 (signed by HashiCorp) HCP Terraform has been successfully initialized! You may now begin working with HCP Terraform. Try running "terraform plan" tosee any changes that are required for your infrastructure. If you ever set or change modules or Terraform Settings, run "terraform init"again to reinitialize your working directory.

Now, navigate to the staging directory. 

$ cd ../staging

Open versions.tf in your code editor, and replace <ORGANIZATION_NAME> in the cloud block with your own HCP Terraform organization name.

staging/versions.tf

terraform {   cloud {    organization = "<ORGANIZATION_NAME>"     workspaces {      name = "learn-terraform-variable-sets-staging"    }  }###

This configuration uses a different workspace, learn-terraform-variable-sets-staging.

Initialize the configuration, which will also create the workspace in HCP Terraform.

$ terraform init Initializing HCP Terraform... Initializing provider plugins...Reusing previous version of hashicorp/random from the dependency lock file- Reusing previous version of hashicorp/aws from the dependency lock file- Installing hashicorp/random v3.1.0...- Installed hashicorp/random v3.1.0 (signed by HashiCorp)- Installing hashicorp/aws v3.63.0...- Installed hashicorp/aws v3.63.0 (signed by HashiCorp) HCP Terraform has been successfully initialized! You may now begin working with HCP Terraform. Try running "terraform plan" tosee any changes that are required for your infrastructure. If you ever set or change modules or Terraform Settings, run "terraform init"again to reinitialize your working directory.

You now have two HCP Terraform workspaces configured for the CLI-driven workflow with remote execution.

Create variable sets

HCP Terraform variable sets are groups of reusable variables created at the organization level. A variable set can have one of three scopes:

  • Global: It will apply to all current and future workspaces within an organization.
  • Project-specific: It will apply to all current and future workspaces within the selected projects.
  • Workspace-specific: It will apply only to the selected workspaces.

Using broader variable set scope enables self-service workflows. For instance, you can create a variable set and apply it to a team-specific project, then grant the team permission to create workspaces within the project. Future workspaces will automatically inherit the variable set without requiring additional work or approval. However, we recommend scoping variable sets that contain credentials as narrowly as possible, to avoid granting access to teams or workspaces that do not need them.

Create a credentials variable set

First, navigate to your organization's settings by clicking Settings in the left navigation. Then, select Variable Sets.

Note

If you already have a variable set for your AWS provider credentials, skip to the next section.

Click Create variable set.

Create HCP Terraform variable set

Name this first variable set AWS credentials.

Warning

When possible, apply credential variable sets to specific projects or workspaces. Avoid global access and follow the principle of least privilege.

Scroll down to the Variable set scope section and select Apply to specific projects and workspaces. Select the learn-terraform-variable-sets-dev and learn-terraform-variable-sets-staging workspaces.

Create HCP Terraform variable set

Click +Add Variable. Define an environment variable named AWS_ACCESS_KEY_ID and set it to your AWS Access Key ID. Mark it as sensitive and click Save variable.

Create HCP Terraform variable set

Then, click +Add Variable again. Define another environment variable named AWS_SECRET_ACCESS_KEY and set it to your AWS Secret access key. Mark it as sensitive and click Save variable.

Tip

If you have temporary AWS credentials, you must also add your AWS_SESSION_TOKEN as an environment variable.

Finally, click Create variable set.

Save variable set

Create configuration settings variable set

You can also use variable sets to define reusable input variables. In this scenario, you will provision DynamoDB tables for two environments, dev and staging. Use a variable set to define the read and write capacities for both.

Create another variable set named Default DynamoDB settings. Once again, apply it to both the learn-terraform-variable-sets-dev and learn-terraform-variable-sets-staging workspaces.

Define two Terraform variables in the variable set:

  1. A Terraform variable named db_write_capacity with a value of 1.
  2. A Terraform variable named db_read_capacity with a value of 1.

Save the variable set.

Rather than individually defining the database read and write capacity in both workspaces, you were able to just define them once as a variable set and apply them to the workspaces that need them.

Review workspace variables

You can review and manage which variable sets apply to the workspace from the workspace itself.

Navigate to your learn-terraform-variable-sets-dev workspace, then select the Variables tab.

Under Variable sets, the workspace lists both your AWS credentials and Default DynamoDB settings variable sets and the variables that they contain.

Multiple variable sets applied to HCP Terraform workspacet

Apply configuration

In your terminal, navigate to your learn-terraform-variable-sets/dev directory.

You already initialized your configuration earlier, so now apply it. Respond yes when prompted to confirm the operation.

$ terraform applyRunning apply in HCP Terraform. Output will stream here. Pressing Ctrl-Cwill cancel the remote apply if it's still pending. If the apply started itwill stop streaming the logs, but will not stop the apply running remotely. Preparing the remote apply... To view this run in a browser, visit:https://app.terraform.io/app/hashicorp-training/learn-terraform-variable-sets-dev/runs/run-fRjkg53BhhhwbEUN Waiting for the plan to start... Terraform v1.0.7on linux_amd64Configuring remote state backend...Initializing Terraform configuration... Terraform used the selected providers to generate the following executionplan. Resource actions are indicated with the following symbols:  + create Terraform will perform the following actions:##...Plan: 2 to add, 0 to change, 0 to destroy. Do you want to perform these actions in workspace "learn-terraform-variable-sets-dev"?  Terraform will perform the actions described above.  Only 'yes' will be accepted to approve.   Enter a value: yes random_pet.table_name: Creating...random_pet.table_name: Creation complete after 0s [id=still-pony]aws_dynamodb_table.table: Creating...aws_dynamodb_table.table: Creation complete after 4s [id=dev-still-pony] Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Go to the learn-terraform-variable-sets-dev workspace to find the resources it manages.

HCP Terraform workspace overview

Now, navigate to your staging directory.

$ cd ../staging

Apply your configuration. Respond yes when prompted to confirm the operation.

$ terraform applyRunning apply in HCP Terraform. Output will stream here. Pressing Ctrl-Cwill cancel the remote apply if it's still pending. If the apply started itwill stop streaming the logs, but will not stop the apply running remotely. Preparing the remote apply... To view this run in a browser, visit:https://app.terraform.io/app/hashicorp-training/learn-terraform-variable-sets-staging/runs/run-WMTfEG5hFmwaNArq Waiting for the plan to start... Terraform v1.0.7on linux_amd64Configuring remote state backend...Initializing Terraform configuration... Terraform used the selected providers to generate the following executionplan. Resource actions are indicated with the following symbols:  + create Terraform will perform the following actions:##...Plan: 2 to add, 0 to change, 0 to destroy. Do you want to perform these actions in workspace "learn-terraform-variable-sets-staging"?  Terraform will perform the actions described above.  Only 'yes' will be accepted to approve.   Enter a value: yes random_pet.table_name: Creating...random_pet.table_name: Creation complete after 0s [id=guiding-kite]aws_dynamodb_table.table: Creating...aws_dynamodb_table.table: Creation complete after 7s [id=staging-guiding-kite] Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Explore variable set precedence

If any of the variable sets associated with the workspace contain a variable of the same type (input or environment variables) with the same name, HCP Terraform will use lexical order to determine variable precedence.

In this scenario, you applied a default set of DynamoDB settings to both tables. While you generally want to use consistent settings across both resources, you may want to perform load testing on one of your tables.

Navigate back to the variable sets page in your organization settings, create a new variable set named Add Capacity - DynamoDB load testing, and apply it to the learn-terraform-variable-sets-staging workspace.

Create two Terraform variables for this variable set:

  1. Set db_write_capacity to 10
  2. Set db_read_capacity to 10

Save your new variable set.

Now, navigate back to your learn-terraform-variable-sets-staging workspace and navigate to the Variables page. It lists 3 variable sets applying to your workspace.

Scroll down to find the Default DynamoDB settings variable set, which shows its values as overwritten.

Load testing variable set overwrites default DynamoDB settings in HCP Terraform workspace

Both your default and load testing variable sets define variables named db_write_capacity and db_read_capacity. Since the load testing variable set name begins with the letter "A", that variable set took precedence over your default settings. If you want HCP Terraform to use the default DynamoDB variable set instead, you can:

  • change the names of the variable sets so that the default set has lexical precedence over the load testing set, or
  • remove the load testing variable set from the workspace.

In your terminal, run terraform apply in your staging directory to update your table's configuration. Respond yes when prompted to confirm the operation. 

$ terraform applyRunning apply in HCP Terraform. Output will stream here. Pressing Ctrl-Cwill cancel the remote apply if it's still pending. If the apply started itwill stop streaming the logs, but will not stop the apply running remotely. Preparing the remote apply... To view this run in a browser, visit:https://app.terraform.io/app/hashicorp-training/learn-terraform-variable-sets-staging/runs/run-1j7aMQm8v4HdMWMR Waiting for the plan to start... Terraform v1.0.7on linux_amd64Configuring remote state backend...Initializing Terraform configuration...random_pet.table_name: Refreshing state... [id=guiding-kite]aws_dynamodb_table.table: Refreshing state... [id=staging-guiding-kite] Terraform used the selected providers to generate the following executionplan. Resource actions are indicated with the following symbols:  ~ update in-place Terraform will perform the following actions:   # aws_dynamodb_table.table will be updated in-place  ~ resource "aws_dynamodb_table" "table" {        id             = "staging-guiding-kite"        name           = "staging-guiding-kite"      ~ read_capacity  = 1 -> 10        tags           = {}      ~ write_capacity = 1 -> 10        # (5 unchanged attributes hidden)         # (3 unchanged blocks hidden)    } Plan: 0 to add, 1 to change, 0 to destroy. Do you want to perform these actions in workspace "learn-terraform-variable-sets-staging"?  Terraform will perform the actions described above.  Only 'yes' will be accepted to approve.   Enter a value: yes aws_dynamodb_table.table: Modifying... [id=staging-guiding-kite]aws_dynamodb_table.table: Modifications complete after 2s [id=staging-guiding-kite] Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

As expected, Terraform used the more recently applied load testing variable set to your configuration and increased the read and write capacities of your table.

Overwrite a variable in a variable set

You can also overwrite a variable defined in a variable set by creating a workspace-specific variable with the same key. HCP Terraform will always use workspace-specific variables over any variables defined in variable sets applied to the workspace.

In a load testing scenario, you may want to scale up your write capacity to test how your application's performance responds. In your learn-terraform-variable-sets-staging workspace, create a workspace-specific input variable named db_write_capacity and set the value to 15.

HCP Terraform now shows the db_write_capacity variable in the load testing variable set as overwritten.

Create workspace-specific HCP Terraform variable

In your terminal, run a terraform apply to further scale your DynamoDB table's write capacity. Respond yes to the prompt to confirm the operation. 

$ terraform applyRunning apply in HCP Terraform. Output will stream here. Pressing Ctrl-Cwill cancel the remote apply if it's still pending. If the apply started itwill stop streaming the logs, but will not stop the apply running remotely. Preparing the remote apply... To view this run in a browser, visit:https://app.terraform.io/app/hashicorp-training/learn-terraform-variable-sets-staging/runs/run-H5aAbCPCZ3xKiw9j Waiting for the plan to start... Terraform v1.0.7on linux_amd64Configuring remote state backend...Initializing Terraform configuration...random_pet.table_name: Refreshing state... [id=guiding-kite]aws_dynamodb_table.table: Refreshing state... [id=staging-guiding-kite] Terraform used the selected providers to generate the following executionplan. Resource actions are indicated with the following symbols:  ~ update in-place Terraform will perform the following actions:   # aws_dynamodb_table.table will be updated in-place  ~ resource "aws_dynamodb_table" "table" {        id             = "staging-guiding-kite"        name           = "staging-guiding-kite"        tags           = {}      ~ write_capacity = 10 -> 15        # (6 unchanged attributes hidden)         # (3 unchanged blocks hidden)    } Plan: 0 to add, 1 to change, 0 to destroy. Do you want to perform these actions in workspace "learn-terraform-variable-sets-staging"?  Terraform will perform the actions described above.  Only 'yes' will be accepted to approve.   Enter a value: yes aws_dynamodb_table.table: Modifying... [id=staging-guiding-kite]aws_dynamodb_table.table: Modifications complete after 3s [id=staging-guiding-kite] Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

Since you set a workspace-specific variable for the write capacity, HCP Terraform prioritized that value and scaled your table's write capacity to 15, overwriting the variable set values.

For more information on how HCP Terraform inherits variable sets and overrides values, see Workspace variable precedence.

Clean up resources

In your staging directory, destroy the table you created. Respond yes when prompted to confirm the operation.

$ terraform destroyRunning apply in HCP Terraform. Output will stream here. Pressing Ctrl-Cwill cancel the remote apply if it's still pending. If the apply started itwill stop streaming the logs, but will not stop the apply running remotely. Preparing the remote apply... To view this run in a browser, visit:https://app.terraform.io/app/hashicorp-training/learn-terraform-variable-sets-staging/runs/run-W8NaRXye3gmGs1cy Waiting for the plan to start... Terraform v1.0.7on linux_amd64Configuring remote state backend...Initializing Terraform configuration...random_pet.table_name: Refreshing state... [id=guiding-kite]aws_dynamodb_table.table: Refreshing state... [id=staging-guiding-kite] Terraform used the selected providers to generate the following executionplan. Resource actions are indicated with the following symbols:  - destroy Terraform will perform the following actions:##...Plan: 0 to add, 0 to change, 2 to destroy. Do you really want to destroy all resources in workspace "learn-terraform-variable-sets-staging"?  Terraform will destroy all your managed infrastructure, as shown above.  There is no undo. Only 'yes' will be accepted to confirm.   Enter a value: yes aws_dynamodb_table.table: Destroying... [id=staging-guiding-kite]aws_dynamodb_table.table: Destruction complete after 3srandom_pet.table_name: Destroying... [id=guiding-kite]random_pet.table_name: Destruction complete after 0s Apply complete! Resources: 0 added, 0 changed, 2 destroyed.

Then, change to your dev directory.

$ cd ../dev

Destroy the infrastructure managed in this directory as well.

$ terraform destroyRunning apply in HCP Terraform. Output will stream here. Pressing Ctrl-Cwill cancel the remote apply if it's still pending. If the apply started itwill stop streaming the logs, but will not stop the apply running remotely. Preparing the remote apply... To view this run in a browser, visit:https://app.terraform.io/app/hashicorp-training/learn-terraform-variable-sets-dev/runs/run-v3Nszw3Eybo432AH Waiting for the plan to start... Terraform v1.0.7on linux_amd64Configuring remote state backend...Initializing Terraform configuration...random_pet.table_name: Refreshing state... [id=still-pony]aws_dynamodb_table.table: Refreshing state... [id=dev-still-pony] Terraform used the selected providers to generate the following executionplan. Resource actions are indicated with the following symbols:  - destroy Terraform will perform the following actions:##...Plan: 0 to add, 0 to change, 2 to destroy. Do you really want to destroy all resources in workspace "learn-terraform-variable-sets-dev"?  Terraform will destroy all your managed infrastructure, as shown above.  There is no undo. Only 'yes' will be accepted to confirm.   Enter a value: yes aws_dynamodb_table.table: Destroying... [id=dev-still-pony]aws_dynamodb_table.table: Destruction complete after 3srandom_pet.table_name: Destroying... [id=still-pony]random_pet.table_name: Destruction complete after 0s Apply complete! Resources: 0 added, 0 changed, 2 destroyed.

Clean up HCP Terraform resources

Then, navigate to your learn-terraform-variable-sets-dev workspace in HCP Terraform and delete the workspace.

Now, navigate to your learn-terraform-variable-sets-staging workspace and delete the workspace.

Finally, navigate to your variable sets list under the organization settings. Delete your Default DynamoDB settings variable set by clicking on it, scrolling to the bottom, and clicking Delete variable set.

Delete HCP Terraform variable set

Delete your load testing variable set, and optionally your AWS credentials variable set as well.

Next steps

In this tutorial, you learned how to create and use variable sets to manage your HCP Terraform workspace's input and environment variables. You also learned about variable precedence between variable sets and how to overwrite variables within a workspace.

Check out the following resources to learn more about HCP Terraform configuration options and features:

Previous

Version state

 Next

Snyk run task