Getting to Know Terraform Environment Variables

Terraform uses environment variables to customize and enhance different aspects of its behavior. In this article, you will learn exactly what are environment variables and how to use them in different scenarios, including debugging all the way to Azure DevOps pipelines!

What are Environment Variables?

Environment variables store information about the operating system and the current shell environment. Information could include the operating system name, processor information, temporary folder locations, and current logged-on user details. Environment variables are scoped to the system, user, or process.

One important environment variable is the path variable. The path variable tells the operating system where to look for executables when you issue a command. This configuration is how you are able to use commands like ping, tracert, and ipconfig at the command line without specifying the full path to the program.

In PowerShell, environment variables are prepended by $env: followed by the variable name. For example, you can list the operating system name, current user name, user profile path, or processor identifier, like this:

# Operating System Type
$env:OS
# Current user name
$env:USERNAME
# Current user home profile path
$env:USERPROFILE
# Processor identifier
$env:PROCESSOR_IDENTIFIER

Terraform can use information stored in these environment variables when reading and deploying configurations. Let’s take a look at a few scenarios.

Debugging Terraform with Environment Variables

You enable detailed logging in Terraform by setting the TF_LOG environment variable. Changing the log level causes detailed logging to appear on stderr. There are six log levels to choose from:

• TRACE
• DEBUG
• INFO
• WARN
• ERROR
• JSON

If you set the TF_LOG level to JSON, Terraform outputs the log at the TRACE level or higher and uses the JSON format. However, Terraform documentation warns that the JSON formatting can change at any time and should be manually parsed using other tools. In the future, Hashicorp will release support tooling meant to interact with these JSON-formatted logs.

If you want to force Terraform to append to a specific log file, use the TF_LOG_PATH environment variable and specify a directory. Note that you still have to set the TF_LOG variable in order to enable any logging.

Now that you understand the different logging levels, let’s take a look at an example. Here is an example output after running terraform apply on a configuration. Note how much information Terraform outputs to the console.

To set the debug level in PowerShell, enter $env:TF_LOG followed by one of the log levels listed above. This example sets the log level to INFO.

$env:TF_LOG = 'INFO'

Re-running terraform apply shows a lot more information written to the console. Examine the difference in the screenshot below.

Setting the log level can help troubleshoot issues with Terraform, providers, or plug-ins during your deployment. If you run into an issue or bug, support may ask for additional logs in order to troubleshoot the issue.

Setting Input Variables

Input variables enable flexible Terraform configurations. You define inputs at runtime to customize your configuration. One method is to use environment variables. Terraform searches the shell environment for environment variables starting with TF_VAR_ followed by the name of the variable. Configuring values using environment variables is useful when you are running a sequence of Terraform commands in succession with the same variables.

For example, here’s a Terraform configuration file that uses the random_pet resource to generate random pet names to use as unique identifiers for other resources. The configuration contains a variable named petcount that determines how many pet names to generate. The list of names is then outputted to the screen.

variable "petcount" {
  type = number
}
resource "random_pet" "mypet" {
  count = var.petcount
  separator = " "
}
output "petname" {
  value = random_pet.mypet.*.id
}

To set the petcount variable using an environment variable, prepend the variable name with $env:TF_VAR_. In this example, you are setting the value to 5.

$env:TF_VAR_petcount = 5

Now you can run any Terraform command without specifying the variable value inline or using a separate Terraform configuration file. For example, running terraform plan does not prompt for the petcount value and shows the proposed changes. The partial screenshot below shows this in action.

Configuring Providers with Environment Variables

Terraform also uses environment variables for configuring providers. Providers allow Terraform to interact with cloud providers, SaaS providers, and other APIs. Providers also dictate what resources or data sources Terraform manages in the configuration.

One example is using an Azure storage account to store the Terraform backend state file. In the Terraform configuration, you define where to store the remote backend state file using the following properties:

• resource_group_name: The name of the resource group with the storage account.
• storage_account_name: The name of the storage account.
• container_name: The container name in the storage account.
• key: the name of the state file, for example, terraform.tfstate.

Here is an updated Terraform configuration file from earlier using the azurerm provider to configure Azure as the remote backend.

terraform {
  backend "azurerm" {
    resource_group_name = "terraformstate-rg"
    storage_account_name = "jbtterraformstateprod"
    container_name = "petnames"
    key = "petnames.terraform.tfstate"
  }
}
resource "random_pet" "mypet" {
  separator = " "
}
output "petname" {
  value = random_pet.mypet.*.id
}

In order for Terraform to read and write the state file in the storage account, Terraform needs to authenticate to the storage account. One way to perform this is to use one of the storage account access keys. However, these access keys provide full access to the storage account, and you do not want to store it in plain text in your Terraform configuration file.

One option is to create an environment variable to store the access key. Like the previous examples, the variable starts with $env: and ends with the provider-specified variable name. For the azurerm provider, the name is ARM_ACCESS_KEY.

Below is an example of setting the access key environment variable using PowerShell, then initializing the Terraform configuration. Note that Terraform outputs it was successfully able to initialize the remote backed into the Azure storage account.

# Configure the ARM_ACCESS_KEY environment variable
$env:ARM_ACCESS_KEY = 'T1OZE3Dt2COABio6SfgsID48gvha9OjT+8ogyV+3EKbisGR6yIsOPhJRz8ZejyvgE9+hz9IW0K3QmAIgRXAhDg=='
# Initialize the Terraform configuration to use the remote backend
terraform init

Incorporating Environment Variables in Azure DevOps Pipelines

One final demonstration is using environment variables with Terraform in Azure DevOps pipelines. This next example expands on the previous section where you defined environment variables to configure the AzureRM provider.

While Azure DevOps pipelines include built-in steps for using Terraform, Julie Ng (Microsoft Engineer) suggests in her article Terraform on Azure Pipelines Best Practices to use the Bash task instead. The remaining example is based on this suggestion.

Take the previous section where you are using Azure as the backend storage for the remote Terraform state. Instead of defining all the Azure backend settings in the Terraform configuration file (like storage account name, container name, etc.), you can use a partial configuration. The example below shows a partial Terraform configuration file where you only define that the back end is being stored in Azure.

terraform {
  backend "azurerm" {      
  }
}

The backend configuration information is stored as pipeline variables instead (note: these are not environment variables). The variables include:

• var-tf-state-blob-account: the name of the storage account (jbtterraformstateprod)
• var-tf-state-blob-container: the name of the container (terraformstate)
• var-tf-state-blob-file: the name of the Terraform state file (prod.terraform.tfstate)
• var-tf-state-blob-key: the access key for the storage account. Note this value is a secret value and hidden from view.

Inside the pipeline, use the Bash task to run Terraform commands just like you would in your local shell. However, use the env setting in the task to pull in the pipeline variable values for your backend configuration. Within the terraform init command, use the -backend-config parameters to specify the storage account name, container name, key file name, and access key.

Summary

In this article, you learned how Terraform incorporates environment variables, from debugging to variables to provider configuration. For more ways on how Terraform uses environment variables, check out Hashicorp’s Environment Variables documentation.

Enjoyed this article? Check out more of my Terraform content here!