Export secrets as environment variables with Vault Agent

14min
|
Beta
Vault

Consul Template and Envconsul tools have been widely used by the Vault practitioners to help integrate Vault in their existing solutions. Vault 1.3.0 introduced the Vault Agent Template feature which provides the workflow that Consul Template provides.

Vault 1.14 introduced the process supervisor mode to retrieve secrets from Vault as environment variables using Consul Template markup.

Prerequisites

To complete this tutorial, you will need:

Vault binary version 1.14.0 or later

Lab setup

Open a terminal and start a Vault dev server with root as the root token.
```
$ vault server -dev -dev-root-token-id root
```
The Vault dev server defaults to running at 127.0.0.1:8200. The server is also initialized and unsealed.
Insecure operation: Do not run a Vault dev server in production. This approach is only used here to simplify the unsealing process for this demonstration.
Export an environment variable for the vault CLI to address the Vault server.
```
$ export VAULT_ADDR=http://127.0.0.1:8200
```

Log into Vault.

$ vault login root Success! You are now authenticated. The token information displayed belowis already stored in the token helper. You do NOT need to run "vault login"again. Future Vault requests will automatically use this token. Key                  Value---                  -----token                roottoken_accessor       9v9YS37o8lvXELvEQY6NqTtVtoken_duration       ∞token_renewable      falsetoken_policies       ["root"]identity_policies    []policies             ["root"]

Tip

Vault server is ready and the token value (root) is stored in the $HOME/.vault-token file.

Note

If you do not have access to an HCP Vault Dedicated cluster, visit the Create a Vault Cluster on HCP tutorial.

Launch the HCP Portal and login.
Click Vault in the left navigation pane.
In the Vault clusters pane, click vault-cluster.
Under Cluster URLs, click Public Cluster URL.
In a terminal, set the VAULT_ADDR environment variable to the copied address.
```
$ export VAULT_ADDR=<Public_Cluster_URL>
```
Return to the Overview page and click Generate token.
Within a few moments, a new token will be generated.
Copy the Admin Token.
Return to the terminal and set the VAULT_TOKEN environment variable.
```
$ export VAULT_TOKEN=<token>
```
Set the VAULT_NAMESPACE environment variable to admin.
```
$ export VAULT_NAMESPACE=admin
```
The admin namespace is the top-level namespace automatically created by HCP Vault. All CLI operations default to use the namespace defined in this environment variable.

$ vault login $VAULT_TOKEN WARNING! The VAULT_TOKEN environment variable is set! The value of thisvariable will take precedence; if this is unwanted please unset VAULT_TOKEN orupdate its value accordingly. Success! You are now authenticated. The token information displayed belowis already stored in the token helper. You do NOT need to run "vault login"again. Future Vault requests will automatically use this token. Key                  Value---                  -----token                hvs.CAESIGFwyvuxqc319bArV9jVMV_3gZH3OV434JYu8niYpDRRGigKImh2cy5oRjlaWVhtYWtBN1U1QUV4cnpZV1Bjb2kuWDhXaDIQyIAMtoken_accessor       Ri7jMehrYjAWvnN7EufsRX9O.X8Wh2token_duration       5h56m42stoken_renewable      falsetoken_policies       ["default" "hcp-root"]identity_policies    []policies             ["default" "hcp-root"]

Tip

Vault server is ready and the token value is store in the $HOME/.vault-token file.

Setup test secrets

In your terminal, clone the learn-vault-agent-envconsul repository which contains the example configuration used in this tutorial.
```
$ git clone https://github.com/hashicorp-education/learn-vault-agent-envconsul.git
```

You can explore this repository by changing directories.

$ cd learn-vault-agent-envconsul

The folder includes the following files:

.├── README.md├── kv-demo.sh├── pki-agent-config.hcl├── pki-demo.sh└── setup-secrets.sh0 directories, 5 files

Run the setup-secrets.sh script. This script enables kv-v2 secrets engine at web-team/ and dev-app/ paths and creates initial test data. Also, it configures PKI secrets engine to generate certificate which you will leverage in later section.
```
$ ./setup-secrets.sh
```

Read the secrets created at web-team/data/api-keys.

$ vault kv get -mount=web-team api-keys ===== Secret Path =====web-team/data/api-keys ======= Metadata =======Key                Value---                -----created_time       2023-06-13T21:05:24.965199Zcustom_metadata    <nil>deletion_time      n/adestroyed          falseversion            1 ==== Data ====Key     Value---     -----key1    4984h33sdkey2    o48r3eqf

Read the secrets created at dev-app/data/creds.

$ vault kv get -mount=dev-app creds === Secret Path ===dev-app/data/creds ======= Metadata =======Key                Value---                -----created_time       2023-06-13T21:05:25.042033Zcustom_metadata    <nil>deletion_time      n/adestroyed          falseversion            1 ====== Data ======Key         Value---         -----password    my-long-passworduser        tester1

Read the secrets created at dev-app/data/creds/database/db-admin.

$ vault kv get -mount=dev-app creds/database/db-admin ============ Secret Path ============dev-app/data/creds/database/db-admin ======= Metadata =======Key                Value---                -----created_time       2023-06-13T21:05:25.10932Zcustom_metadata    <nil>deletion_time      n/adestroyed          falseversion            1 ====== Data ======Key         Value---         -----password    o3ir23irusername    admin

Generate a Vault Agent config file

Vault Agent introduced the generate-config subcommand to auto generate the agent configuration file based on the user input.

Note

Currently, the generate-config subcommand supports kv-v1 and kv-v2 secrets paths.

Usage:

$ vault agent generate-config [options] <path_to_config>

Option	Description
-type	Currently, `env-template` is the only supported type of agent configuration
-exec	Pass the command to execute in agent process. The default is `env`
-path	Path to a kv-v1 or kv-v2 secret. Multiple paths and tail '*' wildcards are allowed.

Run the generate-config subcommand

Generates a Vault Agent configuration file named agent-config.hcl at the current working directory. If your application needs to pull secrets from multiple paths, you can provide multiple -path options as well as the wildcards (*).

Generate a Vault Agent configuration file named agent-config.hcl. Pass the kv-demo.sh as the command to execute.

$ vault agent generate-config -type="env-template" \    -exec="./kv-demo.sh" \    -path="web-team/api-keys" \    -path="dev-app/*" \    agent-config.hcl

Output:

Successfully generated "agent-config.hcl" configuration file!Warning: the generated file uses 'token_file' authentication method, which isnot suitable for production environments.

In the absence of the output file name, the command will generate a configuration file named agent.hcl.

Open the generated configuration file with your preferred text editor to examine.

auto_auth {   method {      type = "token_file"       config {         token_file_path = "/Users/username/.vault-token"      }   }} template_config {   static_secret_render_interval = "5m"   exit_on_retry_failure         = true} vault {   address = "http://127.0.0.1:8200"} env_template "API_KEYS_KEY1" {   contents             = "{{ with secret \"web-team/data/api-keys\" }}{{ .Data.data.key1 }}{{ end }}"   error_on_missing_key = true}env_template "API_KEYS_KEY2" {   contents             = "{{ with secret \"web-team/data/api-keys\" }}{{ .Data.data.key2 }}{{ end }}"   error_on_missing_key = true}env_template "CREDS_PASSWORD" {   contents             = "{{ with secret \"dev-app/data/creds\" }}{{ .Data.data.password }}{{ end }}"   error_on_missing_key = true}env_template "CREDS_USER" {   contents             = "{{ with secret \"dev-app/data/creds\" }}{{ .Data.data.user }}{{ end }}"   error_on_missing_key = true}env_template "DB_ADMIN_PASSWORD" {   contents             = "{{ with secret \"dev-app/data/creds/database/db-admin\" }}{{ .Data.data.password }}{{ end }}"   error_on_missing_key = true}env_template "DB_ADMIN_USERNAME" {   contents             = "{{ with secret \"dev-app/data/creds/database/db-admin\" }}{{ .Data.data.username }}{{ end }}"   error_on_missing_key = true} exec {   command                   = ["./kv-demo.sh"]   restart_on_secret_changes = "always"   restart_stop_signal       = "SIGTERM"}

auto_auth {   method {      type = "token_file"       config {         token_file_path = "/Users/username/.vault-token"      }   }} template_config {   static_secret_render_interval = "5m"   exit_on_retry_failure         = true} vault {   address = "https://vault-cluster-public-vault-XXXXXXXXX.XXXXXXXXX.XX.hashicorp.cloud:8200"} env_template "API_KEYS_KEY1" {   contents             = "{{ with secret \"web-team/data/api-keys\" }}{{ .Data.data.key1 }}{{ end }}"   error_on_missing_key = true}env_template "API_KEYS_KEY2" {   contents             = "{{ with secret \"web-team/data/api-keys\" }}{{ .Data.data.key2 }}{{ end }}"   error_on_missing_key = true}env_template "CREDS_PASSWORD" {   contents             = "{{ with secret \"dev-app/data/creds\" }}{{ .Data.data.password }}{{ end }}"   error_on_missing_key = true}env_template "CREDS_USER" {   contents             = "{{ with secret \"dev-app/data/creds\" }}{{ .Data.data.user }}{{ end }}"   error_on_missing_key = true}env_template "DB_ADMIN_PASSWORD" {   contents             = "{{ with secret \"dev-app/data/creds/database/db-admin\" }}{{ .Data.data.password }}{{ end }}"   error_on_missing_key = true}env_template "DB_ADMIN_USERNAME" {   contents             = "{{ with secret \"dev-app/data/creds/database/db-admin\" }}{{ .Data.data.username }}{{ end }}"   error_on_missing_key = true} exec {   command                   = ["./kv-demo.sh"]   restart_on_secret_changes = "always"   restart_stop_signal       = "SIGTERM"}

The auto_auth stanza is using token_file as a placeholder. You can change the stanza to use desired auth method. See the following tutorials for other auto auth examples:

Vault Agent with AWS demonstrates AWS auth method
Vault Agent with Kubernetes demonstrates Kubernetes auth method

Process supervisor mode

Each generated env_template block maps to a secret key of a KV path that you passed in the command.

Environment Variable Naming Convention

The exec block sets the kv-demo.sh script as the commands that Vault Agent to execute.

Open and examine the kv-demo.sh file. It reads secrets as environment variables.

#!/bin/sh echoecho "==== KV DEMO APP ===="echo "API_KEYS_KEY1     = ${API_KEYS_KEY1}"echo "API_KEYS_KEY2     = ${API_KEYS_KEY2}"echo "CREDS_USER        = ${CREDS_USER}"echo "CREDS_PASSWORD    = ${CREDS_PASSWORD}"echo "DB_ADMIN_USERNAME = ${DB_ADMIN_USERNAME}"echo "DB_ADMIN_PASSWORD = ${DB_ADMIN_PASSWORD}"

This application does not need to know about Vault.

Start a Vault Agent

Start a Vault Agent instance that connects to the Vault server running at VAULT_ADDR.

Start a Vault Agent.

$ vault agent -config=agent-config.hcl -log-level=error

Example output:

==> Vault Agent started! Log data will stream in below:==> Vault Agent configuration:         Api Address 1: http://bufconn                     Cgo: disabled               Log Level: error               Version: Vault v1.14.0-rc1, built 2023-06-06T18:12:53Z            Version Sha: 1327dc6728853611f62708e28dbac3ce6cabad1a==== KV DEMO APP ====API_KEYS_KEY1     = 4984h33sdAPI_KEYS_KEY2     = o48r3eqfCREDS_USER        = tester1CREDS_PASSWORD    = my-long-passwordDB_ADMIN_USERNAME = adminDB_ADMIN_PASSWORD = o3ir23ir

Restarts on secrets change

The secret values could change while your application is running. Examine the behavior when the secret values change.

Add sleep 100 in the kv-demo.sh to mock a long running application.

#!/bin/sh echoecho "==== DEMO APP ===="echo "API_KEYS_KEY1     = ${API_KEYS_KEY1}"echo "API_KEYS_KEY2     = ${API_KEYS_KEY2}"echo "CREDS_USER        = ${CREDS_USER}"echo "CREDS_PASSWORD    = ${CREDS_PASSWORD}"echo "DB_ADMIN_USERNAME = ${DB_ADMIN_USERNAME}"echo "DB_ADMIN_PASSWORD = ${DB_ADMIN_PASSWORD}" sleep 100

Edit the agent-config.sh file to set the static_secret_render_interval to 10 seconds for the purpose of demonstration. This makes Vault Agent to pull the secrets every 10 seconds.

auto_auth {   method {      type = "token_file"       config {         token_file_path = "/Users/username/.vault-token"      }   }} template_config {   static_secret_render_interval = "10s"   exit_on_retry_failure         = true} vault {   address = "$VAULT_ADDR"} ...snip...

Start the Vault Agent again.

$ vault agent -config=agent-config.hcl -log-level=error ==> Vault Agent started! Log data will stream in below: ==> Vault Agent configuration:          Api Address 1: http://bufconn                     Cgo: disabled               Log Level: error               Version: Vault v1.14.0-rc1, built 2023-06-06T18:12:53Z            Version Sha: 1327dc6728853611f62708e28dbac3ce6cabad1a ==== KV DEMO APP ====API_KEYS_KEY1     = 4984h33sdAPI_KEYS_KEY2     = o48r3eqfCREDS_USER        = tester1CREDS_PASSWORD    = my-long-passwordDB_ADMIN_USERNAME     = adminDB_ADMIN_PASSWORD = o3ir23ir

Open another terminal, and connect to the target Vault server.
Tip
Refer back to the Lab setup section.
Set the VAULT_ADDR environment variable.
$ export VAULT_ADDR=http://127.0.0.1:8200
Authenticate with Vault.
$ vault login root
Set the VAULT_ADDR environment variable
$ export VAULT_ADDR=<Public_Cluster_URL>
Set the VAULT_TOKEN environment variable.
$ export VAULT_NAMESPACE=admin
Authenticate with Vault Dedicated using the admin token.
$ vault login <admin_token>

Update the secret values at web-team/data/api-keys.

$ vault kv put web-team/api-keys \   key1="new-key-value-1" \   key2="new-key-value-2"

Return to the terminal where Vault Agent is running and watch the output.

==> Vault Agent started! Log data will stream in below:==> Vault Agent configuration:         Api Address 1: http://bufconn                     Cgo: disabled               Log Level: error               Version: Vault v1.14.0-rc1, built 2023-06-06T18:12:53Z            Version Sha: 1327dc6728853611f62708e28dbac3ce6cabad1a==== KV DEMO APP ====API_KEYS_KEY1     = 4984h33sdAPI_KEYS_KEY2     = o48r3eqfCREDS_USER        = tester1CREDS_PASSWORD    = my-long-passwordDB_ADMIN_USERNAME = adminDB_ADMIN_PASSWORD = o3ir23ir==== KV DEMO APP ====API_KEYS_KEY1     = new-key-value-1API_KEYS_KEY2     = new-key-value-2CREDS_USER        = tester1CREDS_PASSWORD    = my-long-passwordDB_ADMIN_USERNAME = adminDB_ADMIN_PASSWORD = o3ir23ir

Press Ctrl + C to stop the running Vault Agent.

Work with dynamic secrets

Currently, the generate-config subcommand only supports kv-v1 and kv-v2 secrets engines; however, the Vault Agent's process supervisor mode supports dynamic secrets that Envconsul supports today. The difference is that you would have to manually define the env_template blocks.

The setup-secrets.sh script configured the PKI secrets engine and it is ready to serve.

$ vault secrets list

Example output:

Path          Type         Accessor              Description----          ----         --------              -----------cubbyhole/    cubbyhole    cubbyhole_595f8b0f    per-token private secret storagedev-app/      kv           kv_f16c3aac           n/aidentity/     identity     identity_cd1c4e99     identity storepki/          pki          pki_ca9e7398          n/apki_int/      pki          pki_331d3c63          n/asecret/       kv           kv_1bb31236           key/value secret storagesys/          system       system_a6bb36e2       system endpoints used for control, policy and debuggingweb-team/     kv           kv_c5dd184d           n/a

(Optional) Request a certificate.

$ vault write -field=certificate pki_int/issue/example-dot-com \   common_name="test.example.com" ttl="24h"

Example output:

Open the pki-agent-config.hcl to examine the env_template block it defines.

env_template "CERT" {   contents = "{{ with pkiCert \"pki_int/issue/example-dot-com\" \"common_name=test.example.com\" \"ttl=24h\"}}{{ .Data.Cert }}{{ end }}"   error_on_missing_key = true} exec {   command                   = ["./pki-demo.sh"]   restart_on_secret_changes = "always"   restart_stop_signal       = "SIGTERM"}

Open the pki-demo.sh file to review.
```
#!/bin/sh echoecho "==== CERT DEMO APP ===="echo "CERT = ${CERT}"
```
This mock application gets certificate stored in the CERT environment variable. Similar to the kv-demo.sh, this application does not need to know anything about Vault.

Run a Vault Agent with additional env_template definition.

$ vault agent -config=agent-config.hcl \   -config=pki-agent-config.hcl \   -log-level=error

Example output:

==> Vault Agent started! Log data will stream in below:==> Vault Agent configuration:         Api Address 1: http://bufconn                     Cgo: disabled               Log Level: error               Version: Vault v1.14.0-rc1, built 2023-06-06T18:12:53Z            Version Sha: 1327dc6728853611f62708e28dbac3ce6cabad1a==== CERT DEMO APP ====CERT = -----BEGIN CERTIFICATE-----MIIDZjCCAk6gAwIBAgIUdTJEGBZtcnxlo8f0mMLVzKvXbtgwDQYJKoZIhvcNAQELBQAwLTErMCkGA1UEAxMiZXhhbXBsZS5jb20gSW50ZXJtZWRpYXRlIEF1dGhvcml0eTAeFw0yMzA2MTMyMzQ3MDNaFw0yMzA2MTQyMzQ3MzNaMBsxGTAXBgNVBAMTEHRlc3QuZXhhbXBsZS5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC3GXTT5UlMnH5OJKZbx7n9DJs4K4lgrKdirpD4mVL7O4ba3KUA/mam9M6LAqn5YzB6UX6y1+zIyNaakqltxLVxiyLWodqix3fjN9g6jzis5JAQM3UIgVYPBpg5ptKzpVzHASvuDonnPflBYQG609rD56RK8ycyhq0bcUVCa6pEQI3EtJaqj2woOTVWyauo+r7gjrwr49BL0ye0T1IoTSs3l0E5Pk6ZcTDEbK3rNZ+YWSzwcg6bAL1U8kOW3B3jINwIGYXPKVVLM2v6TTYpeYB/fZ6FeQeDasVU3iy3N16HdIHbI2vAQqUkTZcv/LM1ALgfDbEYOUvAcdNCZKifhDTdAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgOoMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAdBgNVHQ4EFgQUBIsJvc4reYPkq1Vytuf3NMAHvD0wHwYDVR0jBBgwFoAUOiYGxwIjDoTz389uuIGb9VWmEKUwGwYDVR0RBBQwEoIQdGVzdC5leGFtcGxlLmNvbTANBgkqhkiG9w0BAQsFAAOCAQEAZyN5dSKo9E8G6kaArX6Jfq8CGdX1jhaqFH8SljYZ5/WngzJwuQNkvCU42TpAbvHsBfen93T/VfGgPPh6GohwhvLbVQjTLzGBlFTq+oQJs0ntQxjL/CMEfqg0aLH1OnZuVULc9J6zLaOHhsV4EHx7c5iSuwwun7NOJgysadboS8EE4BOqkKehaEz/ZXVVM5sJhRpi8KfRI9EA+FNG6URKllW72OM2kbg9HVQsxsIPdG+o/JDsCz48yMwxJDyLAxMH/YtOVrbFj55N7ooFPw/h3BhMP+0zzDrQrl9GPYRCe//10zmTA+4/q2vQ3nPMVeInEa+QdTBihxz5k4LuDjhuoA==-----END CERTIFICATE-----

Similar to what you observed in the restarts on secrets change section, if the certificate expires and the application is still running, Vault Agent can fetch a new certificate.

Clean up

Press Ctrl + C to stop the running Vault Agent.
You can stop the Vault dev server by pressing Ctrl+C where the server is running. Or, execute the following command.
```
$ pgrep -f vault | xargs kill
```
Unset the VAULT_ADDR environment variable.
```
$ unset VAULT_ADDR
```

Press Ctrl + C to stop the running Vault Agent.
Disable the kv-v2 secrets engine enabled at web-team.
```
$ vault secrets disable web-team
```
Disable the kv-v2 secrets engine enabled at dev-app.
```
$ vault secrets disable dev-app
```
Disable the PKI secrets engine enabled at pki.
```
$ vault secrets disable pki
```
Disable the PKI secrets engine enabled at pki_int.
```
$ vault secrets disable pki_int
```
Unset the VAULT_TOKEN, VAULT_NAMESPACE and VAULT_ADDR environment variables.
```
$ unset VAULT_TOKEN VAULT_ADDR VAULT_NAMESPACE
```

Summary

You learned the use of the generate-config subcommand and the process supervisor mode to take advantage of secrets presented as environment variables.

Vault Agent can handle the authentication and secrets retrieval so that your application can remain Vault unaware. This reduces the barrier to adopting Vault and keep your applications secure.

There are several tutorials demonstrates the use of Vault Agent. See the available Vault Agent tutorials.

Help and reference

Cache secrets with Vault Agent

Reload secrets with Vault Agent and .NET