Key Management Secrets Engine with Azure Key Vault

Challenge

You need to use security features of cloud providers, which involve encryption keys issued and stored by the provider in its own key management system (KMS), but you must also maintain root of trust and control of the encryption key lifecycle.

Solution

The Vault Key Management Secrets Engine provides distribution and lifecycle management features for cloud provider keys.

Currently, it manages keys for the following cloud providers.

• Amazon AWS KMS
• Azure Key Vault
• Google Cloud Platform Cloud Key Management Service

Personas

The end-to-end scenario described in this tutorial involves one persona:

• admin a user with with privileged permissions to configure secrets engines.

Prerequisites

To perform the tasks described in this tutorial, you need to have:

• A Microsoft Azure account with sufficient permissions to manage applications, service principles, and key vaults.
• Vault Enterprise with the Advanced Data Protection Module at version 1.6 or later.

Policy requirements

Each persona requires a different set of capabilities. These are expressed in policies. If you are not familiar with policies, complete the policies tutorial.

The admin tasks require these capabilities.

# Mount secrets enginespath "sys/mounts/*" {  capabilities = [ "create", "read", "update", "delete", "list" ]} # Configure the default Key Management Secrets Engine instancepath "keymgmt/*" {  capabilities = [ "create", "read", "update", "delete", "list" ]}

Start Vault dev server

In a new terminal session, start a Vault dev server with the value of the initial root token set to root.

$ vault server -dev -dev-root-token-id root

The Vault dev server defaults to listening at 127.0.0.1:8200. The server is also automatically initialized and unsealed.

In another new terminal session, export an environment variable for the vault CLI to address the dev server.

$ export VAULT_ADDR=http://127.0.0.1:8200

Then export an environment variable for the vault CLI to authenticate with the Vault server.

$ export VAULT_TOKEN=root

Refer to Dev Options to learn more about Vault dev server options.

The Vault dev server is now ready for use.

Scenario Introduction

You will configure a Microsoft Azure Key Vault instance, then use the Vault CLI, HTTP API or web UI to configure a Vault development mode server and enable management of the Key Vault key with the Key Management secrets engine.

After initial configuration, you will perform some lifecycle operations with the secrets engine, like writing, reading, updating, and rotating keys.

1. Configure Azure Key Vault
2. Enable Key Management secrets engine
3. Write keys
4. Read keys
5. Create KMS provider
6. Distribute keys to key vault
7. Rotate key

Configure Azure Key Vault

To follow this tutorial, you must configure an Azure Key Vault instance and assign an access policy that provides the key management policy to a service principal. You then need to generate a credential that Vault will use to connect to and manage the Key Vault.

In this tutorial, the Azure Key Vault instance is named learn-key-vault.

You can refer to the How to: Use the portal to create an Azure AD application and service principal that can access resources and Assign a Key Vault access policy using the Azure portal documentation for further details on these steps.

When complete, you should have a key vault available as shown in this screenshot.

You need to note the value of key vault name (learn-keyvault-52246b in this example) to use when you configure Vault later.

Terraform configuration example

If you are familiar with Terraform, you can use it to deploy the necessary Azure infrastructure for this tutorial with the following configuration example.

provider "azuread" {  version = "=0.11.0"}provider "azurerm" {  features {    key_vault {      purge_soft_delete_on_destroy = true    }  }}resource "random_id" "app_rg_name" {  byte_length = 3}resource "random_id" "keyvault_name" {  byte_length = 3}data "azurerm_client_config" "current" {}resource "azuread_application" "key_vault_app" {  name                       = "app-${random_id.app_rg_name.hex}"  homepage                   = "http://homepage${random_id.app_rg_name.b64_url}"  identifier_uris            = ["http://uri${random_id.app_rg_name.b64_url}"]  reply_urls                 = ["http://replyur${random_id.app_rg_name.b64_url}"]  available_to_other_tenants = false  oauth2_allow_implicit_flow = true}resource "azuread_service_principal" "key_vault_sp" {  application_id               = azuread_application.key_vault_app.application_id  app_role_assignment_required = false}resource "random_password" "password" {  length           = 24  special          = true  override_special = "%@"}resource "azuread_service_principal_password" "key_vault_sp_pwd" {  service_principal_id = azuread_service_principal.key_vault_sp.id  value                = random_password.password.result  end_date             = "2099-01-01T01:02:03Z"}resource "azurerm_resource_group" "key_vault_rg" {  name     = "learn-rg-${random_id.app_rg_name.hex}"  location = "West US"}resource "azurerm_key_vault" "key_vault_kv" {  name                = "learn-keyvault-${random_id.keyvault_name.hex}"  location            = azurerm_resource_group.key_vault_rg.location  resource_group_name = azurerm_resource_group.key_vault_rg.name  sku_name            = "premium"  soft_delete_enabled = true  tenant_id           = data.azurerm_client_config.current.tenant_id  access_policy {    tenant_id = data.azurerm_client_config.current.tenant_id    object_id = data.azurerm_client_config.current.object_id    key_permissions = [      "backup",      "create",      "decrypt",      "delete",      "encrypt",      "get",      "import",      "list",      "purge",      "recover",      "restore",      "sign",      "unwrapKey",      "update",      "verify",      "wrapKey"    ]  }  access_policy {    tenant_id = data.azurerm_client_config.current.tenant_id    object_id = azuread_service_principal.key_vault_sp.object_id    key_permissions = [      "create",      "delete",      "get",      "import",      "update"    ]  }}output "key_vault_1_name" {  value = azurerm_key_vault.key_vault_kv.name}output "tenant_id" {  value = data.azurerm_client_config.current.tenant_id}output "client_id" {  value = azuread_application.key_vault_app.application_id}output "client_secret" {  value = azuread_service_principal_password.key_vault_sp_pwd.value}

Enable Key Management secrets engine

You must first enable an instance of the Key Management Secrets engine before you can use it.

The default behavior without specifying a path to enable the secrets engine on is to enable it at the path keymgmt/.

Enable the secrets engine.

$ vault secrets enable keymgmtSuccess! Enabled the keymgmt secrets engine at: keymgmt/

The secrets engine is now enabled and ready for use.

Write keys

The Key Management secrets engine currently supports generation of the key types specified in Key Types.

Based on the Compatibility section of the documentation, Azure Key Vault currently supports use of RSA-2048, RSA-3072, and RSA-4096 key types.

Write a pair of RSA-2048 keys to the secrets engine.

$ vault write keymgmt/key/rsa-1 type="rsa-2048"Success! Data written to: keymgmt/key/rsa-1

With this command line syntax, you are instructing Vault to write a new key of type rsa-2048 to the path keymgmt/key/rsa-1. The key that is written is for general purpose use, but you will set its specific use later.

You can do the same again to write the second key, but change the path to keymgmt/key/rsa-2.

$ vault write keymgmt/key/rsa-2 type="rsa-2048"Success! Data written to: keymgmt/key/rsa-2

Read keys

Now, to learn about the content of a key, you can read it back. Let's read the rsa-1 key, use JSON as the output, and pipe that into jq.

$ vault read -format=json keymgmt/key/rsa-1 | jq 

Notice the value of latest_version; it is 1 since this is the first version of the key that Vault knows about. This will figure into the example on key rotation later.

Read the second key.

$ vault read -format=json keymgmt/key/rsa-2 | jq 

Create KMS provider

The next step is to create the KMS provider configuration in Vault; this corresponds to the key vault instance you previously created in Azure.

Create the KMS provider at the path keymgmt/kms/keyvault.

$ vault write keymgmt/kms/keyvault \    key_collection="learn-keyvault-52246b" \    provider="azurekeyvault" \    credentials=client_id="c0ff33c3-d073-40ba-a16e-9fb1d39e1973" \    credentials=client_secret="COFFEE.2q-esz4~01d92OLOVE1974C6_t_" \    credentials=tenant_id="c0ff3374-ba40-4725-ae1e-c8675309213c"Success! Data written to: keymgmt/kms/keyvault

In this command line example, you specified 5 pieces of information to successfully configure the key vault instance.

• The full path to this KMS provider instance in Vault (keymgmt/kms/keyvault).
• A key collection, which corresponds to the name of the key vault instance in Azure.
• The KMS provider is set to "azurekeyvault".
• Azure client ID credential, that can also be specified with AZURE_CLIENT_ID environment variable.
• Azure client secret credential, that can also be specified with AZURE_CLIENT_SECRET environment variable.
• Azure tenant ID credential, that can also be specified with AZURE_TENANT_ID environment variable.

Distribute keys to key vault

To use the keys you wrote in step 3, you must first distribute them to the key vault you just created in step 5. Add the rsa-1 key to the key vault at the path keymgmt/kms/keyvault/key/rsa-1.

$ vault write keymgmt/kms/keyvault/key/rsa-1 \    purpose="encrypt,decrypt" \    protection="hsm"Success! Data written to: keymgmt/kms/keyvault/key/rsa-1

Here you are instructing Vault to distribute the key and specify that its purpose is only to encrypt and decrypt. The protection type is dependent on the cloud provider and the value is either hsm or software. In the case of Azure, you specify hsm for the protection type. The key will be securely delivered to the key vault instance according to the Azure Bring Your Own Key (BYOK) specification.

Write the second key to the path keymgmt/kms/keyvault/key/rsa-2 next, but specify that its purpose is only for signing.

$ vault write keymgmt/kms/keyvault/key/rsa-2 \    purpose="sign" \    protection="hsm"Success! Data written to: keymgmt/kms/keyvault/key/rsa-2

Verify keys

You can use the vault CLI to list the keys that have been distributed to the Azure Key Vault instance.

$ vault list keymgmt/kms/keyvault/key/Keys----rsa-1rsa-2

You can also visit the Azure UI to confirm the key distribution.

Inspecting the key rsa-1 shows its current version.

Clicking into the key further also shows more details, like permitted operations, which in this case include only Encrypt and Decrypt.

Inspecting the key rsa-2 shows its permitted operations, which includes only Sign.

Rotate key

Key rotation is a fundamental part of the key management lifecycle. Let's use Vault to rotate the rsa-1 key and then verify its version with the vault CLI and Azure UI.

Rotate the key.

$ vault write -f keymgmt/key/rsa-1/rotateSuccess! Data written to: keymgmt/key/rsa-1/rotate

Confirm key rotation

You can confirm successful key rotation by reading the key with the CLI, and getting the value of .data.latest_version.

$  vault read -format=json keymgmt/key/rsa-1 | jq '.data.latest_version'2

The key is now at version 2; in the Azure UI, the key will be expected to have a different version string than the original value under CURRENT VERSION and the original version string (e27d7baeacc543e4bfde70891d4be4b3) should be present under OLDER VERSIONS as shown in the Azure UI screenshot.

Summary

You have learned the basics about the Key Management secrets engine, including configuring Azure, enabling the secrets engine, and performing basic key lifecycle management operations.

From here, you can learn about other Vault secrets engines by going through the Secrets Management tutorials.

Help and reference

• Vault dev server options
• Key Management Secrets Engines documentation
• Azure Bring Your Own Key specification
• How to: Use the portal to create an Azure AD application and service principal that can access resources
• Assign a Key Vault access policy using the Azure portal