PKI secrets engine with managed keys

11min
|
Enterprise
Vault

Challenge

While using Vault's PKI secrets engine to generate dynamic X.509 certificates, an organization may require their private keys to be created or stored within PKCS#11 hardware security modules (HSMs) to meet regulatory requirements.

Solution

Vault version 1.10 introduced the managed keys system to allow operators to configure access to a key stored in an external KMS. This feature also adds support to the PKI secrets engine for using a configured managed key as the backing private key for an intermediate or root certificate authority (CA).

Managed keys offload some PKI operations to the HSM or cloud KMS.

Key generation: Generate new PKI key pairs and certificates from external HSM or cloud KMS.
Certificate signing: Verify and sign some certificate workflows within external HSM or cloud KMS.

When Vault is configured with managed keys, all operations related to the private key, including generation, happen within the secure boundary of the HSM or cloud KMS external to Vault.

Prerequisites

To perform the tasks described in this tutorial, you need:

Vault Enterprise version 1.10 or later
HSM or AWS KMS environment

Note

PKCS#11 HSM-backed managed keys requires Vault Enterprise license.

Policy requirements

Note

For the purpose of this tutorial, you can use a root token to work with Vault. However, it is recommended that root tokens are only used for just enough initial setup or in emergencies. As a best practice, use tokens with appropriate set of policies based on your role in the organization.

To perform all tasks demonstrated in this tutorial, your policy must include the following capabilities:

# Work with transform secrets enginepath "sys/managed-keys/*" {  capabilities = [ "create", "read", "update", "list" ]} # Enable secrets enginepath "pki/*" {  capabilities = [ "create", "read", "update", "delete", "list" ]} # List enabled secrets enginepath "sys/mounts" {  capabilities = [ "read", "list" ]} # Tune mountspath "sys/mounts/pki/tune" {  capabilities = ["create", "update"]}

If you are not familiar with policies, complete the policies tutorial.

In your organization, the configuration of a managed key and the configuration of a secrets engine may be performed by different people, in which case you might choose to segment the Vault policies accordingly.

Start Vault

For convenience and simplicity in the tutorial, you will operate a single Vault server with a minimal configuration using the default Shamir's Secret Sharing seal with one key share.

Open a terminal and create the initial Vault server configuration.

Note

Be sure to set the license_path to where your Vault license key is located.

$ cat > /tmp/vault-server.hcl << EOFdisable_mlock = truelicense_path  = "/etc/vault.d/license.hclic"api_addr      = "http://127.0.0.1:8200"cluster_addr  = "http://127.0.0.1:8201" storage "raft" {   path    = "/tmp/vault-data"   node_id = "node_1"} listener "tcp" {   address = "0.0.0.0:8200"   tls_disable = true} EOF

Create the Vault server data directory.
```
$ mkdir /tmp/vault-data
```
Start the Vault server.
```
$ vault server -config /tmp/vault-server.hcl
```
The Vault dev server defaults to running at 127.0.0.1:8200. It is uninitialized and not yet ready for use.
Open a new terminal session and export an environment variable for the vault CLI to address the Vault server.
```
$ export VAULT_ADDR=http://127.0.0.1:8200
```
Initialize the Vault server and capture the output to the file /tmp/.vault-init
```
$ vault operator init -key-shares=1 -key-threshold=1 > /tmp/.vault-init
```

Export the VAULT_UNSEAL_KEY environment variable with the unseal key from the file as its value.

$ export VAULT_UNSEAL_KEY="$(grep 'Unseal Key 1' /tmp/.vault-init \    | awk '{print $NF}')"

Export the VAULT_ROOT_TOKEN environment variable with the initial root token from the file as its value.
```
$ export VAULT_TOKEN="$(grep 'Initial Root Token' /tmp/.vault-init \    | awk '{print $NF}')"
```

Unseal Vault.

$ vault operator unseal $VAULT_UNSEAL_KEY

The Vault server is ready.

Configure a Managed Key

Choose either PKCS#11 HSM or AWS KMS, and follow the steps for your KMS choice.

This tutorial uses the SoftHSM2 software (refer to its documentation for setup instructions). The path to the SoftHSM library varies by platform. These instructions apply to an installation of Ubuntu Linux, but other platforms will also work.

Initialize a new PKCS#11 token.

$ softhsm2-util --init-token --free --so-pin=supersecret --pin=prettysecret \   --label="test-kms-root"

SoftHSM example response:

Slot 24 has a free/uninitialized token. The token has been initialized and isreassigned to slot 1601102058

Note

The slot number is returned. In this example, the slot number is 1601102058.

A PKCS#11 HSM requires a shared library to configure Vault to communication with the HSM. To declare this library, edit the Vault server configuration at /tmp/vault-server.hcl/ to add a kms_library stanza.

disable_mlock = truelicense_path  = "/etc/vault.d/license.hclic"api_addr      = "http://127.0.0.1:8200"cluster_addr  = "http://127.0.0.1:8201"   storage "raft" {   path    = "/tmp/vault-data"   node_id = "node_1"} listener "tcp" {   address = "0.0.0.0:8200"   tls_disable = true} kms_library "pkcs11" {    name = "myhsm"    library = "/usr/lib/x86_64-linux-gnu/softhsm/libsofthsm2.so"}

Restart or reload Vault's configuration with SIGHUP.
```
$ kill -HUP $(pidof vault)
```
Each managed key requires type-specific configuration. You must identify the location of a key and PIN to access the HSM. This is very similar to Vault's PKCS#11 auto-unseal mechanism. Configure the managed key.
```
$ vault write sys/managed-keys/pkcs11/learn-managed-key \      library=myhsm slot=1601102058 pin=prettysecret \      key_label=my-hsm-pki-key \      allow_store_key=true \      allow_generate_key=true \      mechanism=0x0001 key_bits=4096 \      any_mount=false
```
The PKCS#11 specific parameters are library, referring to the previously configured kms_library stanza, slot, pin, key_label, and mechanism, which identifies the object in the HSM which will hold the key, and that it's mechanism is CKM_RSA_PKCS (RSA with PKCS#1 v1.5 signatures), and that we will require a 4096 bit key. The allow_generate_key flag indicates that Vault is allowed to request that the HSM generate a key, and allow_store_key indicates that a new key may be stored in the backend. Without this flag, you may configure the key yourself in the HSM and just point Vault at the result. Finally, any_mount means any mount in the namespace may access the managed key. For this example, it is set to false so that you can demonstrate how to lock managed key access down to a specific mount.
Vault output:
```
Success! Data written to: sys/managed-keys/pkcs11/learn-managed-key
```

Read the key back.

$ vault read /sys/managed-keys/pkcs11/learn-managed-key Key                  Value---                  -----UUID                 f1a88f60-3e10-924f-7071-f862250ca4e9allow_generate_key   trueallow_replace_key    falseallow_store_key      trueany_mount            falsekey_bits             4096key_label            my-hsm-pki-keylibrary              myhsmmechanism            1name                 learn-managed-keypin                  redactedslot                 1042332116type                 pkcs11

Test endpoint to validate that we can access the managed key.

$ vault write -force /sys/managed-keys/pkcs11/learn-managed-key/test/sign Success! Data written to: sys/managed-keys/pkcs11/learn-managed-key/test/sign

The key has been generated in the HSM and used to sign and verify a dummy value.

AWS Requirements

To complete this tutorial, you must have an access key and secret key for an AWS account. The AWS account must have permissions to create AWS KMS keys and aliases. To set the configuration through AWS, set the environment variables AWS_ACCESS_KEY, AWS_SECRET_ACCESS_KEY and AWS_REGION with the appropriate values or use AWS CLI. For instructions on how to use the AWS CLI, see the AWS documentation. You can also add these credentials to the managed key configuration itself.

Create the managed key in Vault.
If the AWS credentials have been set in environment variables accessible to the Vault server or via AWS command line tools, run the following command.
$ vault write sys/managed-keys/awskms/learn-managed-key-pki \ kms_key=alias/hashicorp/learn-managed-key-pki \ key_type=rsa \ key_bits=2048 \ allow_generate_key=true
To store the AWS credentials within the managed key configuration, use the optional access_key, secret_key, and region parameters.
$ vault write sys/managed-keys/awskms/learn-managed-key-pki \ kms_key=alias/hashicorp/learn-managed-key-pki \ key_type=rsa \ key_bits=2048 \ allow_generate_key=true \ access_key=<access_key> \ region=<region> \ secret_key=<secret_key>
For AWS KMS, managed keys require an identifier for the key (either an AWS alias or the key ID if using an existing key), the type of key (RSA in this example), and the size of the key (2048 bits in this example). The allow_generate_key parameter indicates that Vault should generate a new KMS key if one doesn't already exist with the provided alias.
Successful output:
```
Success! Data written to: sys/managed-keys/awskms/learn-managed-key-pki
```
This step only creates the managed key configuration in Vault. The AWS KMS key has not been created yet.

Read the key back.

$ vault read sys/managed-keys/awskms/learn-managed-key-pki

Vault returns the managed key configuration and you can verify your configuration.

Example output:

Key                 Value---                 -----UUID                3e57cc66-7551-8c3b-2f91-bd8b36a56da2allow_generate_key  trueallow_replace_key   falseallow_store_key     falseany_mount           falsekey_bits            2048key_type            rsakms_key             alias/hashicorp/learn-managed-key-pkiname                learn-managed-key-pkitype                awskms

If you included the AWS credentials in the managed key configuration, you will also see keys access_key and secret_key with their values redacted.

Key                   Value---                   -----UUID                  da5c28cd-d98d-14b8-ab64-e21c020a7ffdaccess_key            redactedallow_generate_key    trueallow_replace_key     falseallow_store_key       falseany_mount             falsekey_bits              2048key_type              rsakms_key               alias/hashicorp/learn-managed-key-pkiname                  learn-managed-key-pkiregion                us-east-1secret_key            redactedtype                  awskms

Validate that we can access the managed key.
```
$ vault write -f sys/managed-keys/awskms/learn-managed-key-pki/test/sign
```
This may take a couple of seconds.
Vault output:
```
Success! Data written to: sys/managed-keys/awskms/learn-managed-key-pki/test/sign
```
Troubleshooting
If this operation fails, check that your AWS credentials are not expired and that the account has permissions for both AWS KMS and AWS aliases.
Vault will generate an AWS KMS key as part of this test. After this step, you should be able to log into the AWS console and find an AWS KMS key with the alias hashicorp/learn-managed-key-pki.

You may now move on to configuring the PKI secrets engine to use this key.

Configure PKI Secrets Engine

Enable the pki secrets engine at the pki path.

$ vault secrets enable pkiSuccess! Enabled the pki secrets engine at: pki/

Tune the secrets engine to use managed keys.
```
$ vault secrets tune -allowed-managed-keys=learn-managed-key pkiSuccess! Tuned the secrets engine at: pki/
```
You configured the managed key with any_mount=false in the previous step. This command grants access to the learn-managed-key to the PKI secrets engine's mount.
Generate the root certificate.
```
$ vault write -field=certificate pki/root/generate/kms \    managed_key_name=learn-managed-key \    common_name=root.myco.com \    ttl=8760h
```
Note
Notice that the path is pki/root/generate/kms (instead of pki/root/generate/internal or pki/root/generate/exported). This indicates that a managed key will be used. The name of the managed key is specified (learn-managed-key).
Configure the CA and CRL URLs.
```
$ vault write pki/config/urls \    issuing_certificates="$VAULT_ADDR/v1/pki/ca" \    crl_distribution_points="$VAULT_ADDR/v1/pki/crl"
```
Successful output:
```
Success! Data written to: pki/config/urls
```
In a production environment, you may configure an additional mount or mounts for intermediate CAs, and have those intermediates signed by the root. Refer to the Build Your Own Certificate Authority (CA) tutorial.

Test that you can issue a certificate with your managed key-backed root by configuring and using a role.

$ vault write pki/roles/example-dot-com \    allowed_domains="example.com" \    allow_subdomains=true \    max_ttl="720h"

Successful output:

Success! Data written to: pki/roles/example-dot-com

Request a new certificate for the test.example.com domain based on the example-dot-com role.

$ vault write -format=json pki/issue/example-dot-com \common_name="test.example.com" ttl="24h"

Example output:

Key                 Value---                 -----certificate         -----BEGIN CERTIFICATE-----MIIDwzCCAqugAwIBAgIUTQABMCAsXjG6ExFTX8201xKVH4IwDQYJKoZIhvcNAQELBQAwGjEYMBYGA1UEAxMPd3d3LmV4YW1wbGUuY29tMB4XDTE4MDcyNDIxMTMxOVoX             ...-----END CERTIFICATE-----issuing_ca          -----BEGIN CERTIFICATE-----MIIDQTCCAimgAwIBAgIUbMYp39mdj7dKX033ZjK18rx05x8wDQYJKoZIhvcNAQEL             ...-----END CERTIFICATE-----private_key         -----BEGIN RSA PRIVATE KEY-----MIIEowIBAAKCAQEAte1fqy2Ekj+EFqKV6N5QJlBgMo/U4IIxwLZI6a87yAC/rDhmW58liadXrwjzRgWeqVOoCRr/B5JnRLbyIKBVp6MMFwZVkynEPzDmy0ynuomSfJkM             ...-----END RSA PRIVATE KEY-----private_key_type    rsaserial_number       4d:00:01:30:20:2c:5e:31:ba:13:11:53:5f:cd:b4:d7:12:95:1f:82

Next steps

Leveraging managed keys, operators can run a highly secure CA backed by an HSM or cloud KMS, while preserving the simplicity and flexibility of Vault's PKI secrets engine.

To learn more about the PKI secrets engine, see the following tutorials:

Help and reference

PKI Unified CRL and OCSP

SSH secrets engine: One-time SSH password

This tutorial also appears in:

25 tutorials

Vault Enterprise
Learn features that are only available to Vault Enterprise.
- Vault