Write a policy using audit logs

Every Vault operation performed through the command-line interface (CLI), API, or web UI require that the authenticated client is granted access; access defined through policies. Everything in Vault is stored at different paths, like a filesystem, and every action in Vault has a corresponding path and capability. Policies provide a declarative way to grant or forbid access to paths and the capabilities at each path.

In this tutorial, you will learn Vault's policy language and how to query an audit log to define policies.

Prerequisites

• Vault

• jq for consuming Vault JSON formatted output and capturing specific fields.

Start Vault

Start a Vault dev server with root as the root token.

$ vault server -dev -dev-root-token-id root

The Vault dev server defaults to running at 127.0.0.1:8200. The server is initialized and unsealed.

In another terminal, export an environment variable for the vault CLI to address the Vault server.

$ export VAULT_ADDR=http://127.0.0.1:8200

Export an environment variable for the vault CLI to authenticate with the Vault server.

$ export VAULT_TOKEN=root

The Vault server is ready.

Enable audit logs

Audit devices keep a detailed log of all requests and response to Vault. The file audit device appends these events to a file.

Enable the file audit device at a file named vault-audit.log in the current directory.

$ vault audit enable file file_path=$PWD/vault-audit.logSuccess! Enabled the file audit device at: file/

The file audit device is enabled and immediately appends a test message to the audit log file.

Display the contents of the audit log file.

$ cat vault-audit.log | jq

The output displays two entries similar to these examples.

{  "time": "2023-06-21T16:53:09.915468Z",  "type": "request",  "auth": {    "token_type": "default"  },  "request": {    "id": "3f11991f-98d6-d88e-94db-f3e1dfab0c0d",    "operation": "update",    "namespace": {      "id": "root"    },    "path": "sys/audit/test"  }}{  "time": "2023-06-21T16:53:09.925391Z",  "type": "response",  "auth": {    "client_token": "hmac-sha256:2538d1fb96fe45b811af0c6db2c2cbe78e4f10d1d3ad0c8eb6f7dd36828587bb",    "accessor": "hmac-sha256:defd1a0c008c6438b48d733578ca7f50f978c2db2750a1e508b4d1d8674d670d",    "display_name": "token",    "policies": [      "root"    ],    "token_policies": [      "root"    ],    "policy_results": {      "allowed": true,      "granting_policies": [        {          "name": "root",          "namespace_id": "root",          "type": "acl"        }      ]    },    "token_type": "service",    "token_issue_time": "2023-06-21T12:52:53-04:00"  },  "request": {    "id": "e11d675e-6e83-1ce1-48a4-8f82869ccb9b",    "client_id": "0DHqvq2D77kL2/JTPSZkTMJbkFVmUu0TzMi0jiXcFy8=",    "operation": "update",    "mount_point": "sys/",    "mount_type": "system",    "mount_accessor": "system_18f60a0c",    "mount_running_version": "v1.14.0+builtin.vault",    "mount_class": "secret",    "client_token": "hmac-sha256:2538d1fb96fe45b811af0c6db2c2cbe78e4f10d1d3ad0c8eb6f7dd36828587bb",    "client_token_accessor": "hmac-sha256:defd1a0c008c6438b48d733578ca7f50f978c2db2750a1e508b4d1d8674d670d",    "namespace": {      "id": "root"    },    "path": "sys/audit/file",    "data": {      "description": "hmac-sha256:5f9da19df55a1a3f82d640bbf552847a11da9bfd962ef63cbc4848e78bb36ad4",      "local": false,      "options": {        "file_path": "hmac-sha256:b4a151e47019f98aab7b38f0af27bd967cad01d863cb83c75ddb07817029bb02"      },      "type": "hmac-sha256:7967d1e46655340a7dfce10305eebea1a80d70cef13e9b064c84b9e4b415b6a6"    },    "remote_address": "127.0.0.1",    "remote_port": 51132  },  "response": {    "mount_point": "sys/",    "mount_type": "system",    "mount_accessor": "system_18f60a0c",    "mount_running_plugin_version": "v1.14.0+builtin.vault",    "mount_class": "secret"  }}

The audit log appends JSON objects to the log file. The first object describes an audit test message. The second object describes the operation that enabled the audit log.

Enable transit secrets engine

The transit secrets engine handles cryptographic functions on data in-transit through keys that Vault manages.

Enable the transit secrets engine at the default path.

$ vault secrets enable transitSuccess! Enabled the transit secrets engine at: transit/

The transit secrets engine is enabled and the operation that performed it was written to the audit log.

View only the last entry in the audit log file.

$ cat vault-audit.log | jq -s ".[-1]"

The output displays an entry similar to this example.

{  "time": "2023-06-21T16:54:04.026976Z",  "type": "response",  "auth": {    "client_token": "hmac-sha256:2538d1fb96fe45b811af0c6db2c2cbe78e4f10d1d3ad0c8eb6f7dd36828587bb",    "accessor": "hmac-sha256:defd1a0c008c6438b48d733578ca7f50f978c2db2750a1e508b4d1d8674d670d",    "display_name": "token",    "policies": [      "root"    ],    "token_policies": [      "root"    ],    "policy_results": {      "allowed": true,      "granting_policies": [        {          "name": "root",          "namespace_id": "root",          "type": "acl"        }      ]    },    "token_type": "service",    "token_issue_time": "2023-06-21T12:52:53-04:00"  },  "request": {    "id": "8804bf89-b0ab-8368-dc06-52a5f3fbb702",    "client_id": "0DHqvq2D77kL2/JTPSZkTMJbkFVmUu0TzMi0jiXcFy8=",    "operation": "update",    "mount_point": "sys/",    "mount_type": "system",    "mount_accessor": "system_18f60a0c",    "mount_running_version": "v1.14.0+builtin.vault",    "mount_class": "secret",    "client_token": "hmac-sha256:2538d1fb96fe45b811af0c6db2c2cbe78e4f10d1d3ad0c8eb6f7dd36828587bb",    "client_token_accessor": "hmac-sha256:defd1a0c008c6438b48d733578ca7f50f978c2db2750a1e508b4d1d8674d670d",    "namespace": {      "id": "root"    },    "path": "sys/mounts/transit",    "data": {      "config": {        "default_lease_ttl": "hmac-sha256:8ed0513cb3a7070d1f2a5ed643c1b0f455e887b66b82204085ca86c3661b26b4",        "force_no_cache": false,        "max_lease_ttl": "hmac-sha256:8ed0513cb3a7070d1f2a5ed643c1b0f455e887b66b82204085ca86c3661b26b4",        "options": null      },      "description": "hmac-sha256:5f9da19df55a1a3f82d640bbf552847a11da9bfd962ef63cbc4848e78bb36ad4",      "external_entropy_access": false,      "local": false,      "options": null,      "seal_wrap": false,      "type": "hmac-sha256:c27c9dc8f2e0a07cad1e12e722503879100baba3f859ec123ce22755a049ebf1"    },    "remote_address": "127.0.0.1",    "remote_port": 51188  },  "response": {    "mount_point": "sys/",    "mount_type": "system",    "mount_accessor": "system_18f60a0c",    "mount_running_plugin_version": "v1.14.0+builtin.vault",    "mount_class": "secret"  }}

The object describes the time, the authorized token, the request, and the response.

Display the request of the last logged object.

$ cat vault-audit.log | jq -s ".[-1].request"

The output displays an entry similar to this example.

{  "id": "8804bf89-b0ab-8368-dc06-52a5f3fbb702",  "client_id": "0DHqvq2D77kL2/JTPSZkTMJbkFVmUu0TzMi0jiXcFy8=",  "operation": "update",  "mount_point": "sys/",  "mount_type": "system",  "mount_accessor": "system_18f60a0c",  "mount_running_version": "v1.14.0+builtin.vault",  "mount_class": "secret",  "client_token": "hmac-sha256:2538d1fb96fe45b811af0c6db2c2cbe78e4f10d1d3ad0c8eb6f7dd36828587bb",  "client_token_accessor": "hmac-sha256:defd1a0c008c6438b48d733578ca7f50f978c2db2750a1e508b4d1d8674d670d",  "namespace": {    "id": "root"  },  "path": "sys/mounts/transit",  "data": {    "config": {      "default_lease_ttl": "hmac-sha256:8ed0513cb3a7070d1f2a5ed643c1b0f455e887b66b82204085ca86c3661b26b4",      "force_no_cache": false,      "max_lease_ttl": "hmac-sha256:8ed0513cb3a7070d1f2a5ed643c1b0f455e887b66b82204085ca86c3661b26b4",      "options": null    },    "description": "hmac-sha256:5f9da19df55a1a3f82d640bbf552847a11da9bfd962ef63cbc4848e78bb36ad4",    "external_entropy_access": false,    "local": false,    "options": null,    "seal_wrap": false,    "type": "hmac-sha256:c27c9dc8f2e0a07cad1e12e722503879100baba3f859ec123ce22755a049ebf1"  },  "remote_address": "127.0.0.1",  "remote_port": 51188}

The request describes the operation that was performed on the path.

Display the request's path and the request's operation.

$ cat vault-audit.log | jq -s ".[-1].request.path,.[-1].request.operation""sys/mounts/transit""update"

The output displays that this operation performed an update on the path sys/mounts/transit.

ACL Policy workflow

You can use the information you gained from the audit device log to define a policy that grants the capability to update at the path sys/mounts/transit. That policy is expressed in the following example.

path "sys/mounts/transit" {  capabilities = ["update"]}

You can define your policy as files and place them under version control as part of a Vault configuration codification strategy.

Write the above policy example to the file transit-updates.hcl.

$ cat > transit-updates.hcl << EOFpath "sys/mounts/transit" {  capabilities = ["update"]}EOF

Write the policy into your Vault server from the file, and name it transit-updates.

$ vault policy write transit-updates transit-updates.hcl

As part of the policy workflow, you can update existing ACL policies by changing their file content, and simply writing the policy again with the same name. The existing policy will be overwritten, but existing tokens with the policy attached will use the previous policy until a new token is requested.

Additional ACL policy challenges

Use the techniques you've learned here to complete these optional challenges.

Create an encryption key

Create a encryption key named user-data.

$ vault write -f transit/keys/user-dataSuccess! Data written to: transit/keys/user-data

Display the request path and the request operation of the last logged object.

$ cat vault-audit.log | jq -s ".[-1].request.path,.[-1].request.operation""transit/keys/user-data""sys/mounts/transit""update"

Challenge

Define a policy that grants the capability to update at path transit/keys/user-data.

path "transit/keys/user-data" {  capabilities = ["update"]}

Encrypt plaintext with the key

The user-data key is capable of encrypting base64 encoded plaintext and returns the ciphertext.

Encrypt user sensitive data with the user-data key.

$ vault write transit/encrypt/user-data plaintext=$(base64 <<< "sensitive user data")Key            Value---            -----ciphertext     vault:v1:k1nJTMMmYM2nsYitgHRml6ExwWySGtkqgaOz3JI6kV+IT9995O26qOY5Uec/dn+lkey_version    1

Display the request path and the request operation of the last logged object.

$ cat vault-audit.log | jq -s ".[-1].request.path,.[-1].request.operation""transit/encrypt/user-data""transit/encrypt/user-data""create"

Challenge

Define a policy that grants the capability to update at path transit/encrypt/user-data.

path "transit/encrypt/user-data" {  capabilities = ["update"]}

Decrypt ciphertext with the key

The same key is used to decrypt the ciphertext and return the base64 encoded plaintext.

Create a variable named USER_SECRET_DATA that stores the cipher text generated.

$ USER_SECRET_DATA=$(vault write -format=json \   transit/encrypt/user-data plaintext=$(base64 <<< "sensitive user data") \   | jq -r ".data.ciphertext")

Decrypt the user sensitive data with the user-data key.

$ vault write transit/decrypt/user-data ciphertext=$USER_SECRET_DATAKey          Value---          -----plaintext    c2Vuc2l0aXZlIHVzZXIgZGF0YQo=

Display the request path and the request operation of the last logged object.

$ cat vault-audit.log | jq -s ".[-1].request.path,.[-1].request.operation""transit/decrypt/user-data""update"

Challenge

Define a policy that grants the capability to update at path transit/decrypt/user-data.

path "transit/decrypt/user-data" {  capabilities = ["update"]}

Enable a key/value secrets engine

For the next challenges, enable an instance of the Key/Value Secrets Engine version 2 at the path kv-v2.

$ vault secrets enable -version=2 -path=kv-v2 kv

Display the request path and the request operation of the last logged object.

$ cat vault-audit.log | jq -s ".[-1].request.path,.[-1].request.operation""sys/mounts/kv-v2""update"

Write an example secret in the newly enabled secrets engine.

$ vault kv put kv-v2/my-secret vault=awesome==== Secret Path ====kv-v2/data/my-secret ======= Metadata =======Key                Value---                -----created_time       2022-09-01T14:43:41.273398Zcustom_metadata    <nil>deletion_time      n/adestroyed          falseversion            1

Display the request path and the request operation of the last logged object.

$ cat vault-audit.log | jq -s ".[-1].request.path,.[-1].request.operation""kv-v2/data/my-secret""create"

Challenge

Write a policy to Vault that grants the capability to list and read at the paths kv-v2/data/my-secret and kv-v2/metadata/my-secret.

vault policy write my-secret-read-list - <<EOFpath "kv-v2/+/my-secret" {  capabilities = ["read", "list"]}EOF

$ export MY_SECRET_TOKEN="$(vault token create -field=token -policy=my-secret-read-list)"

$ VAULT_TOKEN=$MY_SECRET_TOKEN vault kv get kv-v2/my-secret==== Secret Path ====kv-v2/data/my-secret ======= Metadata =======Key                Value---                -----created_time       2022-09-01T14:43:41.273398Zcustom_metadata    <nil>deletion_time      n/adestroyed          falseversion            1 ==== Data ====Key      Value---      -----vault    awesome

Clean up

Here are the steps you need to completely clean up the scenario content from your local environment.

1. Stop the Vault server process.

   $ pkill vault

2. Remove the example policy file.

   $ rm -f transit-updates.hcl

3. Remove the audit device log file.

   $ rm -f vault-audit.log

4. Unset environment variables.

   $ unset VAULT_ADDR VAULT_TOKEN MY_SECRET_TOKEN

Next steps

In this tutorial, you performed operations and then queried the audit device log to discover the path and action involved in the operations. Learn more about querying the Audit Log, read the Querying Audit Device Logs tutorial.

To define policies requires understanding the paths and capabilities of each authentication method and secrets engine. Learn additional approaches in the Write a Policy from API documentation tutorial.

Help and reference

• Filter concepts documentation
• filter audit entries documentation
• Audit filter API documentation
• Audit filter CLI documentation