News HashiCorp to join IBM to accelerate multi-cloud automation. Read the blog →
Boundary
Host management

Dynamic host catalogs on Azure

  • 20min
  • |
  • Boundary

Dynamic updates to host catalogs is an important differentiator for Boundary from traditional access methods that rely on manual target configuration, and enables even tighter integrations with major cloud providers for seamlessly onboarding cloud tenant identities, roles, and targets.

Boundary 0.7 introduces a key Boundary component by enabling automated discovery of target hosts and services. Starting with support for Azure and AWS, dynamic connections to any service registry ensures that hosts and host catalogs are consistently up-to-date. This critical workflow offers access-on-demand and eliminates the need to manually configure targets for dynamic, cloud-based infrastructure.

This tutorial demonstrates configuring a dynamic host catalog using Microsoft Azure.

Dynamic hosts catalog overview

  1. Get setup
  2. Dynamic host catalogs background
  3. Set up cloud hosts
  4. Build a host catalog
  5. Verify catalog membership

Prerequisites

  • A Boundary binary greater than 0.7.1 in your PATH.

  • This tutorial assumes you can connect to an HCP Boundary cluster or launch Boundary in dev mode.

  • A Microsoft Azure account. This tutorial requires the creation of new cloud resources and will incur costs associated with the deployment and management of these resources.

  • Installing the Azure CLI provides an optional workflow for this tutorial.The executable must be available within your PATH.

  • Installing Terraform 0.14.9 or greater provides an optional workflow for this tutorial. The binary must be available in your PATH.

Get setup

In this tutorial, you will test dynamic host catalog integrations using HCP Boundary or by running a Boundary controller locally using dev mode.

The HCP Quickstart tutorials provide an overview of getting started with an HCP Boundary cluster.

If you have an HCP Boundary cluster deployed, the Access HCP Boundary tutorial provides an overview of configuring your local machine to authenticate with your HCP cluster.

This tutorial provides both CLI and UI workflows for setting up OIDC authentication.

To proceed with the CLI workflow:

  1. Log in to the Boundary web UI and copy your org ID by clicking the copy icon.

    Open a terminal session and set a environment variable for the org ID.

    $ export ORG_ID=<org-id>

  2. In the Boundary web UI, click Projects and copy your project ID by clicking the copy icon.

    Open a terminal session and set a environment variable for the project ID.

    $ export PROJECT_ID=<project-id>

  3. In the Boundary web UI, click Orgs in the left navigation menu to return to the global scope, and then click Auth Methods.

    Click the copy icon for the Password auth method.

    In your terminal set an environment variable named BOUNDARY_AUTH_METHOD_ID to the copied ID.

    $ export BOUNDARY_AUTH_METHOD_ID=<auth-id>

    Close the Boundary web UI.

  4. Return to the HCP web Portal Boundary page, then click the copy icon for the Cluster URL in the Getting started with Boundary section.

    In your terminal, set the BOUNDARY_ADDR environment variable to the copied URL.

    $ export BOUNDARY_ADDR=<actual-boundary-address>

  5. Log in with the administrator credentials you created when you deployed the HCP Boundary instance. Enter your password at the Please enter the password (it will be hidden): prompt.

    $ boundary authenticatePlease enter the login name (it will be hidden):Please enter the password (it will be hidden):Authentication information:  Account ID:      acctpw_VOeNSFX8pQ  Auth Method ID:  ampw_wxzojlKJLN  Expiration Time: Mon, 13 Feb 2023 12:35:32 MST  User ID:         u_1vUkf5fPs9The token was successfully stored in the chosen keyring and is not displayed here.

You are now logged into your HCP Boundary instance's Global scope via the CLI. This is the default scope for all new Boundary clusters.

Dynamic host catalogs background

In a cloud operating model infrastructure resources are highly dynamic and ephemeral. Boundary lacks an on-target agent or daemon, and as a result cannot recognize when a host service migrates or is redeployed. Instead, Boundary relies on an external entity, such as manual configuration by an administrator or IaC application (such as Terraform), to ensure host definitions route to the appropriate network location. This is a pattern followed by many other secure access solutions.

Dynamic host catalog plugins provide an alternative mechanism for automating the discovery and configuration of Boundary hosts and targets by delegating the host registry and their connection information to a cloud infrastructure provider. Administrators provide credentials to the catalog provider and a set of tag-based rules for discovery resources in the catalog. For example, "this catalog contains all VMs in Azure's useast region within the marketing subscription". This model differs from other that rely on either IaC target discovery or agent-based target discovery.

To accommodate this model, Boundary 0.7.0 introduces plugin support via Go-Plugin for expanding the dynamic host catalog ecosystem. While this initial release of Boundary plug-ins are limited to dynamic host catalogs, plugins enable a future ecosystem of partner and community contributed integrations across each step in the Boundary access workflow.

Host tag filtering

To successfully maintain a dynamic host catalog, hosts should be tagged in a logical way that enables sorting into the appropriate host sets identifiable by filters.

For examples, this tutorial configures hosts on Azure using the following tags:

Hosts with these tags will be sorted into any host catalogs and host sets configured using these filtering attributes.

Set up cloud hosts

Warning

This tutorial deploys cloud machines to test host catalog plugin configuration. You are responsible for any costs incurred by following the steps in this tutorial. Recommendations for destroying the associated cloud resources are detailed in the Cleanup and teardown section.

A Microsoft Azure account and sample application are required to setup the Boundary Azure host plugin. If you don't have an account, sign up for Azure. A free account is suitable for the steps outlined in this tutorial, but please note that you are responsible for any charges incurred by following the steps in this tutorial.

This tutorial enables configuration of the test hosts using the Azure CLI, Terraform, or the Azure Portal UI.

There are four tasks necessary to set up hosts using the Azure CLI.

  1. Create an Application in Azure Active Directory with an associated Service Principal
  2. Create a role assignment granting the new application Reader permissions to your subscription
  3. Generate a Client Secret for the Azure Active Directory Application, enabling Boundary to authenticate
  4. Tag the hosts, so they can be filtered into catalogs by Boundary.

Register a new Azure AD application

First, log in to Azure. You will need to know the Tenant domain name. If you need to add a new Tenant to your Subscription, create a new Tenant and then copy its domain.

$ az login --tenant boundary.onmicrosoft.comThe default web browser has been opened at https://login.microsoftonline.com/0e3e2e88-8caf-41ca-b4da-q3b33b6c43tz/oauth2/v2.0/authorize. Please continue the login in the web browser. If no web browser is available or if the web browser fails to open, use device code flow with `az login --use-device-code`.

A browser window will open prompting you to login with your Azure credentials. Upon successful authentication, you should receive a confirmation message that You have logged into Microsoft Azure!, and you can then close the browser tab.

Returning to the terminal session, you should see updated output containing the tenantID.

Example output:

$ az login --tenant boundaryoidc.onmicrosoft.comThe default web browser has been opened at https://login.microsoftonline.com/0e3e2e88-8caf-41ca-b4da-q3b33b6c43tz/oauth2/v2.0/authorize. Please continue the login in the web browser. If no web browser is available or if the web browser fails to open, use device code flow with `az login --use-device-code`.[  {    "cloudName": "AzureCloud",    "homeTenantId": "0e3e2e88-8caf-41ca-b4da-q3b33b6c43tz",    "id": "c9ed8610-47a3-4107-a2b2-a322114dkd78",    "isDefault": true,    "managedByTenants": [],    "name": "Default Subscription",    "state": "Enabled",    "tenantId": "0e3e2e88-8caf-41ca-b4da-q3b33b6c43tz",    "user": {      "name": "admin@boundary.com",      "type": "user"    }  }]

Copy the Tenant ID (0e3e2e88-8caf-41ca-b4da-q3b33b6c43tz in this example).

Additionally, copy the id field, which contains the Subscription ID (c9ed8610-47a3-4107-a2b2-a322114dkd78 in this example). These values will be used in the next step and when configuring Boundary.

Note

Subscription and Tenant IDs can also be gathered using the az account list command.

Export the Tenant ID and Subscription ID as environment variables for use in the following steps.

$ export ARM_TENANT_ID=<Tenant ID> \  export ARM_SUBSCRIPTION_ID=<Subscription ID>

The Azure AD application needs a service principle that enables role-based access control (RBAC) to other resources, in this case the ability to read subscription information. This will allow Boundary to filter for hosts that match its filter criteria later on. This type of application is created using the az ad sp create-for-rbac command.

$ az ad sp create-for-rbac --name "Boundary Dynamic Hosts Test" --role ReaderCreating 'Reader' role assignment under scope '/subscriptions/c9ed8610-47a3-4107-a2b2-a322114dkd78'The output includes credentials that you must protect. Be sure that you do not include these credentials in your code or check the credentials into your source control. For more information, see https://aka.ms/azadsp-cli'name' property in the output is deprecated and will be removed in the future. Use 'appId' instead.{  "appId": "c2f9b423-9aa1-4ed4-809d-0af5b28397f9",  "displayName": "Boundary Dynamic Hosts Test",  "name": "c2f9b423-9aa1-4ed4-809d-0af5b28397f9",  "password": "fdrAfz72~FSw8ZHZG1y2Nd83iBG~k-Y_Io",  "tenant": "0e3e2e88-8caf-41ca-b4da-q3b33b6c43tz"}

Copy the appId from the output (c2f9b423-9aa1-4ed4-809d-0af5b28397f9 in this example). This is the Application or Client ID. Export it as an environment variable.

$ export ARM_CLIENT_ID=<Client ID>

Now, create a role assignment for the new application granting it the Reader role within your subscription. Supply the appID and Subscription ID using the environment variables exported earlier.

$ az role assignment create \  --assignee $ARM_CLIENT_ID \  --role Reader \  --scope /subscriptions/$ARM_SUBSCRIPTION_ID

Example output:

$ az role assignment create \  --assignee $ARM_CLIENT_ID \  --role Reader \  --scope /subscriptions/$ARM_SUBSCRIPTION_ID{  "canDelegate": null,  "condition": null,  "conditionVersion": null,  "description": null,  "id": "/subscriptions/0e3e2e88-8caf-41ca-b4da-q3b33b6c43tz/providers/Microsoft.Authorization/roleAssignments/e8cf2a65-fb1b-4c13-97be-7b9bd6e134fd",  "name": "e8cf2a65-fb1b-4c13-97be-7b9bd6e134fd",  "principalId": "23b2681c-8ba0-4144-9304-7e22f1bf94d1",  "principalName": "c2f9b423-9aa1-4ed4-809d-0af5b28397f9",  "principalType": "ServicePrincipal",  "roleDefinitionId": "/subscriptions/0e3e2e88-8caf-41ca-b4da-q3b33b6c43tz/providers/Microsoft.Authorization/roleDefinitions/acdd72a7-3385-48ef-bd42-f606fba81ae7",  "roleDefinitionName": "Reader",  "scope": "/subscriptions/0e3e2e88-8caf-41ca-b4da-q3b33b6c43tz",  "type": "Microsoft.Authorization/roleAssignments"}

Generate a client secret for the application

With the application created and Reader role assigned, generate an application client secret for Boundary to use for authentication.

The az add app credential reset command is used to append or overwrite an application's password or certificate credentials.

$ az ad app credential reset --append \  --id $ARM_CLIENT_ID \  --credential-description "boundary-secret"

Example output:

$ az ad app credential reset --append \  --id $ARM_CLIENT_ID \  --credential-description "boundary-secret"The output includes credentials that you must protect. Be sure that you do not include these credentials in your code or check the credentials into your source control. For more information, see https://aka.ms/azadsp-cli{  "appId": "c2f9b423-9aa1-4ed4-809d-0af5b28397f9",  "name": "c2f9b423-9aa1-4ed4-809d-0af5b28397f9",  "password": "FJY4nDv7P_bKgQBrY-k2MWwrfm-mj130Bq",  "tenant": "0e3e2e88-8caf-41ca-b4da-q3b33b6c43tz"}

Export the password returned in the output (FJY4nDv7P_bKgQBrY-k2MWwrfm-mj130Bq in this example) as an environment variable for use when configuring Boundary later on. Store this value somewhere safe, because it cannot be accessed again.

$ export ARM_CLIENT_SECRET=FJY4nDv7P_bKgQBrY-k2MWwrfm-mj130Bq

Create hosts

With the Azure AD application in place, hosts can be deployed to test the dynamic host catalog integration.

To easily deploy a set of pre-configured hosts, a template file is available for use with the Azure Resource Manager (ARM).

Clone down the sample template repository.

$ git clone https://github.com/hashicorp-education/learn-boundary-cloud-host-catalogs.git

Navigate into the azure directory.

$ cd learn-boundary-cloud-host-catalogs/azure/

Next, deploy a new resource group using the provided azure-dynamic-hosts template.

First, create a new resource group within your subscription that will contain the hosts. This tutorial uses the eastus location, but you can use any location you want. Check the available locations with az account list-locations.

$ az group create \  --location eastus \  --name boundary-dynamic-hosts_group \  --subscription $ARM_SUBSCRIPTION_ID

Example output:

$ az group create \  --location eastus \  --name boundary-dynamic-hosts_group \  --subscription $ARM_SUBSCRIPTION_ID{  "id": "/subscriptions/0e3e2e88-8caf-41ca-b4da-q3b33b6c43tz/resourceGroups/boundary-dynamic-hosts_group",  "location": "eastus",  "managedBy": null,  "name": "boundary-dynamic-hosts_group",  "properties": {    "provisioningState": "Succeeded"  },  "tags": null,  "type": "Microsoft.Resources/resourceGroups"}

With the new resource group created, the host template file can be deployed.

$ az deployment group create \  --resource-group boundary-dynamic-hosts_group \  --template-file azure-dynamic-hosts.template

The output will prompt Please provide securestring value for 'adminPasswordOrKey' (? for help):

This is a required password or SSH key to access the VMs.

Note

This tutorial does not log into the associated VMs, but it is best practice to copy the password or retain the private key for the duration of the tutorial.

After entering a password, hit Enter to proceed.

The deployment will take a few minutes. When complete, a large amount of output will be displayed containing details for the newly created hosts and associated resources.

Check the template deployment completed successfully by querying the deployment group. List all resources in the boundary-dynamic-hosts_group using the --query option on the az command.

$ az deployment group list --resource-group boundary-dynamic-hosts_group --query "[].properties.dependencies[*].resourceName[]"[  "boundary-vm-test_group-vnet/boundary-subnet",  "boundary-vm-3-production",  "boundary-vm-4-production",  "boundary-vm-NetInt-1",  "boundary-vm-NetInt-2",  "boundary-vm-NetInt-3",  "boundary-vm-NetInt-4",  "boundary-vm1-dev",  "boundary-vm2-dev"]

Gather plugin details

To use the Boundary Azure hosts plugin, the following details must be gathered from Azure:

  • ARM Tenant (Directory) ID
  • ARM Subscription ID
  • ARM Client (Application) ID
  • ARM Client Secret

These values should be available as environment variables within the terminal session, or copied to a safe location for use when setting up Boundary.

Host catalog plugins

For Boundary, the process for creating a dynamic host catalog has two steps:

  1. Create a plugin-type host catalog
  2. Create a host set that defines membership using filters

A plugin-type host catalog can be created using some cloud provider resource details, and the host set is then defined using a filter that selects hosts for membership based on the tags defined when setting up the hosts.

Host set filter expressions are defined by the plugin provider, in this case Azure. The Azure plugin uses simple filter queries to specify tags associated with hosts based on tagName and tagValue.

For example, a host set filter that selects all hosts tagged with "service_type": "database" is written as:

Resources within Azure can generally be filtered by tag names and values, and additional operators such as ne (not equals) can be utilized.

To learn more about Azure filters for listing resources, visit the $filter section in the Azure API docs.

Build a host catalog

With the cloud provider details gathered, a plugin host catalog can now be created that will contain the respective host sets for the database and application filters.

Create a host catalog plugin

Check that the ARM parameter environment variable values were set correctly in your session.

$ echo $ARM_TENANT_ID; echo $ARM_SUBSCRIPTION_ID; echo $ARM_CLIENT_ID; echo $ARM_CLIENT_SECRET0e3e2e88-8caf-41ca-b4da-q3b33b6c43tzc9ed8610-47a3-4107-a2b2-a322114dkd78c2f9b423-9aa1-4ed4-809d-0af5b28397f9mG27Q~Kr842OROnJUwWtG0ygbfG5RKxe.DeZ-

If necessary, authenticate to Boundary as the admin user.

$ boundary authenticatePlease enter the login name (it will be hidden):Please enter the password (it will be hidden):Authentication information:  Account ID:      acctpw_VOeNSFX8pQ  Auth Method ID:  ampw_wxzojlKJLN  Expiration Time: Mon, 13 Feb 2023 12:35:32 MST  User ID:         u_1vUkf5fPs9The token was successfully stored in the chosen keyring and is not displayed here.

Next, create a new plugin-type host catalog with a -plugin-name of azure, providing the ARM Tenant ID, Subscription ID, and Client ID using the -attr flag, and the ARM Client Secret using the -secret flag. These values should map to the environment variables defined above.

$ boundary host-catalogs create plugin \  -scope-id $PROJECT_ID \  -plugin-name azure \  -attr disable_credential_rotation=true \  -attr tenant_id=env://ARM_TENANT_ID \  -attr subscription_id=env://ARM_SUBSCRIPTION_ID \  -attr client_id=env://ARM_CLIENT_ID \  -secret secret_value=env://ARM_CLIENT_SECRET

Command flags:

  • -plugin-name: This corresponds to the host catalog plugin's name, such as azure or aws
  • disable_credential_rotation: This tutorial utilizes a static secret by setting this value to true
  • tenant_id: The ARM Tenant(Directory) ID, supplied as an environment variable
  • subscription_id: The ARM Subscription ID, supplied as an environment variable
  • client_id: The ARM Client (Application) ID, supplied as an environment variable
  • secret_value: The ARM Client Secret, supplied as an environment variable

Note

Although credentials are stored encrypted within Boundary, by default this plugin attempts to rotate credentials supplied through the secrets object during a create or update call to the host catalog resource. The given credentials will be used to create a new credential, and then the given credential will be revoked. In this way, after rotation, only Boundary knows the client secret in use by this plugin. Credential rotation will be generally available in a future release of Boundary.

Sample output:

$ boundary host-catalogs create plugin \  -scope-id $PROJECT_ID \  -plugin-name azure \  -attr disable_credential_rotation=true \  -attr tenant_id=env://ARM_TENANT_ID \  -attr subscription_id=env://ARM_SUBSCRIPTION_ID \  -attr client_id=env://ARM_CLIENT_ID \  -secret secret_value=env://ARM_CLIENT_SECRET  Host Catalog information:  Created Time:                  Mon, 13 Feb 2023 16:15:47 MST  ID:                            hcplg_zZfpE9UHlz  Plugin ID:                     pl_z7Edh0X67z  Secrets HMAC:                  5wVcSKk1YhkqRp41zPjMP88SkiJLv8dcJcHs1UNYCmtc  Type:                          plugin  Updated Time:                  Mon, 13 Feb 2023 16:15:47 MST  Version:                       1   Scope:    ID:                          p_1234567890    Name:                        Generated project scope    Parent Scope ID:             o_1234567890    Type:                        project   Plugin:    ID:                          pl_p1J8mSiPuI    Name:                        azure   Attributes:    client_id:                   c2f9b423-9aa1-4ed4-809d-0af5b28397f9    disable_credential_rotation: true    subscription_id:             c9ed8610-47a3-4107-a2b2-a322114dkd78    tenant_id:                   0e3e2e88-8caf-41ca-b4da-q3b33b6c43tz   Authorized Actions:    no-op    read    update    delete   Authorized Actions on Host Catalog's Collections:    host-sets:      create      list    hosts:      list

Copy the host catalog ID from the output (hcplg_zZfpE9UHlz in this example) and store it in the HOST_CATALOG_ID environment variable.

$ export HOST_CATALOG_ID=hcplg_zZfpE9UHlz

Create the host sets

With the dynamic host catalog created, host sets can now be defined that correspond to the service-type and application tags added to the hosts.

Recall the three host sets we wish to create:

  • All hosts with a service-type tag of database
  • All hosts with an application tag of dev
  • All hosts with an application tag of production

The respective host set filters can be constructed as:

Create the first plugin host set containing hosts tagged with a service-type of database, supplying the host catalog ID copied above and the needed filter using the -attr flag.

$ boundary host-sets create plugin \  -name database \  -host-catalog-id $HOST_CATALOG_ID \  -attr filter="tagName eq 'service-type' and tagValue eq 'database'"

Sample output:

$ boundary host-sets create plugin \  -name database \  -host-catalog-id $HOST_CATALOG_ID \  -attr filter="tagName eq 'service-type' and tagValue eq 'database'" Host Set information:  Created Time:        Mon, 13 Feb 2023 16:35:35 MST  Host Catalog ID:     hcplg_zZfpE9UHlz  ID:                  hsplg_IiQgeZYxKJ  Name:                database  Type:                plugin  Updated Time:        Mon, 13 Feb 2023 16:35:35 MST  Version:             1   Scope:    ID:                p_1234567890    Name:              Generated project scope    Parent Scope ID:   o_1234567890    Type:              project   Plugin:    ID:                pl_z7Edh0X67z    Name:              azure   Attributes:    filter:            tagName eq 'service-type' and tagValue eq 'database'   Authorized Actions:    no-op    read    update    delete

Copy the database host set ID from the output (hsplg_IiQgeZYxKJ in this example) and store it in the DATABASE_HOST_SET_ID environment variable.

$ export DATABASE_HOST_SET_ID=hsplg_IiQgeZYxKJ

Wait a moment, then list all available hosts within the azure host catalog, which contains the newly created database host set.

Note

It may take up to five minutes for the host catalog to sync with the cloud provider.

$ boundary hosts list -host-catalog-id $HOST_CATALOG_ID Host information:  ID:                    hplg_35sD4tOrCP    External ID:    /subscriptions/c9ed8610-47a3-4107-a2b2-a322114dkd78/resourceGroups/boundary-dynamic-hosts_group/providers/Microsoft.Compute/virtualMachines/boundary-vm1-dev    Version:             1    Type:                plugin    Authorized Actions:      no-op      read   ID:                    hplg_cxiWeokxkJ    External ID:    /subscriptions/c9ed8610-47a3-4107-a2b2-a322114dkd78/resourceGroups/boundary-dynamic-hosts_group/providers/Microsoft.Compute/virtualMachines/boundary-vm-4-production    Version:             1    Type:                plugin    Authorized Actions:      no-op      read   ID:                    hplg_s5pQuIjAiP    External ID:    /subscriptions/c9ed8610-47a3-4107-a2b2-a322114dkd78/resourceGroups/boundary-dynamic-hosts_group/providers/Microsoft.Compute/virtualMachines/boundary-vm2-dev    Version:             1    Type:                plugin    Authorized Actions:      no-op      read   ID:                    hplg_xSU4Fs5ZKc    External ID:    /subscriptions/c9ed8610-47a3-4107-a2b2-a322114dkd78/resourceGroups/boundary-dynamic-hosts_group/providers/Microsoft.Compute/virtualMachines/boundary-vm-3-production    Version:             1    Type:                plugin    Authorized Actions:      no-op      read

Troubleshooting

If the boundary hosts list command returns No hosts found, expand the accordion below to check your work.

If the host catalog is misconfigured, hosts will not be discoverable by Boundary. There are two issues to check:

  1. The host catalog ARM ID's supplied as environment variables are incorrect
  2. The Azure subscription role of Reader has not been applied to the application

Note

Depending on the type of configuration issue, you will need to wait approximately 5 - 10 minutes for the existing host catalog or host sets to sync with the provider and refresh their values. If you do not want to wait a new host catalog and host set can be created from scratch, but these will also take several minutes to sync upon creation.

If incorrect, update the managed group filter. This process can also be used to update the managed group filter criteria in the future for any existing managed groups.

First, check the environment variables defined when creating a host catalog plugin. Ensure these are the correct values gathered when setting up the cloud hosts in the Azure portal.

If these are incorrectly defined, set the environment variables again, and update the host catalog:

$ boundary host-catalogs update plugin \  -id $HOST_CATALOG_ID \  -plugin-name azure \  -attr disable_credential_rotation=true \  -attr tenant_id=env://ARM_TENANT_ID \  -attr subscription_id=env://ARM_SUBSCRIPTION_ID \  -attr client_id=env://ARM_CLIENT_ID \  -secret secret_value=env://ARM_CLIENT_SECRET

Second, check is the Reader role assignment to the Azure AD application's subscription. If incorrect permissions are assigned or the wrong application is selected, Boundary will not be able to view the hosts from the application granting it access.

Review the steps for granting the application access to manage resources in your Azure Subscription.

After correcting the role, give Boundary up to five minutes to refresh the connection to Azure, and list the available hosts again.

Now create a host set that correspond to the application tag of dev.

$ boundary host-sets create plugin \  -name dev \  -host-catalog-id $HOST_CATALOG_ID \  -attr filter="tagName eq 'application' and tagValue eq 'dev'"

Sample output:

$ boundary host-sets create plugin \  -name dev \  -host-catalog-id $HOST_CATALOG_ID \  -attr filter="tagName eq 'application' and tagValue eq 'dev'" Host Set information:  Created Time:        Mon, 13 Feb 2023 16:39:41 MST  Host Catalog ID:     hcplg_zZfpE9UHlz  ID:                  hsplg_sqcZJGNxHD  Name:                dev  Type:                plugin  Updated Time:        Mon, 13 Feb 2023 16:39:41 MST  Version:             1   Scope:    ID:                p_1234567890    Name:              Generated project scope    Parent Scope ID:   o_1234567890    Type:              project   Plugin:    ID:                pl_z7Edh0X67z    Name:              azure   Attributes:    filter:            tagName eq 'application' and tagValue eq 'dev'   Authorized Actions:    no-op    read    update    delete

Copy the dev host set ID from the output (hsplg_sqcZJGNxHD in this example) and store it in the DEV_HOST_SET_ID environment variable.

$ export DEV_HOST_SET_ID=hsplg_sqcZJGNxHD

Lastly, create a host set that correspond to the application tag of production.

$ boundary host-sets create plugin \  -name production \  -host-catalog-id $HOST_CATALOG_ID \  -attr filter="tagName eq 'application' and tagValue eq 'production'"

Sample output:

$ boundary host-sets create plugin \  -name production \  -host-catalog-id $HOST_CATALOG_ID \  -attr filter="tagName eq 'application' and tagValue eq 'production'" Host Set information:  Created Time:        Mon, 13 Feb 2023 16:40:29 MST  Host Catalog ID:     hcplg_zZfpE9UHlz  ID:                  hsplg_qUm6k52Tmu  Name:                production  Type:                plugin  Updated Time:        Mon, 13 Feb 2023 16:40:29 MST  Version:             1   Scope:    ID:                p_1234567890    Name:              Generated project scope    Parent Scope ID:   o_1234567890    Type:              project   Plugin:    ID:                pl_z7Edh0X67z    Name:              azure   Attributes:    filter:            tagName eq 'application' and tagValue eq 'production'   Authorized Actions:    no-op    read    update    delete

Copy the production host set ID from the output (hsplg_qUm6k52Tmu in this example) and store it in the PRODUCTION_HOST_SET_ID environment variable.

$ export PRODUCTION_HOST_SET_ID=hsplg_qUm6k52Tmu

Verify catalog membership

With the database, dev, and prod host sets defined within the azure host catalog, the next step is to verify that the four VM hosts listed as members of the catalog are dynamically included in the correct host sets.

Host membership can be verified by reading the host set details and verifying its membership IDs.

First, verify that the database host set contains all four members of the azure host catalog.

Perform a read on the host set named database to view its members.

$ boundary host-sets read -id $DATABASE_HOST_SET_ID Host Set information:  Created Time:        Mon, 13 Feb 2023 16:35:35 MST  Host Catalog ID:     hcplg_zZfpE9UHlz  ID:                  hsplg_IiQgeZYxKJ  Name:                database  Type:                plugin  Updated Time:        Mon, 13 Feb 2023 16:52:00 MST  Version:             3   Scope:    ID:                p_1234567890    Name:              Generated project scope    Parent Scope ID:   o_1234567890    Type:              project   Plugin:    ID:                pl_z7Edh0X67z    Name:              azure   Attributes:    filter:            tagName eq 'service-type' and tagValue eq 'database'   Authorized Actions:    no-op    read    update    delete   Host IDs:    hplg_35sD4tOrCP    hplg_cxiWeokxkJ    hplg_s5pQuIjAiP    hplg_xSU4Fs5ZKc

If the Host IDs section is missing, expand the troubleshooting accordion to diagnose what could be wrong.

If the host catalog is misconfigured, hosts will not be discoverable by Boundary.

At this point in the tutorial hosts are contained within the host catalog, but not appearing in one or more host sets. This implies that the host set itself is misconfigured.

Above, you performed a read on the database host set. Check the Attributes section, and verify it matches the correctly defined filter:

  Attributes:    filter:            tagName eq 'service-type' and tagValue eq 'database'

If the tag is incorrectly assigned, perform an update on the affected host set to fix the filter:

$ boundary host-sets update plugin \  -id $DATABASE_HOST_SET_ID \  -name production \  -attr filter="tagName eq 'application' and tagValue eq 'production'"

After updating the filter, Boundary will automatically refresh the host set.

Note

Depending on the type of configuration issue, you will need to wait approximately 5 - 10 minutes for the existing host catalog or host sets to sync with the provider and refresh their values. If you do not want to wait a new host catalog and host set can be created from scratch, but these will also take several minutes to sync upon creation.

Check that the updated filter is working by performing another read on the database host set.

$ boundary host-sets read -id $DATABASE_HOST_SET_ID

If the dev or production host sets are affected by incorrect filters, follow the same procedure to update their filters accordingly.

Next, read the dev host set details. Verify the Host IDs are the correctly tagged hosts from the cloud provider.

$ boundary host-sets read -id $DEV_HOST_SET_ID Host Set information:  Created Time:        Mon, 13 Feb 2023 16:39:41 MST  Host Catalog ID:     hcplg_zZfpE9UHlz  ID:                  hsplg_sqcZJGNxHD  Name:                dev  Type:                plugin  Updated Time:        Mon, 13 Feb 2023 16:39:56 MST  Version:             2   Scope:    ID:                p_1234567890    Name:              Generated project scope    Parent Scope ID:   o_1234567890    Type:              project   Plugin:    ID:                pl_z7Edh0X67z    Name:              azure   Attributes:    filter:            tagName eq 'application' and tagValue eq 'dev'   Authorized Actions:    no-op    read    update    delete   Host IDs:    hplg_35sD4tOrCP    hplg_s5pQuIjAiP

Notice the Host IDs section of the output, which returns the two dev VMs configured in Azure.

Next, read the production host set and verify its Host IDs.

$ boundary host-sets read -id $PRODUCTION_HOST_SET_ID Host Set information:  Created Time:        Mon, 13 Feb 2023 16:40:29 MST  Host Catalog ID:     hcplg_zZfpE9UHlz  ID:                  hsplg_qUm6k52Tmu  Name:                production  Type:                plugin  Updated Time:        Mon, 13 Feb 2023 16:40:55 MST  Version:             2   Scope:    ID:                p_1234567890    Name:              Generated project scope    Parent Scope ID:   o_1234567890    Type:              project   Plugin:    ID:                pl_z7Edh0X67z    Name:              azure   Attributes:    filter:            tagName eq 'application' and tagValue eq 'production'   Authorized Actions:    no-op    read    update    delete   Host IDs:    hplg_xSU4Fs5ZKc

Notice the Host IDs section of this output. Even though there are two production VMs, only one is listed in the host set.

To figure out what could be wrong, compare the members of the production host set to the members of the database host set. Remember, members of the production and dev host sets are a sub-set of the database host set.

$ boundary host-sets read -id $DATABASE_HOST_SET_ID Host Set information:  Created Time:        Mon, 13 Feb 2023 16:35:35 MST  Host Catalog ID:     hcplg_zZfpE9UHlz  ID:                  hsplg_IiQgeZYxKJ  Name:                database  Type:                plugin  Updated Time:        Mon, 13 Feb 2023 16:52:00 MST  Version:             3   Scope:    ID:                p_1234567890    Name:              Generated project scope    Parent Scope ID:   o_1234567890    Type:              project   Plugin:    ID:                pl_z7Edh0X67z    Name:              azure   Attributes:    filter:            tagName eq 'service-type' and tagValue eq 'database'   Authorized Actions:    no-op    read    update    delete   Host IDs:    hplg_35sD4tOrCP    hplg_cxiWeokxkJ    hplg_s5pQuIjAiP    hplg_xSU4Fs5ZKc

By comparing the Host IDs of the dev host catalog to the production catalog, notice that host hplg_cxiWeokxkJ is missing from the production host set, although it is contained within database.

Update the misconfigured host

Perform a read on the missing host.

$ boundary hosts read -id hplg_cxiWeokxkJ Host information:  Created Time:        Mon, 13 Feb 2023 16:35:58 MST  External ID:  /subscriptions/c9ed8610-47a3-4107-a2b2-a322114dkd78/resourceGroups/boundary-dynamic-hosts_group/providers/Microsoft.Compute/virtualMachines/boundary-vm-4-production  Host Catalog ID:     hcplg_zZfpE9UHlz  ID:                  hplg_cxiWeokxkJ  Type:                plugin  Updated Time:        Mon, 13 Feb 2023 16:35:58 MST  Version:             1   Scope:    ID:                p_1234567890    Name:              Generated project scope    Parent Scope ID:   o_1234567890    Type:              project   Plugin:    ID:                pl_z7Edh0X67z    Name:              azure   Authorized Actions:    no-op    read   IP Addresses:    10.1.0.4    20.124.98.203

The External ID field shows the name of the misconfigured host: boundary-vm-4-production (Scroll to the right in the output to find /virtualMachines/boundary-vm-4-production).

Note

Your configuration may show either boundary-vm-3-production or boundary-vm-4-production as the misconfigured VM. Continue with the host that is misconfigured.

Recall that host set membership is defined based on the VM tags.

List the details for the boundary-vm-4-production VM, and query for its tag values.

$ az vm show --resource-group boundary-dynamic-hosts_group --name boundary-vm-4-production --query 'tags'{  "application": "prod",  "service-type": "database"}

Notice that the application tag is misconfigured as prod, instead of production. An easy mistake to make!

Remember the filter defined for the production host set:

"tagName eq 'application' and tagValue eq 'production'"

The tagValue must equal production exactly to be included in this host set.

Note

It is possible to create a more generic filter that could include tagName values that can be selected by prefix, but this would require a restructure of the tagging schema for the VMs. To learn more about filters and resource tagging, check the Azure API docs.

Update the application tag to production.

$ az vm update --resource-group boundary-dynamic-hosts_group --name boundary-vm-4-production --set tags.application=production

The output will display the updated tags, but you can re-run az vm show to directly query for the tag values.

$ az vm show --resource-group boundary-dynamic-hosts_group --name boundary-vm-4-production --query 'tags'{  "application": "production",  "service-type": "database"}

Boundary will update the production host set automatically the next time it refreshes. This process could take up to ten minutes.

After waiting, read the production host set again and verify that its Host IDs contain the updated host as a member.

$ boundary host-sets read -id $PRODUCTION_HOST_SET_ID Host Set information:  Created Time:        Mon, 13 Feb 2023 16:40:29 MST  Host Catalog ID:     hcplg_zZfpE9UHlz  ID:                  hsplg_qUm6k52Tmu  Name:                production  Type:                plugin  Updated Time:        Mon, 13 Feb 2023 17:25:01 MST  Version:             6   Scope:    ID:                p_1234567890    Name:              Generated project scope    Parent Scope ID:   o_1234567890    Type:              project   Plugin:    ID:                pl_z7Edh0X67z    Name:              azure   Attributes:    filter:            tagName eq 'application' and tagValue eq 'production'   Authorized Actions:    no-op    read    update    delete   Host IDs:    hplg_plnxAhd2Vm    hplg_xSU4Fs5ZKc

Cleanup and teardown

  1. Destroy the boundary-dynamic-hosts_group resource group in Azure.

    Delete all resources using az group delete. Enter y when prompted to confirm the operation.

    $ az group delete --name boundary-dynamic-hosts_group  --subscription $ARM_SUBSCRIPTION_IDAre you sure you want to perform this operation? (y/n): \ Running ..

    This will take some time.

    Verify that the resource group has been destroyed by listing the available resource groups.

    $ az group list

  2. Delete the Boundary Dynamic Hosts Test sample application from Azure AD.

    Delete the Boundary Dynamic Hosts Test app using az ad app delete.

    $ az ad app delete --id $ARM_CLIENT_ID

    Check that the application has been destroyed by attempting to query its ID.

    $ az ad app show --id $ARM_CLIENT_IDResource 'c2f9b423-9aa1-4ed4-809d-0af5b28397f9' does not exist or one of its queried reference-property objects are not present.

    This operation also removes the client secrets and certificates created to access the application.

  3. Stop Boundary

    Log in to the HCP portal and delete the HCP Boundary instance.

Next steps

This tutorial demonstrated the steps to set up a dynamic host catalog using the Azure host plugin. You deployed and tagged hosts within Azure, configured a plugin-type host catalog within Boundary, and created three host sets that filtered for the hosts based on their tag values.

To learn more about integrating Boundary with cloud providers like AWS and Azure, check out the OIDC Authentication tutorial.

Previous

Dynamic host catalogs on AWS

 Next

Secure access to Azure SQL database