Admin Partitions with HCP Consul and Amazon Elastic Container Service
Enterprise Only
Consul admin partitions is a feature of Consul Enterprise. The enterprise license is provided to you when HCP Consul deploys into your account. This tutorial deploys billable resources to your HashiCorp Cloud and AWS accounts.
Consul admin partitions let organizations define administrative boundaries for services using Consul. This helps organizations manage Consul as a global installation, with services hosted across many teams and business units. Teams benefit by managing and customizing their Consul environment in context to their workloads, without creating impact to other teams, or other Consul environments.
The following diagram shows the potential of Admin partitions within a single region. On the left, an organization works with many teams to support many individual, bespoke Consul clusters. On the right, the Consul cluster operates as a single unified server, with workloads registering into it via one, or many Consul client tenant clusters.
With admin partitions, teams do not need to share the responsibility of maintaining individual server clusters with the organization. These teams focus on contextually-relevant tasks and maintenance directly related to the business value they are delivering, such as service discovery and network automation. Organizations utilize the Consul server cluster as a unified control plane for this fleet of Consul client tenant instances.
Admin partitions creates a mechanism for teams to deploy and share services across their organization, and across other Consul client clusters. This automates the provisioning and registration of trusted Consul clients to a cluster. Admin partitions increases the velocity of a team's ability to deliver business value, eliminating the operational overhead of teams requesting resources from an organization. Admin partitions gives teams autonomy and agency, while the organization maintains control and support of the global Consul installation inside the organization.
In this tutorial, you will deploy HashiCups across two admin partitions, using HCP Consul and two Amazon Elastic Container Service (ECS) clusters, to Consul service mesh. Each ECS cluster is assigned to an admin partition, with HashiCups services hosted in both partitions. Configure HashiCups across partitions using Consul service mesh configuration entries to create the HashiCups deployment. Finish by verifying the services in Consul, then confirming the operation of HashiCups in your web browser.
Prerequisites
- An Amazon Web Services (AWS) account with permission to deploy Amazon Elastic Container Service resources.
- Access to AWS credentials to deploy resources via Terraform.
- HashiCorp Cloud (HCP) Service Principal Credentials. Learn how to create Service Principals by reading the HCP Docs Service Principals documentation page.
Configure required project resources
Begin by cloning the repository.
$ git clone https://github.com/hashicorp/learn-consul-terraform.git
Navigate into the repository folder.
$ cd learn-consul-terraform
Fetch the tags from the remote git server to checkout the git tag for this tutorial.
$ git fetch --all --tags && git checkout tags/v0.6
Navigate into the project folder for this tutorial.
$ cd datacenter-deploy-ecs-hcp-ap
Set your HCP service principal credentials as environment variables.
$ export HCP_CLIENT_ID=YOUR_HCP_CLIENT_ID_GOES_HERE
$ export HCP_CLIENT_SECRET=YOUR_HCP_CLIENT_SECRET_GOES_HERE
Deploy required project resources
This tutorial begins by deploying an HCP Cluster, and two Amazon ECS clusters to deploy HashiCups across two ECS clusters. You will build upon this code, using Consul to set the admin partition for each tenant cluster. One partition (default) for private, internal services, another partition (part2) for public-facing services accessed via the internet. The default partition is assigned to HCP Consul, spanning HCP Consul, and one ECS Cluster, clust1. The part2 partition is assigned to ECS cluster, clust2. The following diagram will help you familiarize yourself with this architectural setup.
Note
Using Consul on ECS, admin partitions are assigned to individual ECS Clusters. An ECS cluster can belong to
the default
partition, but cannot be assigned to other partitions assigned to other Amazon ECS clusters.
Initialize the terraform project.
$ terraform init Terraform has been successfully initialized! # . . .
Deploy the initial resources for this tutorial to your AWS and HCP accounts, consisting of your HCP Consul Cluster,
and two Amazon ECS clusters. Use terraform apply
to deploy, which presents a confirmation screen to deploy
the resources. Type “yes” to confirm the deployment of these resources.
$ terraform apply Terraform will perform the following actions: # . . . Plan: 52 to add, 0 to change, 0 to destroy. Do you want to perform these actions?Terraform will perform the actions described above.Only 'yes' will be accepted to approve. Enter a value: yes # . . . Apply complete! Resources: 52 added, 0 changed, 0 destroyed.
The ECS mesh-task terraform submodule creates your application's ECS task definition, including Consul-specific configuration to register the task definition as both a Consul node, and service.
Create HashiCups infrastructure
Your HCP and AWS accounts currently include an HCP Consul cluster, and two Amazon ECS clusters. Continue on, building the Terraform code for the HashiCups application and surrounding infrastructure.
Create a file for the ACL controllers in the current project folder.
$ touch hashicups-acl_controllers.tf
Each ECS cluster and Consul partition uses an ACL controller to manage task access to HCP Consul. Create ACL Controllers for each ECS cluster, noting the highlights which enable partitions, and assigns each ACL controller to an ECS cluster and partition.
hashicups-acl_controllers.tf
module "acl_controller" { for_each = { for cluster in aws_ecs_cluster.clusters : cluster.name => cluster } source = "registry.terraform.io/hashicorp/consul-ecs/aws//modules/acl-controller" version = "0.4.1" log_configuration = { logDriver = var.ecs_ap_globals.cloudwatch_config.log_driver options = { awslogs-group = aws_cloudwatch_log_group.acl_controllers[each.value.name].name awslogs-region = var.region awslogs-stream-prefix = "${each.value.name}-${local.acl_prefixes.logs}" awslogs-create-group = var.ecs_ap_globals.cloudwatch_config.create_groups } } subnets = module.vpc.private_subnets consul_server_http_addr = hcp_consul_cluster.example.consul_public_endpoint_url consul_bootstrap_token_secret_arn = aws_secretsmanager_secret.bootstrap_token.arn region = var.region consul_partitions_enabled = var.ecs_ap_globals.enable_admin_partitions.enabled consul_partition = each.value.name == local.clusters.one ? local.admin_partitions.one : local.admin_partitions.two ecs_cluster_arn = each.value.arn name_prefix = "${local.acl_base}-${each.value.name}"}
Next, create the private and public task definitions for HashiCups using the mesh-task
submodule.
The private task definitions in the first submodule block represent HashiCups services assigned to the default
admin partition on ECS cluster, clust1. The consul_partition
parameter for each mesh-task
represent the
admin partitions to which each group of tasks in mesh-task
is being assigned. local.admin_partitions.one
represents the default
partitions on Amazon ECS cluster, clust1. local.admin_partitions.two
represents the part2
partition on Amazon ECS cluster, clust2.
The tasks in each task definition group (public, and private) are created using by using the
terraform-aws-consul-ecs mesh-task
submodule, once for public, and once for private. To assign a task definition to an admin partition,
the submodule uses parameters for the partition and namespace to assign the tasks to specified values.
Before creating the tutorial's task definitions, review the code sample below to observe the parameters in context of the submodule. To learn more, read the mesh-task usage docs on consul.io.
hashicups-tasks.tf
# This code block isn't intended for the tutorial, but as a reference point for the modules below.module "hashicups-example-ecs-task-definition" { source = "registry.terraform.io/hashicorp/consul-ecs/aws//modules/mesh-task" version = "0.4.1" . . . consul_partition = "partition-example-name" consul_namespace = "namespace-consul-name" . . . }
Create a file for these task definitions, in the current project folder.
$ touch hashicups-tasks.tf
Paste the following code into hashicups-tasks.tf
hashicups-tasks.tf
module "hashicups-tasks-private" { for_each = { for service in var.hashicups_settings_private : service.name => service } source = "registry.terraform.io/hashicorp/consul-ecs/aws//modules/mesh-task" version = "0.4.1" acls = true tls = true consul_image = var.ecs_ap_globals.consul_enterprise_image.enterprise_latest consul_server_ca_cert_arn = aws_secretsmanager_secret.consul_ca_cert.arn gossip_key_secret_arn = aws_secretsmanager_secret.gossip_key.arn consul_client_token_secret_arn = module.acl_controller[local.clusters.one].client_token_secret_arn acl_secret_name_prefix = local.acl_prefixes.cluster_one retry_join = local.retry_join_url consul_datacenter = local.consul_dc consul_partition = local.admin_partitions.one consul_namespace = local.namespace family = each.value.name port = each.value.portMappings[0].hostPort upstreams = length(each.value.upstreams) > 0 ? each.value.upstreams : [] log_configuration = { logDriver = var.ecs_ap_globals.cloudwatch_config.log_driver options = { awslogs-stream-prefix = each.value.name awslogs-region = var.region awslogs-create-group = var.ecs_ap_globals.cloudwatch_config.create_groups awslogs-group = "${local.log_paths.private_hashicups_services}/${each.value.name}" } } container_definitions = [{ essential = true cpu = 0 mountPoints = [] volumesFrom = [] name = each.value.name image = each.value.image logConfiguration = { logDriver = var.ecs_ap_globals.cloudwatch_config.log_driver options = { awslogs-stream-prefix = each.value.name awslogs-region = var.region awslogs-create-group = var.ecs_ap_globals.cloudwatch_config.create_groups awslogs-group = "${local.log_paths.private_hashicups_apps}/${each.value.name}" } } # Create the environment variables so that the frontend is loaded with the environment variable needed to communicate with public-api environment = concat(each.value.environment, [{ name = "NAME" value = "${var.ecs_ap_globals.global_prefix}-${each.value.name}" }]) portMappings = [{ containerPort = each.value.portMappings[0].containerPort hostPort = each.value.portMappings[0].hostPort protocol = each.value.portMappings[0].protocol }] }] task_role = { id = each.value.name arn = aws_iam_role.hashicups[var.ecs_ap_globals.ecs_clusters.one.name].arn } additional_execution_role_policies = [ aws_iam_policy.hashicups.arn ]} module "hashicups-tasks-public" { for_each = { for service in var.hashicups_settings_public : service.name => service } source = "registry.terraform.io/hashicorp/consul-ecs/aws//modules/mesh-task" version = "0.4.1" acls = true tls = true consul_image = var.ecs_ap_globals.consul_enterprise_image.enterprise_latest consul_server_ca_cert_arn = aws_secretsmanager_secret.consul_ca_cert.arn gossip_key_secret_arn = aws_secretsmanager_secret.gossip_key.arn consul_client_token_secret_arn = module.acl_controller[local.clusters.two].client_token_secret_arn acl_secret_name_prefix = local.acl_prefixes.cluster_two retry_join = local.retry_join_url consul_datacenter = local.consul_dc consul_partition = local.admin_partitions.two consul_namespace = local.namespace family = each.value.name port = each.value.portMappings[0].hostPort upstreams = length(each.value.upstreams) > 0 ? each.value.upstreams : [] log_configuration = { logDriver = var.ecs_ap_globals.cloudwatch_config.log_driver options = { awslogs-group = "${local.log_paths.public_hashicups_services}/${each.value.name}" awslogs-region = var.region awslogs-stream-prefix = each.value.name awslogs-create-group = var.ecs_ap_globals.cloudwatch_config.create_groups } } container_definitions = [{ essential = true cpu = 0 name = each.value.name image = each.value.image logConfiguration = { logDriver = var.ecs_ap_globals.cloudwatch_config.log_driver options = { awslogs-group = "${local.log_paths.public_hashicups_apps}/${each.value.name}" awslogs-region = var.region awslogs-stream-prefix = each.value.name awslogs-create-group = var.ecs_ap_globals.cloudwatch_config.create_groups } } # Create the environment variables so that the frontend is loaded with the environment variable needed to communicate with public-api environment = each.value.name == var.ecs_ap_globals.task_families.frontend ? concat(each.value.environment, [ { name = local.env_vars.public_api_url.name value = local.env_vars.public_api_url.value }, { name = "NAME" value = "${var.ecs_ap_globals.global_prefix}-${each.value.name}" } # The else of the ternary begins here. Add the NAME key for the rest of the task definitions. ]) : concat(each.value.environment, [{ name = "NAME" value = "${var.ecs_ap_globals.global_prefix}-${each.value.name}" }] ) portMappings = [ { containerPort = each.value.portMappings[0].containerPort hostPort = each.value.portMappings[0].hostPort protocol = each.value.portMappings[0].protocol } ] mountPoints = [] volumesFrom = [] }] task_role = { id = each.value.name arn = aws_iam_role.hashicups[var.ecs_ap_globals.ecs_clusters.two.name].arn } additional_execution_role_policies = [ aws_iam_policy.hashicups.arn ]}
Next, create data resources for each ECS task definition created from the mesh-task
submodule. The data resources
use metadata from their underlying ECS task definitions, mappingmesh-task
task definitions to AWS ECS Service
resources.
Insert the code below at the end of data.tf
, in the current project folder.
data.tf
data "aws_ecs_task_definition" "public_tasks" { for_each = toset(local.tasks.public) task_definition = each.value depends_on = [module.hashicups-tasks-public]} data "aws_ecs_task_definition" "private_tasks" { for_each = toset(local.tasks.private) task_definition = each.value depends_on = [module.hashicups-tasks-private]} data "consul_services" "all" { query_options { namespace = local.namespace } depends_on = [aws_ecs_service.public_services, aws_ecs_service.private_services]} data "consul_service" "each" { for_each = toset(concat(local.tasks.public, local.tasks.private)) name = each.key query_options { wait_time = "1m" }} locals { tnames = { frontend = data.consul_service.each["frontend"].name payments = data.consul_service.each["payments"].name postgres = data.consul_service.each["postgres"].name public-api = data.consul_service.each["public-api"].name product-api = data.consul_service.each["product-api"].name }}
Create the second admin partition
HCP Consul generates the default admin partition at the time of installation. Subsequent partitions are created by referencing a new partition name in a service configuration file, or by creating the partition resource via Consul cluster configuration. You will create the partition via Consul cluster configuration using Terraform. This partition, part2, consists of public-facing HashiCups services on ECS cluster, clust2.
Create a file for the admin partition in the current project folder.
$ touch consul-admin_partitions_part2.tf
Paste the code below, creating the part2
admin partition.
consul-admin_partitions_part2.tf
resource "consul_admin_partition" "partition-two" { name = local.admin_partitions.two description = "Admin Partition for public facing HashiCups services" depends_on = [hcp_consul_cluster.example]}
Create exported services
HashiCups services span two Amazon ECS Clusters, in different admin partitions. Consul's Exported Services feature defines which services can communicate outside its admin partition.
Create exported service entries for product-api and public-api. These two services communicate with each other in HashiCups.
Create a file for the exported services, in the current project folder.
$ touch consul-exported_services.tf
Insert the following code for product-api and public-api.
consul-exported_services.tf
resource "consul_config_entry" "export_product_api_to_part2" { kind = "exported-services" name = var.ecs_ap_globals.admin_partitions_identifiers.partition-one partition = var.ecs_ap_globals.admin_partitions_identifiers.partition-one namespace = var.ecs_ap_globals.namespace_identifiers.global config_json = jsonencode({ Services = [ { Name = var.ecs_ap_globals.task_families.product-api Consumers = [ { Partition = consul_admin_partition.partition-two.name } ] } ] }) depends_on = [aws_ecs_service.public_services]} resource "consul_config_entry" "export_public_api_to_default" { kind = "exported-services" name = consul_admin_partition.partition-two.name partition = consul_admin_partition.partition-two.name namespace = var.ecs_ap_globals.namespace_identifiers.global config_json = jsonencode({ Services = [ { Name = var.ecs_ap_globals.task_families.public-api Consumers = [ { Partition = var.ecs_ap_globals.admin_partitions_identifiers.partition-one } ] } ] }) depends_on = [aws_ecs_service.public_services]}
Create service defaults
The product-api uses a service defaults configuration in Consul Service Mesh to declare its service protocol as a default global value.
Create a file for the service defaults, in the current project folder.
$ touch consul-service_defaults.tf
Paste the code below into consul-service_defaults.tf
.
consul-service_defaults.tf
resource "consul_config_entry" "product-api" { kind = "service-defaults" name = data.consul_service.each["product-api"].name config_json = jsonencode({ Protocol = local.consul_service_defaults_protocols.tcp })}
Create service intentions
Service intentions permit access between source and destination services in the Consul service mesh. Create service intentions for services which communicate with each other in the HashiCups application.
Create a file for the service intentions, in the current project folder.
$ touch consul-service_intentions.tf
Paste the code below, into consul-service_intentions.tf
.
consul-service_intentions.tf
resource "consul_config_entry" "product_api_intentions_to_public_api_on_part2" { kind = "service-intentions" name = local.tnames.product-api namespace = var.ecs_ap_globals.namespace_identifiers.global partition = var.ecs_ap_globals.admin_partitions_identifiers.partition-one config_json = jsonencode({ Sources = [ { Action = "allow" Type = "consul" Precedence = 9 Name = local.tnames.public-api Namespace = var.ecs_ap_globals.namespace_identifiers.global Partition = consul_admin_partition.partition-two.name } ] })} resource "consul_config_entry" "public_api_intentions_to_frontend_on_part2" { kind = "service-intentions" name = var.ecs_ap_globals.task_families.public-api namespace = var.ecs_ap_globals.namespace_identifiers.global partition = consul_admin_partition.partition-two.name config_json = jsonencode({ Sources = [ { Action = "allow" Type = "consul" Precedence = 9 Name = local.tnames.frontend Namespace = var.ecs_ap_globals.namespace_identifiers.global Partition = consul_admin_partition.partition-two.name } ] })} resource "consul_config_entry" "payments_intentions_to_public_api_on_part2" { kind = "service-intentions" name = local.tnames.payments namespace = var.ecs_ap_globals.namespace_identifiers.global partition = var.ecs_ap_globals.admin_partitions_identifiers.partition-one config_json = jsonencode({ Sources = [ { Action = "allow" Type = "consul" Precedence = 9 Name = local.tnames.public-api Namespace = var.ecs_ap_globals.namespace_identifiers.global Partition = consul_admin_partition.partition-two.name } ] })} resource "consul_config_entry" "postgres_intentions_to_product_api_on_default" { kind = "service-intentions" name = local.tnames.postgres partition = var.ecs_ap_globals.admin_partitions_identifiers.partition-one config_json = jsonencode({ Sources = [ { Action = "allow" Precedence = 9 Type = "consul" Name = local.tnames.product-api Namespace = var.ecs_ap_globals.namespace_identifiers.global Partition = var.ecs_ap_globals.admin_partitions_identifiers.partition-one } ], })} resource "consul_config_entry" "deny_all" { kind = "service-intentions" name = "*" config_json = jsonencode({ Sources = [ { Action = "deny" Name = "*" Precedence = 9 Type = "consul" Namespace = "*" } ] }) depends_on = [consul_admin_partition.partition-two]} resource "consul_config_entry" "deny_all_part2" { kind = "service-intentions" name = "*" partition = "part2" config_json = jsonencode({ Sources = [ { Action = "deny" Name = "*" Precedence = 9 Type = "consul" Namespace = "*" } ] }) depends_on = [consul_admin_partition.partition-two]}
Create HashiCups Amazon ECS services
To deploy HashiCups, each mesh-task
definition operates as an Amazon ECS Service. The aws_ecs_service
terraform
resource creates the deployment for the task definition. When the ECS service finishes deploying, each task definition
is represented in ECS as an active task. In HCP Consul each active task is represented as a service in Consul service
mesh, and as a node in the Consul cluster.
Create a file for the ECS Services, in the current project folder.
$ touch hashicups-ecs-services.tf
Place the code into the file.
hashicups-ecs-services.tf
resource "aws_ecs_service" "private_services" { for_each = data.aws_ecs_task_definition.private_tasks desired_count = 1 enable_execute_command = true cluster = aws_ecs_cluster.clusters[local.clusters.one].arn launch_type = local.launch_fargate propagate_tags = local.service_tag name = each.value.family task_definition = each.value.arn network_configuration { subnets = module.vpc.private_subnets security_groups = [aws_security_group.example_client_app_alb.id] assign_public_ip = false }} resource "aws_ecs_service" "public_services" { for_each = data.aws_ecs_task_definition.public_tasks desired_count = 1 enable_execute_command = true cluster = aws_ecs_cluster.clusters[local.clusters.two].arn launch_type = local.launch_fargate propagate_tags = local.service_tag name = each.value.family task_definition = each.value.arn network_configuration { assign_public_ip = true subnets = module.vpc.private_subnets security_groups = [aws_security_group.example_client_app_alb.id] } dynamic "load_balancer" { # Only configure load balancing targets for tasks that require it, namely, any entity present in the local.entities list that filters the required tasks. # The for_each evaluates true when the container name and task definition match each other. for_each = { for e in local.load_balancer_public_apps_config : e.container_name => e if each.value.task_definition == e.container_name } content { container_name = each.value.task_definition container_port = load_balancer.value.container_port target_group_arn = load_balancer.value.target_group } }}
Create an outputs file, in the current project folder.
$ touch outputs.tf
You will create ouputs from the deployed resources to log in to HCP Consul, and observe the HashiCups application in a web browser.
Place the following code in outputs.tf
.
output "outputs_sensitive" { value = { consul_bootstrap_token = hcp_consul_cluster.example.consul_root_token_secret_id } sensitive = true} output "outputs_not_sensitive" { value = { consul_ui_address = hcp_consul_cluster.example.consul_public_endpoint_url hashicups_url = "http://${aws_lb.example_client_app.dns_name}" }}
Using terraform apply
, deploy the HashiCups application and related configuration.
$ terraform apply Plan: 66 to add, 0 to change, 0 to destroy. Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes # . . . Apply complete! Resources: 66 added, 0 changed, 0 destroyed. Outputs: outputs_not_sensitive = { "consul_ui_address" = "https://dc1-uO0.consul.223350f5-f0b9-4d89-959d-aa8eab3184e5.aws.hashicorp.cloud" "hashicups_url" = "http://ap-1445958408.us-east-1.elb.amazonaws.com"} outputs_sensitive = <sensitive>
Validate services
Using terraform output
, retrieve the login token in your shell. Copy the HCP Consul URL in thevalue.consul_ui_address
stanza of the json output.
$ terraform output -json
{ "outputs_not_sensitive": { "sensitive": false, "type": [ "object", { "consul_ui_address": "string", "hashicups_url": "string" } ], "value": { "consul_ui_address": "https://dc1-VwY.consul.223350f5-f0b9-4d89-959d-aa8eab3184e5.aws.hashicorp.cloud", "hashicups_url": "http://ap-1741556077.us-east-1.elb.amazonaws.com" } }, "outputs_sensitive": { "sensitive": true, "type": [ "object", { "consul_bootstrap_token": "string" } ], "value": { "consul_bootstrap_token": "5e157744-d58f-bffb-c403-8bf896e9d8fe" } }}
Navigate to the Consul UI URL in your browser. Log in with the token.
After logging in, click the “Admin Partition” dropdown menu in the top-left corner, selecting an admin partition to observe services for the selected partition.
Click on Intentions to observe the Service Intentions in each partition.
Visit HashiCups application
Next, navigate to the HashiCups URL in your browser. Retrieve the URL with terraform output
. Copy the value in the
outputs_not_sensitive.value.hashicups_url
stanza of your json output.
$ terraform output outputs_not_sensitive -json
{ "outputs_not_sensitive": { "sensitive": false, "type": [ "object", { "consul_ui_address": "string", "hashicups_url": "string" } ], "value": { "consul_ui_address": "https://dc1-VwY.consul.223350f5-f0b9-4d89-959d-aa8eab3184e5.aws.hashicorp.cloud", "hashicups_url": "http://ap-1741556077.us-east-1.elb.amazonaws.com" } }}
When the page loads, the HashiCups application renders on-screen, with a collection of beverages to choose from, for (fictional) purchase. This confirms HashiCups services are communicating across partitions, across Amazon ECS clusters. This concludes the tutorial.
Clean up
Bring down the infrastructure using terraform destroy
.
$ terraform destroy -auto-approve
The clean-up process takes up to 20 minutes.
Next steps
In this tutorial, you deployed HCP Consul, two Amazon ECS clusters, HashiCups tasks as services deployed to Consul on ECS. These clusters comprised of services deployed across admin partitions, in individual Amazon ECS clusters. To learn more, take the following tutorials and read the docs to learn more about Amazon ECS at HashiCorp, and Consul admin partitions.
- Take the Deploy an application to Amazon Elastic Container Service course on Learn to learn how to use HashiCorp Waypoint with ECS.
- Deploy a Vault agent to Amazon ECS with the Vault Agent with Amazon Elastic Container Service course on Learn.
- Read the Admin Partitions docs on consul.io to learn more about the functionality Consul Admin Partitions.