In our last post, What is a Service Mesh? I said Mesh, not Mess, we introduced the concepts of a service mesh and explained how it differs from Service Discovery and API gateways. In that article, we talked about the architecture, and introduced the concepts of the solutions, and explained several use cases.
In this article, we start our journey into deploying a Service Mesh based on Consul, a Hashicorp product.
Why Consul as a Service Mesh?
Hashicorp Consul product is a mature and fully featured product that has almost nine years of development (at the time of writing (January 2023)). Hashicorp has improved its functionality incrementally over this period and recently improved the ease of deploying the solution by deploying the Raft Consensus protocol (2019) into the product to simplify the clustering component of the product. Further, with the release of Consul 1.14, they introduced the concept of the Sidecar into consul to remove the requirement for deploying the Consul Client on Kubernetes. This is a vast improvement in ease of use and significantly lowers the bar to entry; with this change, Consul is now truly a Service Mesh.
There are many advantages to this new architecture. Instead of using the gossip protocol, the Consul Dataplane needs one gRPC connection to the Consul servers. Additionally, any peer networks between the Consul servers and runtimes running workloads are not necessary. There is no requirement to configure the gossip encryption key because, as already stated, the Consul Dataplane does not use the gossip protocol. Furthermore, it is no longer necessary to distribute unique ACL tokens for every client agent.
Our First Decision on deploying Consul as a Service Mesh is where.
Usually, the first discussion point is why, but we assume you have already had that discussion and understand your use cases, requirements and value proposition for deploying a Service Mesh, or you read the previous section. There are three main methods of deploying Consul in the wild. These are:
- Roll your own with VM’s
- Roll your own on Kubernetes
- Deploying Consul on HCP
Each option has its benefits and constraints, but the endpoint is a standardised interface for your service mesh deployment. For the benefits of this article, we will only be deploying the features of the free open-source product. Yes, there is much value in the free version.
Deploying Consul Service the Hard way (roll your own)
Deploying HashiCorp consul services in a cloud environment by rolling your own can be quite a complicated process. But one of the best reasons is that you can utilise the open-source and free, as in beer. Therefore, you can dip your toes into Service Mesh, and enhance your deployment without incurring external costs. There are however some functional limitations.
However, since HashiCorp have the ability to deploy a Consul cluster on Hashicorp’s managed platform HCP, things are significantly simpler to deploy.
Currently the Consul service can be deployed on both AWS and Azure from the HCP portal, we have discussed HCP before, so we are making the assumption you already have a login to the platform.
Once you have successfully logged in you will be presented with the familiar home portal
Click on the Deploy Consul link to start the process of deploying your first Cluster.
As shown above, you can deploy into both AWS and Azure, yes it is still limited regions with AWS have the greater global reach. To start we will look at deploying on AWS, then we will look at the deployment process for Azure.
Select the AWS box and click the next button at the bottom of the form.
You are now presented with two options, for this article we will concentrate on manually deploying the service. So make sure that you have selected “HCP UI workflow” and click the “next” button to continue. The form that you are presented with is quite busy so we will look at each section individually start with the network option.
Chose your network name and desired region, if this is a production environment it is important that you consider your network naming well, because if you need to change it at a later date it will mean a complete redeploy of the environment. We have chosen London as our region, and as this is a test environment, we are not selecting a custom CIDR and accepting the default one provided by HashiCorp. Once you have completed your settings, we move on to the customisation of the cluster.
Give your Cluster a meaningful name, we chose “amazic-consul-cluster”. As with the naming of the network the “Cluster Tier” side needs some consideration if it is to be a production server as it is not possible to change this after deployment. Even though this is a test environment, we decided to go with a standard deployment, as we want to show the cluster functionality. There is also the added advantage that an HCP deployed Consul Cluster is using the Enterprise binaries, so fully featured. Finally we need to set the cluster size.
As this is a Test environment with minimal expected throughput, we chose to deploy the small cluster. It is important to remember that here you are not stuck, if your needs change and you need more throughput or are adding more services, you can edit your deployment’s host sizes. In a production Cluster, it is recommended that you protect your Service Mesh, and not set it to allow public connections, again as this is a test environment and ease of access is desired we set it to allow connections. Finally, we set our version to the latest and greatest as per the deployment recommendations. To deploy this cluster, simply click the deploy “Cluster” button.
NOTE: Amazic is not responsible for any charges relating to this guide. HCP Consul service is a costed deployment and you will incur service costs.
The deployment will take approximately 10 minutes.
While the AWS version is deploying, let’s have a look at the process for deploying on Azure.
As we already have a deployment in place the view is slightly different. Now, when we login to the dashboard, we can see that it recognises that we have a cluster.
Click the “View Consul” button and you will immediately notice that this is significantly different.
To get started we now need to click the “+ Create Cluster” button.
Now we are back in familiar territory.
Make sure you have the radio button in the “Microsoft Azure” box has been selected and click “Next” to continue.
Again we will go down the “HCP UI workflow” route so make sure that the radio button is selected in that box and click “Next” to continue.
Here we can see a slight difference, but this is not due to any difference between Azure HCP and AWS HCP; but due to the fact that we have deployed a Vault Cluster on Azure. If we had not deployed any resources in Azure on HCP this would look exactly the same. If you wish, click the “+ Create new network” link to add a second network to Azure.
Next we look at the cluster customization section, here we can see that there are less options.
The remainder of the Azure customization form reflect the AWS deployment form. If you wish click “Create Cluster” to deploy the server on Azure
NOTE: This is a costed deployment, Amazic is not responsible for any charges incurred.
Whilst we have been reviewing the deployment process on Azure, our AWS cluster had finished deploying and is now running.
Logging on to Consul for the first time.
To logon to Consul for the first time from HCP, click on the “amazic-consul-cluster” link. This will open your cluster environment.
To login to your consul service, you will need to create an access token. To do this click the “Access Consul” dropdown
Click on the “Generate admin token” link.
Copy the token and click the “Launch UI Publicly” link.
Click the log in button and enter the previously copied token into the SecretID box, then click login.
Proof it is alive!
Next we need to connect our AWS VPC to the HVN to that workload can communicate with HCP Consul.
Return to the HCP Portal, and start to configure AWS to HVN peering.
Click the dropdown button and for simplicity we will chose Automatic peering connection
This will direct you to your AWS environment, if you are currently logged in a cloud formation stack page will open. We are going to accept the defaults, the only decision needed to to select the VPC that will be peered.
Click the check box acknowledging that Cloud formation will create ant necessary resources. And click “Create Stack” to create the peering. Once complete you should have something similar to the image below.
Return back to the HCP portal and you will notice that the display has changed
Click on the view existing button to verify the connection.
You have two options to connect further AWS VPC’s, you can either click the “Connect more” button from the currently displayed form or return to the previous form and click the “Connect” button.
The final stage is to Click the “Deploy” button
We will leave this as an exercise for the reader.
Summary
As you can see, Consul is reasonably simple to deploy when using HashiCorp HCP Platform. In the next article ,we will look at automating the creation using Terraform and in the final article in the series we will show how to create service discovery and a service mesh on your environment.
If you have questions related to this topic, feel free to book a meeting with one of our solutions experts, mail to sales@amazic.com.
