HashiCorp has announced yet another new beta product in Consul-Terraform-Sync. This product appears to allow the automation of Consul services using Terraform. Personally, I struggled long and hard on this concept. Yes, I understand the concept of Network Infrastructure Automation, (NIA) but I have struggled with understanding the role of Consul.
I have found the way that Hashicorp has defined Consul confusing, with no real definition of what it is at its core. Is it a Multi-Platform Service Mesh? Is it a Service Discovery and Health Checker? Or is it a mechanism that supports Network Infrastructure Automation; it is this split personality that confuses me. It is the only HashiCorp product that has this dichotomy. Every other HashiCorp product follows the traditional Linux dichotomy of having a single-use and doing it well. Packer creates images and does it well. Terraform is an IaC product, and the list continues.
HashiCorp Consul is a multiplex, it has many functions, and this is both its power and its problem. HashiCorp’s marketing is muddled as it is not a single function. There are often conflicting stories being told about the product. I must point out that this is not a blight against Consul itself or HashiCorp, but more to the fact that Consul is a complicated product, both from the position of its self as software and from the position of its messaging.
It is with this in mind that I have struggled to understand the purpose of Consul-Terraform-Sync. And this is the reason that this post was not released earlier.
What is Consul-Terraform-sync?
The concept was introduced in October during HashiCorp’s HashiConf where they defined it as a product to reduce the burden on operators by automating Day Two networking tasks thereby reducing the need for manual work. Now there is a lot to dissect in that statement. Not least of which is what are Day Two networking tasks?
What are Day Two networking tasks?
Day Two Operations are traditionally those tasks that keep a service running: patching, BIOS updates, new VLAN creation adding new LUNs to your storage arrays, new S3 buckets, backup and continuity plans and their testing. How do these things which appear to be throwbacks to the strictures of legacy environments transfer to this modern DevOps dominated world? Isn’t the concept of patching now just a release cycle a part of the infinity loop? But Day Two operations also means keeping things going and it is this to this that the product focuses on with the concept of tasks.
So how does Consul-Terraform-Sync help here?
HashiCorp’s main goal with this product is to enable organizations to manage their entire network infrastructure with Consul and Terraform. This is a laudable goal. Server and application teams are well versed in the benefits of automating common tasks to remove the human factor, but the concept of NetOps is still quite a fledgling concept within networking teams.
Terraform is one of the foremost IaC products and with the proper provisioners can and does build a complex environment, in AWS, GCP, Azure and also on traditional locally deployed infrastructure like VMware-based environments. Where IaC has not been very present in the network space. HashiCorp hopes to change that fact with this product. Vendors like Palo Alto Networks, Cisco, Checkpoint and F5 are launch partners to this product enabling automation across a large majority of installed physical networking devices in a homogenous manner for the first time.
The Networking arena is one of the last bastions of point and click, or more likely an environment that is still automated by a mishmash of bash scripts and controlled by locally installed user credentials across a number of vendor-specific products. The ability of Consul-Terraform-Sync to utilize the passing of credentials, certificates and tokens by integrating HashiCorp Vault will significantly increase the security of these very sensitive and critical locally installed infrastructure components, coupled with the ability to have a single IaC product to manage changes to the networking environments will streamline application deployments where there are physical devices involved.
However the real power with this product is the Consul integration, this allows true automation the vendor providers that integrate with Terraform have always been able to create, change and delete VLANs, Ports, firewall rules and more. What the Consul part of this product does is monitor changes to the consul environment and sync those changes automatically to terraform, Consul becomes your source of network truth. If tickets still need to be raised for process requirements, your workflow managers can still do so automatically changing the status as the change moves through the system.
Summary
CTS is still an early stage beta, and as such it is most likely going to undergo a significant cycle of changes before it goes GA. But this product has promise, any in-roads that brings core networking infrastructure into IaC is to be lauded. Not everywhere is using Software-defined network programs like VMware NSX which has a fully defined API to build against.
Further, it is good to see that companies like Cisco and F5 building providers for Terraform. These are powerhouses of the physical networking space and it shows a change in their perspectives towards being more inclusive and understanding of the new world space. Consul-Terraform-Sync as a product has the potential to be game-changer.
