It’s no surprise that Terraform is an increasingly popular IaC tool for many organizations. This is partly due to its broad support for cloud platforms, the excellent documentation, and the large community. The platform becomes more mature every day and innovation thrives. Despite the advantages of it, there are also organizational complexities that need to be taken into account. Suppose your organization has already decided to go for it, what other (not technical) aspects pop up? This article aims to explore a number of them that help decision-makers implement Terraform to ease the path to great adoption. Terraform providers: how to deal with organizational complexity.
Introduction/context
Terraform providers are a set of plugins that Terraform requires to operate. Without these plugins, users are unable to interact with cloud providers, SaaS providers, or APIs of supported tools. Some providers require specific endpoints and/or specific configurations to work correctly. In most cases, they also require authentication with sufficient permissions to interact with the above-mentioned tools.
Please find more information about Terraform providers at the website of Hashicorp – the supplier of Terraform. At the time of writing, there are more than 3000 providers ready for you to use.
Governance
As with all pieces of software in large organizations, IT governance plays a crucial role in its success. For Terraform providers, this is no different.
First of all, Terraform as the main IaC tool should have a clear (business) owner and Product Owner with a responsible team. End-users require a single point of contact for questions, requests, or (technical) problems. Terraform providers are integration points with other tools that are governed by other teams, perhaps even other departments. This might not known to the end-users. They should not care.
However, in case of problems, it should be very clear that there is a shared responsibility. Suppose there are network-related problems that prevent the Terraform AWS provider from interacting with the AWS cloud that is used throughout the entire organization, teams need to be aware of who to contact and the implications of a potential problem.
Terraform providers also require a single owner. Someone who has the final decision on which providers will be made available to end users. Furthermore, someone who keeps an eye on important topics such as security, the providers’ feature road map, open issues, etc.
Without proper governance, all of these aspects remain vague and might prevent end users from delivering software in the best possible way.
Requirements
A lot of companies have a hard time collecting proper requirements. Often they know the technical features they wish, but business requirements are a lot harder to define. With this in mind, there should also be clear requirements for any Terraform provider it wishes to support.
The following requirements and considerations are a starting point for selecting the providers of choice:
- First of all, if you’re in a restricted company, you might want to stick to only selected providers. The registry of Terraform offers so much, that this can be very overwhelming. Decide which tiers are allowed to be used: official, partner, or community. This greatly helps to focus to select only allowed providers.
- Secondly, decide about key criteria which are “must haves” to select a provider. What are the minimum set of features it should support. And besides that, think of the number of maintainers, how the provider deals with upgrades and patches, and support in case of problems. For example, there are already 10+ providers for Keycloak. You need to select the one that best matches your criteria. Pay special care to reviews of practitioners which went ahead.
- Third, check the compatibility of the intended provider. Does it work with the tech stacks it should support? Try to run a small Pilot to find this out.
- Last but not least: assess the actual product. Like a due diligence you would do to a potential supplier. Scan the source code for known or unknown issues and vulnerabilities. Conduct a risk assessment from a business- as well as a security point of view.
With all of these in mind, decide whether or not this is a “safe choice” for your organization.
Life Cycle Management
In the end, every provider is like a standalone piece of software with its life cycle management. End users can view the current version, the last time a version was released, and also some statistics regarding the number of downloads (per period).
Keep in mind that updates might happen automatically (or not at all) and this greatly influences the users who use the provider. Some changes are not backwards compatible so scripts that are negatively affected might break. This has an impact on everyone who uses the provider. The owner of the provider should communicate this to the end users so they become aware of this new feature. Bugs and other (new) features of the provider are also collected on the Github “issues page” of the provider in charge.
Specific versions
Restricted companies that offer “Terraform as a Service” to their end users might also restrict specific versions of providers. They force everyone to use a pinned version. Most of the time, there is a required minimum version. This ensures everyone is up-to-date all the time and also that end users won’t keep using legacy and thus insecure versions. The great thing about it is that end users notice that immediately once they try to roll out their infrastructure resources. An example of a snippet like this can be found below:
terraform {
required_providers {
elestio = {
source = "elestio/elestio"
version = "0.2.0" # check out the latest version in the release section
}
}
}
Testing
Testing is part of Life Cycle management as well. Before providers are made “generally available” to end users, the internal owner of the Terraform needs to make sure it works as expected. So the team in charge must have a set of test scripts that provision pieces of infrastructure or configure a certain server. It would be best to evaluate the desired end-state of this system and compare it with the proposed intentions. Every end user should be able to rely on the outcome of these tests so they are confident it works as expected.
Replacement
Special care should be taken when providers are to be replaced. Sometimes a provider gets outdated and it won’t be maintained anymore. Or there are too many (security) issues that keep hampering teams that use it. The owner needs to decide when to swap the provider for another one (if possible). Or mitigate the issues (themself). This last solution can be a challenge since another party might be the real owner.
Before conducting a replacement, it is important to map the current and future features to see if there are any gaps. Basically find the delta between both implementations. You need to carry out a comparison between the old and the new provider. Sometimes you lose a specific feature. Whether or not this is accepted, leads to a business decision. The outcome of it also needs to be communicated to the end users.
There are more “what if” scenarios which tend to pop up now and then. Take special care to be prepared for them. Then no one should be unpleasantly surprised.
Security
As always, security plays an important role in the selection and maintenance of Terraform providers. Just like any piece of software that is to be used in your organization, this needs to be evaluated. Similar to plugins for Jenkins for example or complete packages which you download from the Maven repository or Dockerhub.
Practical tips as a list of check-boxes:
Every Terraform provider that has its source code on Github should have a security policy and/or published security advisories. This gives you information about how the maintainer deals with security issues and which security advice helps you to safely use the provider.
Besides viewing security-related information which is published on the internet, it is also a good idea to analyze the source code of the provider itself. Use free source code analyzer tools like SonarQube to scan the source code and examine the results. If your screen turns red, you might want to search for an alternative.
Good to know that there are also more than 350 providers in the security domain. These also help to improve the security of your other software stacks and resources. Essentially, your security provider of choice sometimes can help to secure your other Terraform providers.
Wrap up
As seen in this article, Terraform offers a wide range of plugins, the so-called providers. There is a provider for a wide variety of use cases such as cloud provisioning, tool configuration, and security scanning. When selecting and operating these providers in larger organizations, not only the technical aspects count. It is also important to dedicate some time to governance, life cycle management, and security-related topics. Clear requirements and careful selection of the vast number of providers help to get the best one for your organization.
For sure, there is a lot more to explore, I hope this article acts as a useful start for you.
