Developers can and will utilize whatever resources cloud providers offer in simple ways. Infrastructure components can be setup in a matter of minutes. Creating and using cloud resources can quickly get out of control. Especially when it comes to forgotten and thus unused cloud resources. In my previous article I explained common type of resources which can become unused very quickly.
Unused resources still incur money. In the end, you might pay a high monthly bill without having the benefits of those resources. Your business case can even become a negative one because of this. Read on to see how to clean up your unused cloud resources to reduce your bill.
Implications
The most simple implications of too much unused cloud resources is the costs. You will get a bill every month which provides details on the cloud resources you use and how you interact with them. An example: your bill tells you how much Virtual Machines of a specific type (e.g. small, medium, large) you have created. It does not matter if you actually used them or not. You might not be aware of this, but you can be faced with unpleasant surprises.
In case you do not properly “tag” or “group” those resources, no one might know the intended reasons to create those virtual machines. It becomes hard to decide whether or not to delete them.
Another example. Imagine using a feature called “server-less” (functions as a service). Every business feature of an application can be split into a large number of functions. The number of functions can quickly go up. Those are harder to manage than Virtual Machines. For this reasons it’s very important to have clear visibility into the owner and meaning of these functions. Without it, it creates even more chaos. Those functions do not cost any money upfront, but the logs of those functions do cost money – for storage.
A changed mindset
Identifying and deleting unneeded cloud resources becomes harder as soon as more resources of different types and with very different names are created. Developers lose valuable hours to track all of these and decide what to do with it since they cannot work on more important tasks.
Preventing unused cloud resources starts with discipline and a changed mindset. Whoever in your organization interacts with the cloud provider should delete any unneeded resources when they are not needed anymore. Even if they do not cost any money up front. The owners of those resources know best what they created so they are in charge here first. This mindset should be build-in. No one leaves for the weekend keeping any “orphaned” cloud resources. 😉
A second important aspect is naming conventions. Without proper naming conventions, it’s harder to determine which resources should belong to a specific (functional or technical) domain. It’s also more difficult to keep a good overview of them. To Clean up resources based upon technical criteria only is very hard. Also, it’s much harder to automate the cleanup process since those resources can’t be classified and grouped together.
Optimization
Another way to save money is optimization.
One of the biggest benefits of the cloud is the capability to create and destroy resourced on demand. Scaling up and down is a key feature which can really help you keeping costs low. Whenever there is a peak demand, for example around the Christmas period: scale the number of Virtual Machines up. When Christmas is over, scale down. This way, you are only being charged for a limited amount of days since you only pay when the Virtual Machines are actually created and used.
The same principle applies for storage. Choose your storage solutions wisely. Do not use very expensive storage solutions for data which is not so important. An example here. Generate thumbnails on the fly and don’t store them separately if you already store the original images. When data is accessed only once a month (for example salary reports) don’t store them on storage solutions which are optimized to frequently access your data as quickly and as fast as possible.
The role of Infrastructure as Code
Infrastructure as Code (IaC) plays a very important role when maintaining resources in the cloud. Without IaC it quickly becomes a mess. Use IaC to create and maintain resources in a consistent and automated way. Train your developers to use IaC for all of their cloud resources.
When using IaC they have the ability to create and share scripts to provision resources. When committing code to a source code repository, the author is shown. As soon as developer X executes an IaC script to create resources, other developers in the team know who created the resources. IaC helps to keep naming conventions consistent, since other developers read the scripts and can use those as a base for their resources. No need to think about names over and over again. Re-usability is king here.
One command is enough to delete a whole set of (related) cloud resources. No need to login into the web console of your cloud provider to find & seek unused cloud resources. These arguments show the “must have” of practical IaC knowledge for every organization.
Pipelines
The role of pipelines are important when it comes to IaC. Pipelines help to automate a repeated deployment process. No manual commands, less human errors and more ways to integrate the provisioning of cloud resources with other tools.
For example: when a pipeline is finished and the resources are created to deliver an application to a test environment, the Product Owner gets a message. The product owner is aware of the release and can instruct the test team to do their tests. Without this notification the product owner had to ask the release team itself. The process is broken as soon as the Product Owner has a day off or so. Completed tests should store the test results and trigger a pipeline script to destroy the resources immediately. Pipelines help to keep unneeded resources to a minimum.
Special software
Putting IaC and pipelines into action costs time and money. Developers need to learn new tools and new ways of working. If this is (yet) a bridge too far, you can also consider buying special software which help you clean up unused resources. Several packages exist today: Corenttech, CloudCheckr and Nutanix to name a few.
Most of these tools can help you identify unused and underutilized cloud resources by monitoring and logging all of cloud resources you create. It hooks in on all of the interactions between your teams and the cloud platform.
These tools can also optimize your cloud resources by looking at usage trends from the past and provide suggestions/advise on how to optimize it for the future. Tool users can also provide input which the tools use to do suggestions.
For example: when a Virtual Machine is “flagged” it should only be active during office hours, the tools can automatically switch off the Virtual Machine after 5PM and restart it on the following morning when the first people arrive at the office.
It’s also possible to create cost utilization reports customized by date range or number of active resources. Going one step further: it can also create an overview of how much CPU power or memory consumption is being used in a certain period of time.
Just a simple example. If the CPU power or memory consumption is very low for a specific application you can choose to use a cheaper (smaller) Virtual Machine or allocate less memory. Imagine if this applies to a lot of developer teams for all of their environments (Development, Test, Acceptance). This way you can save a lot of money.
Conclusion
Unused cloud resources quickly eat up your cloud budget. The tips and tricks in this article help you to keep your cloud bill low. It helps you to use the full potential of the cloud. The money you save can be spend on other useful projects which are more important for your business.
