Shift security left: secure your CloudFormation templates

DevOps practitioners use Infrastructure-as-Code (IaC) to provision resources they need in the public Cloud. AWS offers CloudFormation as their native IaC solution. CloudFormation templates are written in JSON or YAML format. Nearly every service or feature that you can create using the AWS Console can also be created using CloudFormation. It’s important to learn the best practices of the language since it’s easy to make mistakes. Some of them teach you a good lesson, while others lead to big security risks. Several tools help you to prevent these common pitfalls. In this post, we’ll talk about how to shift security left: secure your CloudFormation templates before they are rolled out to production.

Set the rules

Organizations need to define and set the security and compliance rules before you can actually put “shift security left” into practice. Without any rules, it’s every DevOps team on their own and, creating compliance chaos. Governance of the rules and how to deal with them is critically important to speed up in the long run.

To make things a bit more complicated, organizations struggle to define who should set the rules. There is no absolute answer. The following departments are common ones to handle this:

• The CISO department which works together with the compliance departments
• A specialized Center of Expertise on software security
• A Cloud Competence Center which also defines security baselines

Whichever department or team sets the rules, they need to work closely together with the DevOps teams’ representatives. When so-called “Quality Gates” kicks in, it is critically important to have the teams on-boarded. They are impacted immediately.

Which rules are we talking about here? Let’s clarify this question with some examples:

• Only allow secure (HTTPS/SSL) endpoints, avoid insecure connections by blocking HTTP based traffic (normally port 80 for webservers).
• Enforce the usage of encrypted EBS volumes which also adhere to a specific type.
• Provide a standard list of allowed EC2 instance types and only allow these instances to be provisioned. Prevent over-provisioning and save money. By doing so, it’s also easier to compare costs across different applications or teams.

Business considerations

Security and compliance rules help to “shift left” and put security into the hands of DevOps teams, instead of making security concerns a responsibility of a separate entity like a CISO department or First Line Risk officer. Ultimately, the DevOps teams gain enough knowledge to know how to deal with that kind of risk. Whether they can accept a given risk, or the need to fix them immediately, it’s up to the team itself. In the end, it’s the business representative who has to find the right balance between (perceived) risk and (potential) opportunity for their applications.

Business considerations include:

• Handling risks takes time. If speed of delivery is critical, business representatives might want to accept the risks if the expected benefits/revenue are relatively great. Every organization has to decide for themselves to find the right balance, also taking into account the context of the application and what this means for the rest of the applications and departments.
• Communication and interaction between DevOps teams and the team/department who sets the rules can lead to confusion. These kinds of challenges require proper governance and very clear and consistent communication between departments.
• It’s important to set realistic rules which are aligned with the majority of the DevOps teams. If 90% of the teams do require a certain type of EC2-instance, it’s unrealistic to set a rule to prevent it. Costs might go up as soon as endless discussions continue to frustrate everyone. DevOps teams are also more likely to bypass the rules. Workarounds like this create shadow IT and this poses even more risks for the organization.

Whatever is decided, it’s wise to use tools to define, store and automate the rules. No excels or word docs need to be passed around since they are a thing of the past.

Tools explored

Various tools exist already today. Popular ones are AWS CloudFormation Guard, Checkov (which also supports Terraform and Serverless), CNF Nag, and CNRipper.

It’s impossible to describe all of them. The good news is that they all have the following in common:

• Installation and execution can be done in a local development environment
• Use the CLI to check one or more CloudFormation templates for compliance and security issues.
• Define & write policies and rules in JSON or YAML format which is easy to understand.
• All of the tools are easy to integrate into a CI/CD pipeline. Optionally break the build if the CloudFormation template does not pass the checks.

All tools work on all major operating systems like MacOS, Linux and Windows. You can download prepackaged binaries and build them from source as well.

AWS CloudFormation Guard

AWS offers the official open-source tool called AWS CloudFormation Guard. This tool focuses on two things:

• Check CloudFormation templates against policies and compliance rules. It uses policy-as-code, a declarative syntax.
• It supports “reverse engineering” by generating rules based on existing CloudFormation templates.

Besides checking CloudFormation templates using the CLI, it can also use an AWS Lambda function. This way you can further integrate your cloud way of working alongside a local validation of your templates. To make things more practical: use the AWS Lamda function to integrate this step into your CodePipeline. Lambda can trigger other services, for example, to send out a notification or send a message to a message queue.

AWS CloudFormation guard is like a guard dog – always on the lookout.

A snippet of a CloudFormation template to validate whether an EC2 disk volume is encrypted and which will be deployed in the EU-WEST-1B availability zone:

{
    "Resources": {
        "DummyVolume" : {
            "Type" : "AWS::EC2::Volume",
            "Properties" : {
                "Size" : 100,
                "Encrypted": false,
                "AvailabilityZone" : "eu-west-1b"
            }
        }
    }
}

The accompanying rules would be:

let encryption_flag = true

AWS::EC2::Volume Encrypted == %encryption_flag
AWS::EC2::Volume AvailabilityZone == eu-west-1a

And the step to check if your resource passes the test:

cfn-guard check -t dummy-volume-template.json -r dummy-ec2-volume.ruleset

The output of this rule would be:

[DummyVolume] failed because [Encrypted] is [false] and the permitted value is [true]
[DummyVolume] failed because [AvailabilityZone] is [eu-west-1b] and the permitted value is [eu-west-1a]
Number of failures: 2

Use rulegen

In case you want to use the CloudFormation template to reverse engineer the rules, you would use:

cfn-guard rulegen dummy-volume-template.json

and store the output in a “rule-file”.

Help the business

Given this example, your business and security departments are helped since:

• You do not deploy resources to the wrong availability zone.
• Your security department is happy since you are enforced to encrypt your EC2 volumes, thus securing the data.
• There is no time wasted to validate your resources afterward and communicate back the results to your DevOps teams. No feedback loop needed.
• Besides this, the PO is happy since he/she does not need to define a backlog item to correct these issues. He/she can deliver more and better features to the end-consumers of the application.

In the end, this helps to grow the revenue of the organization. In case you need to convince someone to put more focus on these kinds of rules, these arguments might help you.

Next steps

Get more information on how to use the tool and how to write down (advanced) rules to support various use-cases across your entire organization. It also explains how you can use & customize rulegen to generate advanced rules from existing templates.

CFN Nag

Stelligent is the organization behind CNF Nag. They offer an open-source solution to look for patterns in CloudFormation templates that may indicate insecure infrastructure resources. Besides checking the creation and configuration of resources, it also checks for other security-related aspects such as:

• Wildcards in security group rules: “permissions too open”.
• Scan for password literals inside your templates.
• Access logs that are not enabled for a certain resource such as an Elastic LoadBalancer or CloudFront Distribution

CFN Nag takes a CloudFormation template written in JSON as input, validates it using a rules script written in JSON as well and outputs in table- or JSON format.

Critical issues result in a failure of the scan, less critical issues result in a warning only.

Example to validate if a security group allows outbound traffic to every internet address while this might be restricted (based on what you need or what your organization has defined):

---
| WARN
|
| Resources: ["dummy-security-group"]
|
| Security Groups found with an egress' CIDR group open to the world
Failures count: 0
Warnings count: 1

The return code of the scan is either 0 or 1 and can be used to break the CI/CD pipeline when wrapped around a simple bash script. It is extremely easy to use this simple script in every CI/CD tool, but it offers a great way to prevent insecure deployments.

CFRipper

Yet another tool that does the same as CFN-guard and CFN-Nag is CFRipper.

Powerful features of the tool which are worth mentioning here are:

• Very extensive set of rules which also provide suggestions to fix the security issues which are found.
• Being able to overwrite the default behavior of a certain rule. You can change the rule mode and risk value. This comes into scope for applications that require less strict rules than the ones being set for the other applications.
• More exit codes beyond 0 or 1. Sometimes a scan fails (partially). It would be waste of time to break the entire build if only one out of larger number of CloudFormation templates fails. Thus adding more context to exit codes is beneficial here.
• When using the feature called “resolve parameters” during the scan will resolve/replace invalid parameters inside the scanned CloudFormation templates with correct (valid) values. This helps to speed up development of compliant templates since no new scanning iteration is needed.
• Disable rules based on prefixed names. For example: disable rules which scan resources that use an IAM rule which starts with “sandbox-“. This feature helps to keep the source code consistent. There is no need anymore to comment out specific pieces of source code per environment or to use different rules-files across different environments.

Wrap up

CloudFormation offers a powerful way to provision infrastructure in the AWS cloud. The tools which are highlighted in this article help DevOps teams, security departments and business representatives to deploy only secure & compliant infrastructure- and application components. By doing so, the shift left approach becomes embedded in your DevOps way of working thus speeds up the delivery of features for your end-users. Be sure to check the tools out and make your organization more secure.