How to do DevSecOps with Chef InSpec and the DevSec.io Policy-as-Code framework

It doesn’t matter what operating system you are running, you are always at risk of being attacked. Common sense will help you in defending your assets but unfortunately, this will not suffice. For instance, Sophos recently discovered “Cloud Snooper” that targets Windows and Linux. This rootkit allows a malicious actor to communicate with command and control servers through the firewall. Choosing Linux over Windows for security reasons was probably a no-brainer a decade ago but as this example shows, Linux administrators aren’t free of worry. Read on to do DevSecOps by implementing Policy-as-Code with InSpec and dev-sev.io.

Security on paper is not security at all

There’s more than one way to skin a cat. And surely that can be said about how to secure a Linux server. Many system administrators apply checklists or best practices to their systems. Sometimes it feels like the more senior a system administrator the better the checklist. On the other side of things, you have enterprises that are required to follow certain cybersecurity standards to do business. Most known here are ISO 27002, PCI DSS, NIST, and OWASP. These standards by nature are bulky documents and filled with legalese, which makes them hard to actually implement on production systems as security measures. The Center for Internet Security (CIS) provides a hands-on approach by ‘translating’ the security measures to actual security measures for several operating systems. The CIS Policy-as-Code ‘benchmarks’ are the only best-practice security configuration guides that are both developed and accepted by the government, business, industry, and academic institutions.

Manually enforcing security measures is a source of human error

The CIS benchmarks contain specific actions to take and the rationale behind them. As an example, we will look at the controls defined by the CIS Debian Linux 10 benchmark. Go to https://workbench.cisecurity.org/ to download the latest version. The PDF version counts five hundred pages. It’s a daunting task to verify all the controls listed in the benchmark. Seasoned system administrators amongst you will already know that many of the controls listed are already covered. Either because you know it’s the default setting upon installation or it’s enforced by your own best practices. Applying the controls can be challenging in cloud environments that are continuously changing. Manually executing the benchmark each time is time-consuming and error-prone; Policy-as-Code automates the enforcement of security measures.

Chef InSpec

Chef InSpec (website) is an open-source Policy-as-Code testing framework for infrastructure with a human- and machine-readable language for specifying compliance, security and policy requirements. Written in Ruby makes it portable. Inspec can inspect a wide variety of UNIX flavours and Windows releases.

Policy-as-Code

Inspec helps system administrators and security officers validate the compliance of their computing environment by letting them talk the same language and by letting them use the same tool. Environments already practising infrastructure as code are familiar with writing down their infrastructure requirements as code. Inspec extends this approach by letting practitioners write down their compliance rules in an easy to understand language.
One of the CIS rules is to “ensure the use of dedicated administrative accounts”. How can we translate this to something implementable and verifiable? The CIS benchmark has the answer. It says it’s required to ensure that you can’t log in as root through ssh. This can be achieved by setting the PermitRootLogin parameter to no. It forces SSH access through individual accounts before elevating to root privilege. This limits the opportunity for non-repudiation. It also provides a clear audit trail as sudo logs successful and failed attempts. An Inspec control that describes this compliance rule looks like this:

control 'example-1.0' do
  impact 0.9
  title 'Ensure login disabled'
  desc 'An optional description...'
  describe sshd_config do
    its('PermitRootLogin') {
      should_not cmp 'yes'
    }
  end
end

The rule above tests if the sshd configuration file contains the configuration setting PermitRootLogin and that is not set to yes. Note that we didn’t define the location of the configuration file. Inspec will provide this information for us.

Executing inspec with the above code gives us:

$ inspec exec mytest
Profile: Chef InSpec Profile (example_profile)
Version: 0.1.0
Target:  local://

✔  example-1.0: Ensure root login is disabled via SSH
✔  SSHD Configuration PermitRootLogin should not cmp == "yes"

Profile Summary: 1 successful control, 0 control failures, 0 controls skipped
Test Summary: 1 successful, 0 failures, 0 skipped

Armed with our CIS benchmark and Chef Inspec we can define our requirements as code. Given the big list of controls it’s a very daunting task to do all this work yourself. Fortunately, we can use the resources provided by the DevSec Hardening Framework.

DevSec Hardening Framework Baselines

The dev-sec.io community aims to define an easy to use framework to harden operating systems and services. The framework consists of two parts. It provides Inspec profiles implementing CIS benchmarks (Distribution Independent Linux, Kubernetes, Docker, Apache, SSH, …) and hardening scripts for those benchmarks in Chef, Puppet and Ansible.

In this article, we will show how to verify the security of a default installation of Debian GNU/Linux running SSH and Apache.

We installed Debian GNU/Linux v10 by going through the interactive installer and selected the defaults. At the end of the installation, we chose to install the web server and SSH server software selection options. We didn’t install the desktop environment. Note that Debian by default installs a print server and several standard system utilities. After the installation, we verified we can log in through SSH.

On our workstation, we installed Inspec by following the steps described in the Inspec installation documentation.

Inspec can run in different scenarios. Here we are going to run inspec against a remote Linux machine over SSH. Running against Windows through WinRM or Docker is also possible. The inspec executable has several subcommands but we focus on inspec exec:

inspec exec -t ssh://mylogin:mypass@192.168.0.161 https://github.com/dev-sec/linux-baseline

We run inspec with the -t parameter to pass our credentials and the network address of the machine we want to inspect. We also pass the location of the Linux-baseline git repository.

If you would have set up SSH key-based authentication you can remove your password from the command line. By default Inspec logs in as root onto remote targets so we have overridden the login name. See the CLI documentation for further options, e.g. configuring the usage of sudo after log in.

The output of our command looks like:

The last lines show: “65 successful tests and 44 failures”, which isn’t very good. You can go to https://dev-sec.io/baselines/linux/ to get a better understanding of what each test is doing.

The tests in linux-baseline are based on guides from Deutsche Telekom (German), BetterCrypto and the NSA hardening guide. If the previous benchmark makes you depressed don’t run the CIS distribution independent Linux benchmark against the default Debian installation. We had 737 successful and 320 failures on the benchmark. The CIS benchmark is more extensive.

Not just for Linux

Now that we have established a baseline of controls and have a way to automatically verify our system against those controls we can start our journey to make our system (more) compliant. Fortunately, the community behind the DevSec project has created Chef cookbooks, Puppet modules and Ansible playbooks to bring our system in compliance. These “scripts” will apply the required configurations changes to bring our system into compliance.

After applying our Ansible playbook we rerun our linux-baseline inspec controls to get a perfect score: 120 successful and 0 failures. Even the CIS distribution independent benchmark got better results: 817 successful, 271 failures.

Until now we have focussed on operating system hardening. During the installation of our machine, we enabled SSH and Apache. The DevSec project also provides security baselines for SSH, Apache and MySQL. We can now verify how secure the default configuration on Debian is and harden those applications. The table below gives you a summary of the Inspec tests before and after hardening. Unfortunately, the Apache hardening playbook breaks the setup.

We can conclude for SSH that the provided playbook has secured our configuration and brought it almost 100% in compliance. For Apache, we can conclude it broke it.

Security vs. Usability: DevSecOps

That brings us to an unpleasant but important consequence of applying, or worse enforcing, security checklists. Blindly applying checklists without investigating the risks upfront can, and will probably have destructive consequences for your applications. It also won’t make you loved by your teammates and application users. Another issue to be aware of is that the DevSec project has a limited scope. For example, patch management isn’t included. See https://dev-sec.io/project/ for the full scope.

To make your systems compliant you need to work together with the teams in charge of managing the operating systems and applications. You need to create a common understanding between security, developers and operations: DevSecOps.

We can build this common understanding because you will speak the language of the developers and system administrators. Developers and operations teams will probably be using a configuration management tool and a Continuous Integration tool. Their source code is stored in a version control repository. As a security engineer, you can leverage these tools because Inspec can be called from the CI pipeline to verify compliance. The controls are written down in an easy to understand code that can be stored in version control. It’s clear for the team why something is enforced. When a control is changed there is transparency who made the change and it can go through a review process before being applied. The CI pipeline can make sure the required feedback is given to the correct teams as soon as possible. Next to the security team developers and system administrators should contribute their own compliance rules. It’s also possible to override or selectively include controls by using inheritance (see https://blog.chef.io/understanding-inspec-profile-inheritance/).

Wrapping up

In this article, we looked at Policy-as-Code using Inspec and the DevSec hardening framework. We briefly touched on how you could apply them in a DevSecOps workflow to achieve and maintain compliance. At the same time, you will be reducing and eliminating attack vectors that other teams might be unaware of.