Lately I’ve been researching possibilities to detect anomalies during runtime in containers. I came across the open source project Sysdig Falco which seems to do just that. Quoted from their site: “Falco, the open-source cloud-native runtime security project, is the defacto Kubernetes threat detection engine. Falco detects unexpected application behavior and alerts on threats at runtime.”
Sounds good..
Originally posted on Iron Flower
Reposted with permission of Burak Kelebek.
— How does it work?
Falco runs as a privileged container and uses eBFP to capture system calls. Events that match a certain filter expression (rule) result in an alert that can be outputted in different ways (for example as a push message to Slack). It comes pre-equipped with a lot of essential rules and has the possibility to be extended with custom rules.
Below is a simple example taken from Falco’s manual of a condition that alerts whenever a bash shell is run inside a container.
container.id != host and proc.name = bash
The first clause checks that the event happened in a container (Sysdig events have a container field that is equal to “host” if the event happened on a regular host). The second clause checks that the process name is bash. Note that this condition does not even include a clause with a system call! It only checks event metadata. Because of that, if a bash shell does start up in a container, Falco outputs events for every syscall that is performed by that shell.
Here is another example of a rule that triggers when an attacker manages to start netcat on a victim’s container.
- macro: network_tool_procs condition: (proc.name in (network_tool_binaries)) - list: network_tool_binaries items: [nc, ncat, nmap, dig, tcpdump, tshark, ngrep, telnet, mitmproxy, socat] - rule: Launch Suspicious Network Tool in Container desc: Detect network tools launched inside container condition: > spawned_process and container and network_tool_procs output: > Network tool launched in container (user=%user.name command=%proc.cmdline parent_process=%proc.pname container_id=%container.id container_name=%container.name image=%container.image.repository:%container.image.tag) priority: NOTICE tags: [network, process, mitre_discovery, mitre_exfiltration]
Fun fact: the ‘condition’ field uses the Sysdig filter syntax.
To make things more clear, I’ve made a demo below which shows Falco in action.
— Set-up Demo
My test set-up consists of two docker containers. One is Falco running as a privileged container and the second one is a vulnerable website built by DVWA running in a separate container. As can be seen below, once I try to execute a command in the container through command injection in the website Falco matches the event with one of its rules and forwards the alert to Slack.
— Further reads
I would strongly recommend to read Skyscanner’s article about their experiences implementing Sysdig Falco within their Kubernetes clusters.
