Kubernetes configuration files and object definitions are most commonly written in YAML files. As these files define how things run on our clusters, improving YAML file security and flow is important. Among the most crucial things, our files need to be audited and validated before being applied to our Kubernetes clusters. Misconfigurations such as escalated container privileges and missing keys have to be avoided at all costs, as these can open access to unauthorized users.
Luckily, there are many Kubernetes security tools that have been emerging such as Kubescape and ValidKube. In order to take advantage of these tools, they should be implemented in your Kubernetes workflow - especially in the case of ValidKube, as will be described.
Kubescape
Kubescape is a Kubernetes command line tool that audits and scans YAML files and clusters using the NSA-CISA and MITRE ATT&CK frameworks. Kubescape is an especially attractive tool, as it will give you recommendations on how you can fix detected vulnerabilities.
ValidKube
On the other side, we have ValidKube from Komodor which is a free tool used to validate and enforce Kubernetes best practices on your manifests and configuration files, folding in multiple open-source tools.
ValidKube tool is used to:
- Validate configuration files
- Clean manifest files
- Scan and secure your YAML files by ensuring best practice
You don't need special skills or cram commands just to use ValidKube, as it is an online web tool. All you need to do is paste your YAML contents and choose if you want to clean, audit, secure, or validate the YAML contents.
In this tutorial, you will learn how to clean and secure your manifests using ValidKube and Kubescape.
This tutorial will cover the following points
- Scanning and securing your manifests using Validkube
- How to audit and validate your YAML files using Kubescape, and scanning clusters against security frameworks
- Conclusion
Prerequisites
You need a running Kubernetes cluster, as well as the KUBECONFIG for it.
Scanning and securing your manifests using Validkube
ValidKube is great at catching common Kubernetes manifest misconfigurations, such as missing keys and null values. You can validate any type of Kubernetes resource or object. In this tutorial, I will show you how to validate and secure a deployment object. Here are the contents of a sample deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: boemo-app
namespace: default
labels:
app: boemo-app
spec:
replicas: 1
selector:
matchLabels:
app: boemo-app
template:
metadata:
labels:
app: boemo-app
spec:
containers:
- name: server
image: nginx:1.17
volumeMounts:
- name: boemo-app
mountPath: /usr/share/nginx/html
ports:
- containerPort: 80
protocol: TCP
resources:
requests:
cpu: 100m
memory: "128M"
limits:
cpu: 100m
memory: "256M"
env:
- name: LOG_LEVEL
value: "DEBUG"
volumes:
- name: boemo-app
configMap:
name: boemo-app
items:
- key: body
path: index.html
Copy the above YAML and paste it on ValidKube, click on the validate text button to validate the YAML file:
When you click on the validate button ValidKube immediately outputs the results, which should appear on the right hand side showing the following:
- errors: []
filename: /tmp/yaml/target_yaml.yaml
kind: Deployment
status: valid
But, If I enter an invalid value on the replica field, changing it from 1 to a string, ValidKube will give out the expected value type which is the integer type:
- errors:
- 'spec.replicas: Invalid type. Expected: [integer,null], given: string'
filename: /tmp/yaml/target_yaml.yaml
kind: Deployment
status: invalid
Validkube uses Kubeval to validate the deployment object. Kubeval is a command line that validates YAML files by comparing them to Kubernetes generated API schemas.
Cleaning a YAML file using ValidKube
Next, click on the clean text button on ValidKube to clean up the YAML file. Under the hood, ValidKube uses Kube-neat to format, reorganize, and remove clutter from the provided YAML file. Here are the results of the cleaning procedure using kube-neat.
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: boemo-app
name: boemo-app
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: boemo-app
template:
metadata:
labels:
app: boemo-app
spec:
containers:
- env:
- name: LOG_LEVEL
value: DEBUG
image: nginx:1.17
name: server
ports:
- containerPort: 80
protocol: TCP
resources:
limits:
cpu: 100m
memory: 256M
requests:
cpu: 100m
memory: 128M
volumeMounts:
- mountPath: /usr/share/nginx/html
name: boemo-app
volumes:
- configMap:
items:
- key: body
path: index.html
name: boemo-app
name: boemo-app
ValidKube has cleaned the deployed YAML file by formatting the contents and added the volumeMount which was missing. This process of linting, or removing clutter and formatting files the right way makes it easy for code reviewers because they won't be reviewing unnecessary code lines.
Carrying out security analysis using ValidKube
ValidKube uses Trivy from AquaSecurity to check security vulnerabilities. Trivy is a tool that scans images and clusters looking for security vulnerabilities. Here are the security scan results after clicking on the secure text button on our sample deployment file from earlier:
ArtifactName: /tmp/yaml
ArtifactType: filesystem
Metadata:
ImageConfig:
architecture: ""
config: {}
created: "0001-01-01T00:00:00Z"
os: ""
rootfs:
diff_ids: null
type: ""
The results will start by summing up the failure and specifying the detected misconfiguration together with its description:
Results:
- Class: config
MisconfSummary:
Exceptions: 0
Failures: 6
Successes: 22
Misconfigurations:
- Description: A program inside the container can elevate its own privileges and
run as root, which might give the program control over the container and node.
ID: KSV001
IacMetadata: {}
Layer: {}
Message: Container 'server' of Deployment 'boemo-app' should set 'securityContext.allowPrivilegeEscalation'
to false
Namespace: appshield.kubernetes.KSV001
PrimaryURL: https://avd.aquasec.com/appshield/ksv001
Query: data.appshield.kubernetes.KSV001.deny
References:
- https://kubernetes.io/docs/concepts/security/pod-security-standards/#restricted
- https://avd.aquasec.com/appshield/ksv001
Next, you will get a suggested solution to the detected vulnerabilities:
Resolution: Set 'set containers[].securityContext.allowPrivilegeEscalation' to
'false'.
Severity: MEDIUM
Status: FAIL
Title: Process can elevate its own privileges
Type: Kubernetes Security Check
- Description: The container should drop all default capabilities and add only those
that are needed for its execution.
ID: KSV003
IacMetadata: {}
Layer: {}
Message: Container 'server' of Deployment 'boemo-app' should add 'ALL' to 'securityContext.capabilities.drop'
Namespace: appshield.kubernetes.KSV003
PrimaryURL: https://avd.aquasec.com/appshield/ksv003
Query: data.appshield.kubernetes.KSV003.deny
References:
- https://kubesec.io/basics/containers-securitycontext-capabilities-drop-index-all/
- https://avd.aquasec.com/appshield/ksv003
Resolution: Add 'ALL' to containers[].securityContext.capabilities.drop.
Severity: LOW
Status: FAIL
Title: Default capabilities not dropped
Type: Kubernetes Security Check
- Description: '''runAsNonRoot'' forces the running image to run as a non-root user
to ensure least privileges.'
ID: KSV012
IacMetadata: {}
Layer: {}
Message: Container 'server' of Deployment 'boemo-app' should set 'securityContext.runAsNonRoot'
to true
Namespace: appshield.kubernetes.KSV012
PrimaryURL: https://avd.aquasec.com/appshield/ksv012
Query: data.appshield.kubernetes.KSV012.deny
References:
- https://kubernetes.io/docs/concepts/security/pod-security-standards/#restricted
- https://avd.aquasec.com/appshield/ksv012
Resolution: Set 'containers[].securityContext.runAsNonRoot' to true.
Severity: MEDIUM
Status: FAIL
Title: Runs as root user
Type: Kubernetes Security Check
- Description: An immutable root file system prevents applications from writing
to their local disk. This can limit intrusions, as attackers will not be able
to tamper with the file system or write foreign executables to disk.
ID: KSV014
IacMetadata: {}
Layer: {}
Message: Container 'server' of Deployment 'boemo-app' should set 'securityContext.readOnlyRootFilesystem'
to true
Namespace: appshield.kubernetes.KSV014
PrimaryURL: https://avd.aquasec.com/appshield/ksv014
Query: data.appshield.kubernetes.KSV014.deny
References:
- https://kubesec.io/basics/containers-securitycontext-readonlyrootfilesystem-true/
- https://avd.aquasec.com/appshield/ksv014
Resolution: Change 'containers[].securityContext.readOnlyRootFilesystem' to 'true'.
Severity: LOW
Status: FAIL
Title: Root file system is not read-only
Type: Kubernetes Security Check
- Description: Force the container to run with user ID > 10000 to avoid conflicts
with the host's user table.
ID: KSV020
IacMetadata: {}
Layer: {}
Message: Container 'server' of Deployment 'boemo-app' should set 'securityContext.runAsUser'
> 10000
Namespace: appshield.kubernetes.KSV020
PrimaryURL: https://avd.aquasec.com/appshield/ksv020
Query: data.appshield.kubernetes.KSV020.deny
References:
- https://kubesec.io/basics/containers-securitycontext-runasuser/
- https://avd.aquasec.com/appshield/ksv020
ValidKube will categorize the level of impact the vulnerability can make if exploited. The levels range is Low, Medium and High. Vulnerabilities that are classified as high should be solved immediately, but you should not forget about the lower-classified issues either. They will be marked as failures alongside your other results:
Severity: LOW
Status: FAIL
Auditing YAML files with ValidKube
For the auditing procedure, ValidKube uses the Fairwinds Polaris tool which scans clusters looking for misconfigurations. Here are the results of the audit scan after clicking on the audit button:
AuditTime: "2022-06-03T08:28:12Z"
ClusterInfo:
Controllers: 1
Namespaces: 0
Nodes: 0
Pods: 0
Version: unknown
DisplayName: /tmp/yaml/target_yaml.yaml
PolarisOutputVersion: "1.0"
Results:
- CreatedTime: "0001-01-01T00:00:00Z"
Kind: Deployment
Name: boemo-app
Namespace: default
PodResult:
ContainerResults:
- Name: server
ValidKube will categorize issues found in the audit into efficiency and security groups. If a vulnerability belongs to the efficiency category it means that it affects the cluster's performance. For example, the followin report notes efficiency issues around CPU limits and requests not being configured:
Results:
cpuLimitsMissing:
Category: Efficiency
Details: null
ID: cpuLimitsMissing
Message: CPU limits are set
Severity: warning
Success: true
cpuRequestsMissing:
Category: Efficiency
Details: null
ID: cpuRequestsMissing
Message: CPU requests are set
Severity: warning
Success: true
Alongside the efficiency issues, you will get notified if your container has dangerous capabilities:
dangerousCapabilities:
Category: Security
Details: null
ID: dangerousCapabilities
Message: Container does not have any dangerous capabilities
Severity: danger
Success: true
All fields that do not have values will be detected and displayed:
hostPortSet:
Category: Security
Details: null
ID: hostPortSet
Message: Host port is not configured
Severity: warning
Success: true
How to audit and validate your YAML files using Kubescape
Kubescape introduction
Unlike ValidKube which is runs on the web, Kubescape is a handy command line tool that scans your cluster and detects vulnerabilities. After scanning your cluster it gives you a security analysis report of every object and resource in your cluster. This report is very detailed in a way that it gives you solutions and a documentation link of the detected security vulnerability. DevSecOps just got easier!
In contrast to ValidKube, which uses tools such as Polaris, Trivy, Kubeval, and Kube-neat under the hood, Kubescape uses security guidelines such as the NSA's Kubernetes hardening advice and the MITRE framework to audit your cluster. It is highly recommended that you use a tool such as Kubescape to audit your cluster before applying for any security compliance certifications. If your cluster passes the Kubescape scan without issues, then it is compliant with the NSA and MITRE security guidelines and therefore complies with the best in class advice.
Setting up Kubescape
On MacOS, Kubescape can be installed using Homebrew with the following commands:
brew tap armosec/kubescape
brew install kubescape
You can use the following command to install Kubescape, if you are a Windows user:
iwr -useb https://raw.githubusercontent.com/armosec/kubescape/master/install.ps1 | iex
Check if Kubescape has been installed successfully using the following command:
kubescape --version
If it is, you will be shown the installed version of Kubescape that shows that it has been installed successfully:
kubescape version v2.0.152
After installing Kubescape, use the kubescape -h command to get all the available commands, as there are many interesting features which we do not cover in this guide. In this tutorial, we will only use it to scan a YAML file that has a deployment. You may recognize it from earlier:
apiVersion: apps/v1
kind: Deployment
metadata:
name: boemo-app
namespace: default
labels:
app: boemo-app
spec:
replicas: 1
selector:
matchLabels:
app: boemo-app
template:
metadata:
labels:
app: boemo-app
spec:
containers:
- name: server
image: nginx:1.17
volumeMounts:
- name: boemo-app
mountPath: /usr/share/nginx/html
ports:
- containerPort: 80
protocol: TCP
resources:
requests:
cpu: 100m
memory: "128M"
limits:
cpu: 100m
memory: "256M"
env:
- name: LOG_LEVEL
value: "DEBUG"
volumes:
- name: boemo-app
configMap:
name: boemo-app
items:
- key: body
path: index.html
Use the following command to scan the above to scan the deployment YAML file, assuming you've saved it as deployment.yaml:
kubescape scan deployment.yaml --format pdf --output kubescape-scan.pdf
The --format flag specifies which output format should the scan results be presented using. The -output flag specifies the name of the pdf that will contain the scan results.
You can also use the following command which displays the results on the CLI:
kubescape scan deployment.yaml
You will get more details if it is not in pdf format:
Analyzing Kubescape results
The Kubescape scan results include the severity level of the vulnerability or threat detected column. The severity column has three levels which are Medium, Low, and High. All threats that are classified as high should be solved immediately. The "Control Name" column states the component of the cluster or manifest being scanned and inspected.
Sometimes, the scan results will have the "Docs" and "ASSISTANT REMEDIATION" columns which give you more information on how to solve the threat. The "Docs" column will give you the link that refers you to the threat documentation while the "ASSISTANT REMEDIATION" column will suggest ways on how you can solve the threat.
At the end of the scan results, you will see a summary that gives you the risk score based on different frameworks such as the MITRE frameworks.
Scanning a cluster using Kubescape
Since you now know how to scan a YAML file using Kubescape, let's now use it to scan a full cluster. Kubescape will give you an insight on how your cluster performs in terms of security vulnerabilities. Every object available in your cluster will be audited and included in the security report. The report is very detailed to a point where it will give you documentation of the security vulnerability and how to start going about fixing it.
This is where you need a cluster that is running, and the KUBECONFIG file for that cluster. Kubescape will use the currently-active KUBECONFIG context by default. If you do not have a KUBECONFIG configured, you will get an error if you scan a cluster with Kubescape.
Here is the Kubescape command used to scan your cluster:
kubescape scan --submit --enable-host-scan --format-version v2 --verbose
You will get similar output to this:
The above output is just an example of the many object results that Kubescape will give.
Scanning using a specific security guideline and framework
Kubescape allows you to scan your cluster or manifest using a specific framework. Use the following command to scan your cluster using the NSA security framework that we mentioned above:
kubescape scan framework nsa --submit
Use the following command to scan your cluster using the MITRE ATT&CK framework:
kubescape scan framework mitre --submit
Conclusion
ValidKube and Kubescape are tools that you should use to combat security vulnerabilities on a daily basis. Prevention is better than the cure.
ValidKube will surely improve your productivity and workflow as it audits, secures and validates files you pass to it, and bundles multiple useful scans.
Kubescape, on the other hand, can be configured to run as part of your CI/CD pipeline meaning that anything that is misconfigured or gets flagged due to not matching a particular security framework can halt your deployment and allow developers to fix the issue before it becomes a problem.
