Kubernetes Security – Securing Container Images with CI/CD pipelines

Kubernetes has become the de facto operating system of the cloud. Kubernetes makes it easy for developers like us to package our applications into portable microservices. However, Kubernetes is challenging to operate and it takes time to master the entire cluster. We often put off addressing security processes until we are ready to deploy code into production. And also our legacy applications fall short of meeting the cloud-native requirements.

One of the important aspects of cloud-native security is addressing container security risks. Deferring it to a later stage in the development life cycle slows down the pace of cloud adoption while raising security and compliance risks.

Kubernetes requires a new approach to security as Kubernetes attack surface is huge, starting from Container base OS vulnerabilities, Cluster hardening and also focusing on runtime security.

Image scanning in CI/CD Pipeline

As a DevOps engineer, we need to ensure that the containers we are shipping are secure. The best way to do this is to include image scanning in the CI/CD pipelines. Early detection of security issues in the pipeline, makes it easier to fix. Most of the issues can be detected before the final deployment. This reduces the chances of outages due to security incidents by a significant percentage.

During the CI (Continuous Integration) cycle we build container images that are pulled and deployed in the further environments. We may be building images from scratch or modifying any third party images, it’s important to identify any known vulnerabilities that may be present in those images.

What is image scanning?

Docker images are composed of several immutable layers, which is nothing but the diff over the base image, that adds files and other changes. Any new Docker image that we create is most likely based on an existing image (FROM statement in the Dockerfile).

Things to remember when creating images:

• Update and rebuild images periodically to grab the newest security patches
• Live-patching containers is not a good practice, we should rebuild the entire image with each update
• Setup a pre-production testbench to make sure these updates are not breaking production
• Checking the software packages, binaries, libraries, operative system files and more against well-known vulnerabilities databases
• Look for security-sensitive configurations like running as privileged (root) user, exposing insecure ports. It is good practice to run a container as non-root. Although user namespace mapping is available, it is not enabled by default. Running as root means that if an attacker were to successfully escape from the container, they would gain root access to the host. Use the USER instruction to create a user in the Dockerfile
• Installing unnecessary packages will increase the attack surface. It is recommended that you keep your image slim. Learn to build smaller images with multi-stage builds so that the surface area of security attack is reduced

Image Scanner

One of the most popular open-source tools for scanning and analyzing container images for security vulnerabilities and policy issues is Anchore Engine. It provides a centralized service for inspection, analysis and applies user-defined acceptance policies to allow automated validation and certification of container images. In the next blog, I will demonstrate to seamlessly plug Anchor Engine into the CI/CD workflow.

Certified Kubernetes Security Specialist – CKS

Looking at the high adoption rate of Kubernetes in the production environment, its need of the time to emphasize on security. Even CNCF newly introduced a Certified Kubernetes Security Specialist certification program announced which is now generally available. This is the third Kubernetes Certification program after CKA and CKAD. Stay tuned to get more information on CKS exam preparation.