17. Building OCI Images¶
Part of the Technical Story: Evaluate Knative build Related to the ADR-0013 Source-to-Image Workflow.
17.1. Context and Problem Statement¶
We want to build OCI images based on Dockerfiles inside our Kubernetes cluster.
17.2. Decision Drivers¶
- MUST run completely in userspace (no privileged rights required)
- MUST be runable on Kubernetes
- MUST be controlled in an automatic manner (CI/CD)
- SHOULD be compatible with Knative Build
17.3. Considered Options¶
- Docker out of Docker
- Docker in Docker
- BuildKit
- img
- Jib
- Buildah
- Kaniko
17.4. Decision Outcome¶
Chosen option: Kaniko, because it is designed for the use case we need. It works on Kubernetes and build OCI images without any daemon and without privileged rights.
17.5. Pros and Cons of the Options¶
17.5.1. Docker out of Docker¶
- Good, because Docker daemon is already running in the Kubernetes cluster
- Bad, because it would interfere the Kuberentes scheduling
- Bad, because it requires privileged mode in order to function, which is a significant security concern
- Bad, because it is not compatible with Knative Build
17.5.2. Docker in Docker¶
- Good, because it is a well known approach
- Bad, because it requires privileged mode in order to function, which is a significant security concern (still better than Docker out of Docker)
- Bad, because it generally incurs a performance penalty and can be quite slow
- Bad, because it is not compatible with Knative Build
17.5.3. BuildKit¶
- Good, because it is created by Docker (Moby) as a “next generation
docker build” - Good, because it is already successfully used by the OpenFaaS Cloud
- Good, because it can run on Kubernetes without privileged rights
- Good, because an official Knative Build Template exists
- Bad, because it needs a deamon
17.5.4. img¶
- Good, same as BuildKit but daemonless
- Bad, because it needs a special Linux Capability “RawProc to create nested containers
- Bad, because currently no official Knative Build Template exists (nevertheless it could be created ourself)
17.5.5. Jib¶
- Good, because it is fast and daemonless
- Good, because an official Knative Build Template exists
- Bad, because it is designed for Java applications only
17.5.6. Buildah¶
- Good, because it is a popular tool for creating OCI container images and maintained by Red Hat (also included by default in RHEL)
- Good, because it is daemonless
- Good, because an official Knative Build Template exists
- Bad, because it requires privileged rights (same as Docker daemon), rootless mode is planned
17.5.7. Kaniko¶
- Good, because it is a popular open source project created by Google
- Good, because it is well documented and easy to use (readymade executor image
gcr.io/kaniko-project/executorexists) - Good, because it is designed to work on Kubernetes
- Good, because it is daemonless
- Good, because it does not need privileged rights (however it requires to run as root inside the build container)
- Good, because an official Knative Build Template exists
- Bad, because there are security concerns about malicious Dockerfiles due to the lack of isolation Issue #106