This morning, Docker has announced that they are extracting the core functionality of the docker engine, called containerd. It has been open sourced under a new project and will be governed by a yet-to-be-named independent foundation. This is a huge win for everyone using containers; from folks building applications using containers to the public clouds and third parties who provide container orchestration. Let’s talk about why.
For those of us who have been working with Docker for a while, we have been witness to some fairly contentious discussions. Many of these revolve around the core component of containers – the container runtime. It is the container runtime that is responsible for determining how a container image should be instantiated as a Linux process, “contained” by the various kernel mechanisms (such as namespaces and cgroups) and made available to the network stack and storage.
The root of these discussions has been who should determine what should be included in the runtime as well as what should be included. As of late (especially following the 1.12 release of the Docker engine), this hit a fevered pitch and includes strong opinions levied by proponents of Kubernetes. Not to mention talk of forking docker itself.
Summarizing the argument goes like this. Docker is innovating, and the functionality which they are adding to the docker engine (the core runtime that most orchestration stacks are using today, including Kuberenetes) cause other orchestration bits to break. Because of this, folks that have decided to use Kubernetes, Mesos/Marathon, etc., are put into a position whereby they cannot leverage the latest version of the docker engine without modifying the orchestration platforms. The changes would need to either leverage new features or remove/modify existing functions based on docker features that have been deprecated. So, it has been proposed that the “boring” container runtime bits be offered outside of the docker engine itself.
Santa delivered the boring bits under the Christmas tree today.
See, Docker (the Company and steward of the docker project) has been working to spin out all of the technology which has been developed to meet their commercial customer demand. Starting with libcontainer, then libnetwork, runC, SwarmKit, and most recently InfraKit, Docker has been modularizing the pieces of what make up the Docker engine. And work has been underway for some time to do the same with containerd, which made its appearance when Docker cutover to runC and containerd in engine version 1.11. Thus it only makes sense that containerd be stand-alone as well.
This move helps facilitate one of the most important aspects of container adoption: How to standardize the fundamental functionality of container instantiation. By providing containerd to the larger community as an open source project with independent governance, virtually any technology vendor can consume it as the container runtime and, fingers crossed, it can become the center building block of any product that uses containers. In my opinion, the single greatest benefit is supporting the Open Container Initiative (OCI) image and runtime specification.
Having this standard is not necessarily the first step fueling broader container adoption, but it arguably the most important. It will be very interesting to see how other vendors and projects that are building orchestrators and container clustering technologies view the announcement. My hope is that there is now a consensus on how to move forward with building out containerd to provide cross-platform support, integrate other OCI specs, and allow companies to focus on higher-level features that are targeted on real-world container deployment challenges.
For more information about containerd, you can visit the project GitHub repository, where there is extensive documentation including architecture and roadmap READMEs.