004:神秘的数组初始化_使容器神秘化101：面向初学者的深入研究容器技术

004:神秘的数组初始化

by Will Wang

王Will

介绍 (Introduction)

Regardless of whether you are a student in school, a developer at some company, or a software enthusiast, chances are you heard of containers. You may have also heard that containers are lightweight virtual machines, but what does that really mean, how exactly do containers work, and why are they so important?

无论您是学校的学生，某公司的开发人员还是软件爱好者，您都有可能听说过容器。您可能还听说过容器是轻量级虚拟机，但这实际上是什么意思，容器如何工作，它们为什么如此重要？

This story serves as a look into containers, their key great technical ideas, and the applications. I won’t assume any prior knowledge in this field other than a basic understanding of computer science.

这个故事是对容器，其关键的重要技术思想以及应用程序的考察。除了对计算机科学的基本了解之外，我不会假定在该领域有任何先验知识。

内核和操作系统 (The Kernel and the OS)

Your laptop, along with every other computer, is built on top of some pieces of hardware like the CPU, persistent storage (disk drive, SSD), memory, network card, etc.

您的笔记本电脑以及其他所有计算机都建立在某些硬件之上，例如CPU，持久性存储(磁盘驱动器，SSD)，内存，网卡等。

To interact with this hardware, a piece of software in the operating system called the kernel serves as the bridge between the hardware and the rest of the system. The kernel is responsible for scheduling processes (programs) to run, managing devices (reading and writing addresses on disk and memory), and more.

为了与此硬件进行交互，操作系统中的一个称为内核的软件充当了硬件与系统其余部分之间的桥梁。内核负责调度要运行的进程 (程序)，管理设备(在磁盘和内存上读取和写入地址)等。

The rest of the operating system serves to boot and manage the user space, where user processes are run, and will constantly interact with the kernel.

其余的操作系统用于引导和管理用户空间(运行用户进程)，并且将不断与内核进行交互。

虚拟机 (The Virtual Machine)

So you have a computer that runs MacOS and an application that is built to run on Ubuntu. Hmmm… One common solution is to boot up a virtual machine on your MacOS computer that runs Ubuntu and then run your program there.

因此，您有一台运行MacOS的计算机和一个可以在Ubuntu上运行的应用程序。嗯……一种常见的解决方案是在运行Ubuntu的MacOS计算机上启动虚拟机，然后在其中运行程序。

A virtual machine is comprised of some level of hardware and kernel virtualization on which runs a guest operating system. A piece of software called a hypervisor creates the virtualized hardware which may include the virtual disk, virtual network interface, virtual CPU, and more. Virtual machines also include a guest kernel that can talk to this virtual hardware.

虚拟机由运行客户操作系统的某种级别的硬件和内核虚拟化组成。称为管理程序的软件会创建虚拟化的硬件，其中可能包括虚拟磁盘，虚拟网络接口，虚拟CPU等。虚拟机还包括可以与该虚拟硬件通信的来宾内核。

The hypervisor can be hosted, which means it is some software that runs on the Host OS (MacOS) as in the example. It can also be bare metal, running directly on the machine hardware (replacing your OS). Either way, the hypervisor approach is considered heavy weight as it requires virtualizing multiple parts if not all of the hardware and kernel.

可以托管虚拟机管理程序，这意味着它是某些在示例示例中在主机OS(MacOS)上运行的软件。它也可以是裸机，直接在机器硬件上运行(替换您的OS)。无论哪种方式，系统管理程序方法都被视为繁重的工作，因为它需要虚拟化多个部分(如果不是全部的硬件和内核)。

When there needs to be multiple isolated groups on the same machine, running a VM for each of these groups is way too heavy and wasteful of resources to be a good approach.

当同一台计算机上需要有多个隔离的组时，为这些组中的每个组运行VM太繁琐且浪费资源，因此不是一个好的方法。

VMs require hardware virtualization for machine level isolation whereas containers operate on isolation within the same operation system. The overhead difference becomes really apparent as the number of isolated spaces increase. A regular laptop can run tens of containers but can struggle to run even one VM well.

VM需要硬件虚拟化以实现计算机级别的隔离，而容器则在同一操作系统内以隔离方式运行。随着隔离空间数量的增加，开销的差异变得非常明显。一台普通的笔记本电脑可以运行数十个容器，但是即使要很好地运行一个虚拟机，也很难。

小组 (cgroups)

In 2006, engineers at Google invented the Linux “control groups”, abbreviated as cgroups. This is a feature of the Linux kernel that isolates and controls the resource usage for user processes.

2006年，Google的工程师发明了Linux“控制组”，缩写为cgroup 。这是Linux内核的功能，可隔离和控制用户进程的资源使用情况。

These processes can be put into namespaces, essentially collections of processes that share the same resource limitations. A computer can have multiple namespaces, each with the resource properties enforced by the kernel.

这些进程可以放在命名空间中 ，本质上是共享相同资源限制的进程的集合。一台计算机可以有多个命名空间，每个命名空间都具有内核强制执行的资源属性。

The resource allocation per namespace can be managed in order to limit the amount of the overall CPU, RAM, etc that a set of processes can use. For example, a background log aggregation application will probably need to have its resources limit in order to not accidentally overwhelm the actual server it’s logging.

可以管理每个名称空间的资源分配，以限制一组进程可以使用的总体CPU，RAM等的数量。例如，后台日志聚合应用程序可能需要具有其资源限制，以免意外使它正在记录的实际服务器不堪重负。

While not an original feature, cgroups in Linux were eventually reworked to include a feature called namespace isolation. The idea of namespace isolation itself is not new, and Linux already had many kinds of namespace isolation. One common example is process isolation, which separates each individual process and prevents such things like shared memory.

虽然不是原始功能，但Linux中的cgroup最终经过了重新设计，以包含称为名称空间隔离的功能。名称空间隔离的思想本身并不是什么新鲜事物，Linux已经有了许多种名称空间隔离。一个常见的示例是进程隔离，它隔离每个单独的进程并防止诸如共享内存之类的事情。

Cgroup isolation is a higher level of isolation that makes sure processes within a cgroup namespace are independent of processes in other namespaces. A few important namespace isolation features are outlined below and pave the foundation for the isolation we expect from containers.

Cgroup隔离是更高级别的隔离，可确保cgroup命名空间中的进程独立于其他命名空间中的进程。下面概述了一些重要的名称空间隔离功能，这些功能为我们期望与容器隔离提供了基础。

PID (Process Identifier) Namespaces: this ensures that processes within one namespace are not aware of process in other namespaces.
PID(进程标识符)命名空间：这可确保一个命名空间中的进程不知道其他命名空间中的进程。
Network Namespaces: Isolation of the network interface controller, iptables, routing tables, and other lower level networking tools.
网络命名空间：网络接口控制器，iptables，路由表和其他较低级别的网络工具的隔离。
Mount Namespaces: Filesystems are mounted, so that the file system scope of a namespace is limited to only the directories mounted.
挂载命名空间：挂载文件系统，以便命名空间的文件系统范围仅限于挂载的目录。
User Namespaces: Limits users within a namespace to only that namespace and avoids user ID conflicts across namespaces.
用户命名空间：将命名空间中的用户限制为仅该命名空间，并避免跨命名空间的用户ID冲突。

To put it simply, each namespace would appear to be its own machine to the processes within it.

简单来说，每个名称空间对于其中的进程而言似乎都是其自己的机器。

Linux容器 (Linux Containers)

Linux cgroups paved the way for a technology called linux containers (LXC). LXC was really the first major implementation of what we know today to be a container, taking advantage of cgroups and namespace isolation to create virtual environment with separate process and networking space.

Linux cgroup为称为linux容器 (LXC)的技术铺平了道路。 LXC实际上是我们今天所知的第一个主要实现，它是一个容器，它利用cgroup和名称空间隔离来创建具有独立进程和网络空间的虚拟环境。

In a sense, this allows for independent and isolated user spaces. The idea of containers follows directly from LXC. In fact, earlier versions of Docker were built directly on top of LXC.

从某种意义上讲，这允许独立且隔离的用户空间 。容器的思想直接来自LXC。实际上，早期版本的Docker是直接在LXC之上构建的。

码头工人 (Docker)

Docker is the most widely used container technology and really what most people mean when they refer to containers. While there are other open source container techs (like rkt by CoreOS) and large companies that build their own container engine (like lmctfy at Google), Docker has become the industry standard for containerization. It is still built on the cgroups and namespacing provided by the Linux kernel and recently Windows as well.

Docker是使用最广泛的容器技术，实际上，大多数人指的是容器。尽管还有其他开源容器技术(例如CoreOS的rkt )和大型公司构建自己的容器引擎(例如Google的lmctfy )，但Docker已成为容器化的行业标准。它仍然基于Linux内核以及最近的Windows提供的cgroup和命名空间。

A Docker container is made up of layers of images, binaries packed together into a single package. The base image contains the operating system of the container, which can be different from the OS of the host.

Docker容器由图像层组成，二进制文件打包在一起形成一个包。基本映像包含容器的操作系统，该操作系统可能与主机的OS不同。

The OS of the container is in the form an image. This is not the full operating system as on the host, and the difference is that the image is just the file system and binaries for the OS while the full OS includes the file system, binaries, and the kernel.

容器的操作系统采用映像形式。这不是主机上的完整操作系统，区别在于映像只是OS的文件系统和二进制文件，而完整OS包括文件系统，二进制文件和内核。

On top of the base image are multiple images that each build a portion of the container. For example, on top of the base image may be the image that contains the apt-get dependencies. On top of that may be the image that contains the application binary, and so on.

在基础图像之上是多个图像，每个图像都构成了容器的一部分。例如，在基本映像的顶部可能是包含apt-get依赖项的映像。最重要的是包含应用程序二进制文件的映像，依此类推。

The cool part is if there are two containers with the image layers a, b, c and a, b, d, then you only need to store one copy of each image layer a, b, c, d both locally and in the repository. This is Docker’s union file system.

最酷的部分是，如果有两个容器分别具有图像层a, b, c和a, b, d ，那么您只需要在本地和存储库中存储每个图像层a, b, c, d一个副本。这是Docker的联合文件系统 。

Each image, identified by a hash, is just one of many possible layers of images that make up a container. However a container is identified only by its top level image, which has references to parent images. Two top level images (Image 1 and Image 2) shown here share the first three layers. Image 2 has two additional configuration related layers, but shares the same parent images as Image 1.

每个由哈希标识的图像只是构成容器的许多可能的图像层之一。但是，容器仅由其顶层图像标识，该顶层图像引用了父图像。此处显示的两个顶级图像(图像1和图像2)共享前三层。映像2具有两个附加的与配置相关的层，但与映像1共享相同的父映像。

When a container is booted, the image and its parent images are downloaded from the repo, the cgroup and namespaces are created, and the image is used to create a virtual environment. From within the container, the files and binaries specified in the image appear to be the only files in the entire machine. Then the container’s main process is started and the container is considered alive.

启动容器时，将从存储库中下载映像及其父映像，创建cgroup和名称空间，然后使用该映像创建虚拟环境。从容器内部，映像中指定的文件和二进制文件似乎是整个计算机中唯一的文件。然后，启动容器的主要过程，并认为该容器处于活动状态。

Docker has some other really really cool features, such as copy on write, volumes (shared file systems between containers), the docker daemon (manages containers on a machine), version controlled repositories (like Github for containers), and more. To learn more about them and see some practical examples of how to use Docker, this Medium article is extremely useful.

Docker还有其他一些非常酷的功能，例如写入时复制，卷(容器之间的共享文件系统)，docker守护进程(管理机器上的容器)，版本控制的存储库(例如Github用于容器)等等。要了解有关它们的更多信息并查看一些有关如何使用Docker的实际示例，这篇中型文章非常有用。

为什么选择集装箱 (Why Containers)

Aside from process isolation, containers have many other beneficial properties.

除了过程隔离之外，容器还具有许多其他有益的特性。

The container serves as a self isolated unit that can run anywhere that supports it. And in each of these instances, the container itself will be exactly identical. It won’t matter if the host OS is CentOS, Ubuntu, MacOS, or even something non UNIX like Windows — from within the container the OS will be whatever OS the container specified. Thus you can be sure the container you built on your laptop will also run on the company’s servers.

容器是一个自我隔离的单元，可以在支持它的任何位置运行。在每种情况下，容器本身都是完全相同的。主机操作系统是CentOS，Ubuntu，MacOS还是什至是非UNIX之类的Windows，都无所谓-从容器内部，操作系统将是容器指定的任何操作系统。因此，您可以确定在笔记本电脑上构建的容器也将在公司的服务器上运行。

The container also acts as a standardized unit of work or compute. A common paradigm is for each container to run a single web server, a single shard of a database, or a single Spark worker, etc. Then to scale an application, you simply need to scale the number of containers.

容器还充当工作或计算的标准化单位。一个常见的范例是，每个容器运行单个Web服务器，数据库的单个碎片或单个Spark工作者等。然后要扩展应用程序，只需要扩展容器的数量即可。

In this paradigm, each container is given a fixed resource configuration (CPU, RAM, # of threads, etc) and scaling the application requires scaling just the number of containers instead of the individual resource primitives. This provides a much easier abstraction for engineers when applications need to be scaled up or down.

在此范例中，为每个容器分配了固定的资源配置(CPU，RAM，线程数等)，并且缩放应用程序仅需要缩放容器的数量，而不是单个资源原语。当需要按比例放大或缩小应用程序时，这为工程师提供了更容易的抽象。

Containers also serve as a great tool to implement micro service architecture, where each microservice is just a set of co-operating containers. For example the Redis micro service can be implemented with a single primary container and multiple replica containers.

容器也是实现微服务体系结构的好工具，其中每个微服务只是一组协作容器。例如，Redis微服务可以通过单个主容器和多个副本容器来实现。

This (micro)service orientated architecture has some very important properties that make it easy for engineering teams to create and deploy applications (see my earlier article for more details).

这种面向(微)服务的体系结构具有一些非常重要的属性，这些属性使工程团队可以轻松创建和部署应用程序(有关更多详细信息，请参阅我之前的文章 )。

编排 (Orchestration)

Ever since the time of linux containers, users have tried to deploy large scale applications over many virtual machines where each process runs in its own container. Doing this required being able to efficiently deploy tens to thousands of containers across potentially hundreds of virtual machines and manage their networking, file systems, resources, etc. Docker today makes this a little easier as it exposes abstractions to define container networking, volumes for file systems, resource configurations, etc.

自从使用Linux容器以来，用户一直试图在许多虚拟机上部署大型应用程序，其中每个进程都在其自己的容器中运行。为此，必须能够在潜在的数百个虚拟机中有效地部署数以万计的容器，并管理其网络，文件系统，资源等。如今，Docker公开了定义容器网络，文件卷的抽象概念，使这一过程变得容易一些系统，资源配置等

However a tool is still needed to:

但是，仍然需要一种工具来：

actually take a specification and assign containers to machines (scheduling)
实际采用规格并将容器分配给机器(计划)
actually boot the specified containers on the machines through Docker
实际上通过Docker引导机器上的指定容器
deal with upgrades/rollbacks/the constantly changing nature of the system
处理升级/回滚/系统不断变化的性质
respond to failures like container crashes
应对容器崩溃等故障
and create cluster resources like service discovery, inter VM networking, cluster ingress/egress, etc.
并创建群集资源，例如服务发现，VM间网络连接，群集入口/出口等。

This set of problems relates to the orchestration of a distributed system built on top of a set of (possibly transient or constantly changing) containers, and people have built some really miraculous systems to solve this problem.

这一系列问题与在一组(可能是瞬态或不断变化的)容器之上构建的分布式系统的编排有关，人们已经构建了一些真正神奇的系统来解决此问题。

In my next story I will talk in depth about the implementation of Kubernetes, the major open source orchestrator, along with two equally important but lesser known ones, Mesos and Borg.

在我的下一个故事中，我将深入探讨主要的开源编排Kubernetes的实现，以及同等重要但鲜为人知的两个Mesos和Borg。

This story is part of a series. I am an undergrad at UC Berkeley. My research is in distributed systems and I am advised by Scott Shenker.

这个故事是系列的一部分。我是加州大学伯克利分校的本科生。我的研究是在分布式系统上，我得到了Scott Shenker的建议。