sonic容器构建

jiezi

5 年前

sonic 中大量的组件运行在 docker 容器中，用于隔离彼此的运行环境，从而解决相互之间的互斥问题。下面我们分析一下 sonic 中各个容器的构建过程。

sonic 中的容器 Dockerfile 文件是通过 jinjia2 模板文件生成的，使用 j2 命令。

sonic 在编译过程中首先会构建一个叫 sonic-slave 的容易，该容器用来做编译容器，后续所有的编译过程都是在该容器中进行的。

编译容器使用 slave.mk 作为 makefile，其中有这样一段代码将 Dockerfile.j2 转换成 Dockerfile

# Targets for building docker images
$(addprefix $(TARGET_PATH)/, $(SONIC_DOCKER_IMAGES)) : $(TARGET_PATH)/%.gz : .platform docker-start $$(addprefix $(DEBS_PATH)/,$$($$*.gz_DEPENDS)) $$(addprefix $(FILES_PATH)/,$$($$*.gz_FILES)) $$(addprefix $(PYTHON_WHEELS_PATH)/,$$($$*.gz_PYTHON_WHEELS)) $$(addsuffix -load,$$(addprefix $(TARGET_PATH)/,$$($$*.gz_LOAD_DOCKERS))) $$($$*.gz_PATH)/Dockerfile.j2
        $(HEADER)
        # Apply series of patches if exist
        if [-f $($*.gz_PATH).patch/series ]; then pushd $($*.gz_PATH) && QUILT_PATCHES=../$(notdir $($*.gz_PATH)).patch quilt push -a; popd; fi
        mkdir -p $($*.gz_PATH)/debs $(LOG)
        mkdir -p $($*.gz_PATH)/files $(LOG)
        mkdir -p $($*.gz_PATH)/python-wheels $(LOG)
        sudo mount --bind $(DEBS_PATH) $($*.gz_PATH)/debs $(LOG)
        sudo mount --bind $(FILES_PATH) $($*.gz_PATH)/files $(LOG)
        sudo mount --bind $(PYTHON_WHEELS_PATH) $($*.gz_PATH)/python-wheels $(LOG)
        # Export variables for j2. Use path for unique variable names, e.g. docker_orchagent_debs
        $(eval export $(subst -,_,$(notdir $($*.gz_PATH)))_debs=$(shell printf "$(subst $(SPACE),\n,$(call expand,$($*.gz_DEPENDS),RDEPENDS))\n" | awk '!a[$$0]++'))
        $(eval export $(subst -,_,$(notdir $($*.gz_PATH)))_whls=$(shell printf "$(subst $(SPACE),\n,$(call expand,$($*.gz_PYTHON_WHEELS)))\n" | awk '!a[$$0]++'))
        $(eval export $(subst -,_,$(notdir $($*.gz_PATH)))_dbgs=$(shell printf "$(subst $(SPACE),\n,$(call expand,$($*.gz_DBG_PACKAGES)))\n" | awk '!a[$$0]++'))
        j2 $($*.gz_PATH)/Dockerfile.j2 > $($*.gz_PATH)/Dockerfile  #使用环境变量编译 Dockfile.j2
        docker build --squash --no-cache \
                --build-arg http_proxy=$(HTTP_PROXY) \
                --build-arg https_proxy=$(HTTPS_PROXY) \
                --build-arg user=$(USER) \
                --build-arg uid=$(UID) \
                --build-arg guid=$(GUID) \
                --build-arg docker_container_name=$($*.gz_CONTAINER_NAME) \
                -t $* $($*.gz_PATH) $(LOG)
        docker save $* | gzip -c > $@
        # Clean up
        if [-f $($*.gz_PATH).patch/series ]; then pushd $($*.gz_PATH) && quilt pop -a -f; popd; fi
        $(FOOTER)

编译的过程首先构建一个 docker-base 容器。

docker-base.j2

# 根容器是 debian:jessie
FROM debian:jessie 

#set up proxy 构建 apt 代理，会进行 deb 包缓存，加速编译速度
RUN echo 'Acquire::http {Proxy"http://172.17.0.2:3142";};' >>/etc/apt/apt.conf.d/01proxy
RUN echo 'Acquire::HTTP::Proxy::download.docker.com"Direct";' >>/etc/apt/apt.conf.d/01proxy
RUN echo 'Acquire::HTTP::Proxy::get.docker.io"Direct";' >>/etc/apt/apt.conf.d/01proxy

# Clean documentation in FROM image
RUN find /usr/share/doc -depth \(-type f -o -type l \) ! -name copyright | xargs rm || true

# Clean doc directories that are empty or only contain empty directories
RUN while [-n "$(find /usr/share/doc -depth -type d -empty -print -exec rmdir {} +)" ]; do :; done
RUN rm -rf               \
    /usr/share/man/*     \
    /usr/share/groff/*   \
    /usr/share/info/*    \
    /usr/share/lintian/* \
    /usr/share/linda/*   \
    /var/cache/man/*     \
    /usr/share/locale/*

# Make apt-get non-interactive
ENV DEBIAN_FRONTEND=noninteractive

# Configure data sources for apt/dpkg
COPY ["dpkg_01_drop", "/etc/dpkg/dpkg.cfg.d/01_drop"]
COPY ["sources.list", "/etc/apt/sources.list"]
COPY ["no_install_recommend_suggest", "/etc/apt/apt.conf.d"]
RUN apt-get update

# Pre-install fundamental packages
RUN apt-get -y install \
    rsyslog            \
    vim-tiny           \
    perl               \
    python             \
    less               \
    gdb                \
    lsof               \
    strace             \
    tcpdump            \
    pstack             \
    net-tools          \
    sysstat            \
    dstat              \
    iotop              \
    htop               \
    lrzsz              \
    linux-base         \
    linux.perf         \
    linux-tools-3.16


# Pre-install troubleshooting packages
RUN apt-get -y install socat

COPY ["etc/rsyslog.conf", "/etc/rsyslog.conf"]
COPY ["etc/rsyslog.d/*", "/etc/rsyslog.d/"]
COPY ["root/.vimrc", "/root/.vimrc"]

# Install dependencies of supervisor
RUN apt-get -y install python-pkg-resources python-meld3

RUN mkdir -p /etc/supervisor
RUN mkdir -p /var/log/supervisor

COPY ["etc/supervisor/supervisord.conf", "/etc/supervisor/"]

RUN apt-get -y purge   \
    exim4              \
    exim4-base         \
    exim4-config       \
    exim4-daemon-light
#根据环境变量进行渲染，决定拷贝哪些 deb 到容器中
{% if docker_base_debs.strip() -%}
# Copy built Debian packages
{%- for deb in docker_base_debs.split(' ') %}
COPY debs/{{deb}} debs/
{%- endfor %}

# Install built Debian packages and implicitly install their dependencies
{%- for deb in docker_base_debs.split(' ') %}
RUN dpkg_apt() { [ -f $1] && {dpkg -i $1 || apt-get -y install -f;} || return 1; }; dpkg_apt debs/{{deb}}
{%- endfor %}
{%- endif %}

{% if docker_base_dbgs.strip() -%}
# Install common debug-packages
{%- for dbg_pkg in docker_base_dbgs.split(' ') %}
RUN apt-get -y install {{dbg_pkg}}
{%- endfor %}
{% else %}
RUN ln /usr/bin/vim.tiny /usr/bin/vim
{%- endif %}

# Clean up apt
# Remove /var/lib/apt/lists/*, could be obsoleted for derived images
# 删除所有下载的安装包，删除所有以前下载的安装包，卸载所有没有使用的包
RUN apt-get clean -y; apt-get autoclean -y; apt-get autoremove -y
RUN rm -rf /var/lib/apt/lists/*

RUN rm -rf /tmp/*

渲染后会生成 docker-base/Dockerfile 文件，所以要修改容器构建 Dockerfile 时，不能直接修改，而是修改 Dockerfile

在 docker-base 容器的基础上安装一些 python 工具以及安装一些编译生成的 python 库，该容器没有实际的内容，只是基础容器的功能增强。

docker-config-engine/Dockerfile.j2

# 在 docker-base 容器上构建
FROM docker-base

## Make apt-get non-interactive
ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update

# Dependencies for sonic-cfggen
RUN apt-get install -y python-lxml python-yaml python-bitarray python-pip python-dev python-natsort

RUN pip install --upgrade pip

RUN pip install netaddr ipaddr jinja2 pyangbind==0.5.10


COPY \
python-wheels/swsssdk-2.0.1-py2-none-any.whl python-wheels/sonic_config_engine-1.0-py2-none-any.whl python-wheels/
RUN pip install \
python-wheels/swsssdk-2.0.1-py2-none-any.whl python-wheels/sonic_config_engine-1.0-py2-none-any.whl ## Clean up
RUN apt-get remove -y python-pip python-dev; apt-get clean -y; apt-get autoclean -y; apt-get autoremove -y
RUN rm -rf /debs /python-wheels

构建好了基础镜像后，后面的镜像都在 docker-config-engine 或者 docker-base 的基础上构建，这样的例如 docker-orchagent-bfn，docker-fpm-frr，docker-teamd，docker-database 等。

我们以 docker-orchagent-bfn 为例看一下功能镜像的 Dockerfile 文件:docker-orchagent-bfn/Dockerfile.j2

FROM docker-config-engine

ARG docker_container_name
RUN [-f /etc/rsyslog.conf] && sed -ri "s/%syslogtag%/$docker_container_name\/%syslogtag%/;" /etc/rsyslog.conf

## Make apt-get non-interactive
ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update

RUN apt-get install -f -y ifupdown arping libdbus-1-3 libdaemon0 libjansson4

## Install redis-tools dependencies
## TODO: implicitly install dependencies
RUN apt-get -y install libjemalloc1

COPY \
{% for deb in docker_orchagent_debs.split(' ') -%}
debs/{{deb}}{{' '}}
{%- endfor -%}
debs/

RUN dpkg -i \
{% for deb in docker_orchagent_debs.split(' ') -%}
debs/{{deb}}{{' '}}
{%- endfor %}

## Clean up
RUN apt-get clean -y; apt-get autoclean -y; apt-get autoremove -y
## 镜像构建完毕后，删除 /debs 目录
RUN rm -rf /debs

COPY ["files/arp_update", "/usr/bin"]
COPY ["enable_counters.py", "/usr/bin"]
COPY ["start.sh", "orchagent.sh", "swssconfig.sh", "/usr/bin/"]
COPY ["supervisord.conf", "/etc/supervisor/conf.d/"]

## Copy all Jinja2 template files into the templates folder
COPY ["*.j2", "/usr/share/sonic/templates/"]

ENTRYPOINT ["/usr/bin/supervisord"]

sonic容器构建

sonic 容器构建

Dockerfile 文件的生成过程

编译容器生成 Dockerfile

根容器：docker-base

docker-config-engine

功能镜像