镜像加速

编者按：本文具体介绍蚂蚁平安科技应用龙蜥社区技术进行镜像减速的实际过程，能够让您理解如何基于龙蜥社区推出的容器镜像，Nydus 与 Dragonfly 镜像减速技术和 LifseaOS 为容器的启动减速。文章转自金融级分布式架构，以下为全文： 01 背景简介ZOLOZ是龙蜥社区理事单位蚂蚁团体旗下的寰球平安风控平台，通过业内当先的生物辨认、大数据分析和人工智能技术，为用户和机构提供平安又便捷的平安风控解决方案。ZOLOZ 已为中国、印尼、马来西亚、菲律宾等 14 个国家和地区的 70 余家合作伙伴提供数字化转型过程中的平安风控技术支持。目前，曾经笼罩金融、保险、证券、信贷、电信、公众服务等畛域，累计服务用户超 12 亿。 随着 Kubernetes 和云原生的大暴发，ZOLOZ 利用开始在私有云上进行大规模容器化部署。ZOLOZ 业务的镜像通过长期保护和更新，无论是镜像层数还是整体大小都达到了一个较大的量级（数百 MB 或者几个 GB）。特地是 ZOLOZ AI 算法推理利用的根底镜像大小要远大于个别利用镜像（Docker Hub 上 PyTorch/PyTorch：1.13.1-CUDA 11.6-cuDNN 8-Runtime 有 4.92GB，同比 CentOS：latest 只有约 234MB），对于容器冷启动，即在本地无镜像的状况下，须要先从 Registry 下载镜像能力创立容器，在生产环境中，容器的冷启动往往耗时数分钟，并且随规模扩大会导致 Registry 因集群内网络拥挤而无奈疾速地下载镜像，如此宏大的镜像给利用的更新和扩容等操作都带来了不少挑战。在私有云上容器化继续推动的当下，ZOLOZ 利用次要遇到了三大挑战： 1.算法镜像大，推送到云上镜像仓库耗时长，开发过程中，在应用测试环境进行测试时，往往心愿疾速迭代，疾速验证，然而每次改完一个分支公布验证都要通过几十分钟，开发效率非常低下。 2.拉取算法镜像耗时长，在集群扩容大量机器拉取镜像文件会容易导致集群网卡被打满，影响业务失常运行。 3.集群机器拉起工夫长，难以满足流量突增时，弹性主动扩缩容。 尽管也尝试过各种折中的解决方案，但这些计划都有缺点，咱们当初联合蚂蚁、阿里云、字节跳动等多个技术团队打造了一套更通用的私有云上解决方案，该计划革新成本低，性能好，其中大部分技术都在龙蜥社区中发动、倒退、开源，目前看来是比拟现实的计划。 02 术语及定义OCI：Open Container Initiative，凋谢容器打算是一个 Linux 基金会我的项目，由 Docker 在 2015 年 6 月启动，旨在为操作系统级虚拟化（最重要的是 Linux 容器）设计凋谢规范。 OCI Manifest：遵循 OCI Image Spec 的制品。 BuildKit：是 Docker 公司出品的一款更高效、Docekrfile 无关、更符合云原生利用的新一代 Docker 构建工具。 ...

由于不能说的原因，安装部分包时经常断掉，或者非常缓慢，在没有科学上网的条件下，选择镜像就是非常靠谱的办法 临时使用阿里的镜像pip install -i https://mirrors.aliyun.com/py... some-package清华的镜像（推荐教育网使用）pip install -i https://pypi.tuna.tsinghua.ed... some-package注意，simple 不能少, 是 https 而不是 http 全局设置设为默认 pip config set global.index-url https://mirrors.aliyun.com/py...或修改对应的文件~/.pip/pip.conf或%HOMEPATH%\pip\pip.ini [global] trusted-host = mirrors.aliyun.com index-url = https://mirrors.aliyun.com/pypi/simple

此处仅列出了常用的一些，还有很多没有列出来，需要的话可以从文章最下方链接中查找相应的国内镜像Ubuntuaptvim /etc/apt/sources.list清华源16.04 LTS# 默认注释了源码镜像以提高 apt update 速度，如有需要可自行取消注释deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial main restricted universe multiverse# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial main restricted universe multiversedeb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial-updates main restricted universe multiverse# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial-updates main restricted universe multiversedeb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial-backports main restricted universe multiverse# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial-backports main restricted universe multiversedeb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial-security main restricted universe multiverse# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial-security main restricted universe multiverse# 预发布软件源，不建议启用# deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial-proposed main restricted universe multiverse# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ xenial-proposed main restricted universe multiverse18.04 LTS# 默认注释了源码镜像以提高 apt update 速度，如有需要可自行取消注释deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic main restricted universe multiverse# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic main restricted universe multiversedeb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-updates main restricted universe multiverse# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-updates main restricted universe multiversedeb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-backports main restricted universe multiversedeb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-security main restricted universe multiverse# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-security main restricted universe multiverse# 预发布软件源，不建议启用# deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse18.10# 默认注释了源码镜像以提高 apt update 速度，如有需要可自行取消注释deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ cosmic main restricted universe multiverse# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ cosmic main restricted universe multiversedeb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ cosmic-updates main restricted universe multiverse# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ cosmic-updates main restricted universe multiversedeb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ cosmic-backports main restricted universe multiverse# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ cosmic-backports main restricted universe multiversedeb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ cosmic-security main restricted universe multiverse# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ cosmic-security main restricted universe multiverse# 预发布软件源，不建议启用# deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ cosmic-proposed main restricted universe multiverse# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ cosmic-proposed main restricted universe multiverse其他版本详见 https://mirror.tuna.tsinghua.edu.cn/help/ubuntu/中科大源 https://mirrors.ustc.edu.cn/help/ubuntu.html阿里源 https://opsx.alibaba.com/mirror 找到 Ubuntu，点击帮助网易源 https://mirrors.163.com/.help/ubuntu.html 不全CentOSyum/etc/yum.repos.d/CentOS-Base.repo清华源CentOS 7# CentOS-Base.repo## The mirror system uses the connecting IP address of the client and the# update status of each mirror to pick mirrors that are updated to and# geographically close to the client. You should use this for CentOS updates# unless you are manually picking other mirrors.## If the mirrorlist= does not work for you, as a fall back you can try the# remarked out baseurl= line instead.##[base]name=CentOS-$releasever - Basebaseurl=https://mirrors.tuna.tsinghua.edu.cn/centos/$releasever/os/$basearch/#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=osgpgcheck=1gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7#released updates[updates]name=CentOS-$releasever - Updatesbaseurl=https://mirrors.tuna.tsinghua.edu.cn/centos/$releasever/updates/$basearch/#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updatesgpgcheck=1gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7#additional packages that may be useful[extras]name=CentOS-$releasever - Extrasbaseurl=https://mirrors.tuna.tsinghua.edu.cn/centos/$releasever/extras/$basearch/#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extrasgpgcheck=1gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7#additional packages that extend functionality of existing packages[centosplus]name=CentOS-$releasever - Plusbaseurl=https://mirrors.tuna.tsinghua.edu.cn/centos/$releasever/centosplus/$basearch/#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplusgpgcheck=1enabled=0gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7CentOS 6# CentOS-Base.repo## The mirror system uses the connecting IP address of the client and the# update status of each mirror to pick mirrors that are updated to and# geographically close to the client. You should use this for CentOS updates# unless you are manually picking other mirrors.## If the mirrorlist= does not work for you, as a fall back you can try the# remarked out baseurl= line instead.##[base]name=CentOS-$releasever - Basebaseurl=https://mirrors.tuna.tsinghua.edu.cn/centos/$releasever/os/$basearch/#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=osgpgcheck=1gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6#released updates[updates]name=CentOS-$releasever - Updatesbaseurl=https://mirrors.tuna.tsinghua.edu.cn/centos/$releasever/updates/$basearch/#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updatesgpgcheck=1gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6#additional packages that may be useful[extras]name=CentOS-$releasever - Extrasbaseurl=https://mirrors.tuna.tsinghua.edu.cn/centos/$releasever/extras/$basearch/#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extrasgpgcheck=1gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6#additional packages that extend functionality of existing packages[centosplus]name=CentOS-$releasever - Plusbaseurl=https://mirrors.tuna.tsinghua.edu.cn/centos/$releasever/centosplus/$basearch/#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplusgpgcheck=1enabled=0gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6#contrib - packages by Centos Users[contrib]name=CentOS-$releasever - Contribbaseurl=https://mirrors.tuna.tsinghua.edu.cn/centos/$releasever/contrib/$basearch/#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=contribgpgcheck=1enabled=0gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6CentOS 5# CentOS-Base.repo## The mirror system uses the connecting IP address of the client and the# update status of each mirror to pick mirrors that are updated to and# geographically close to the client. You should use this for CentOS updates# unless you are manually picking other mirrors.## If the mirrorlist= does not work for you, as a fall back you can try the# remarked out baseurl= line instead.##[base]name=CentOS-$releasever - Basebaseurl=https://mirrors.tuna.tsinghua.edu.cn/centos/$releasever/os/$basearch/#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=osgpgcheck=1gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5#released updates[updates]name=CentOS-$releasever - Updatesbaseurl=https://mirrors.tuna.tsinghua.edu.cn/centos/$releasever/updates/$basearch/#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updatesgpgcheck=1gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5#additional packages that may be useful[extras]name=CentOS-$releasever - Extrasbaseurl=https://mirrors.tuna.tsinghua.edu.cn/centos/$releasever/extras/$basearch/#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extrasgpgcheck=1gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5#packages used/produced in the build but not released[addons]name=CentOS-$releasever - Addonsbaseurl=https://mirrors.tuna.tsinghua.edu.cn/centos/$releasever/addons/$basearch/#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=addonsgpgcheck=1gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5#additional packages that extend functionality of existing packages[centosplus]name=CentOS-$releasever - Plusbaseurl=https://mirrors.tuna.tsinghua.edu.cn/centos/$releasever/centosplus/$basearch/#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplusgpgcheck=1enabled=0gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5#contrib - packages by Centos Users[contrib]name=CentOS-$releasever - Contribbaseurl=https://mirrors.tuna.tsinghua.edu.cn/centos/$releasever/contrib/$basearch/#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=contribgpgcheck=1enabled=0gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5清华源 https://mirror.tuna.tsinghua.edu.cn/help/centos/中科大 https://mirrors.ustc.edu.cn/help/centos.html阿里源 https://opsx.alibaba.com/mirrorDocker安装清华源提供 Debian/Ubuntu/Fedora/CentOS/RHEL 的 docker 软件包Ubuntusudo apt-get remove docker docker-engine docker.iosudo apt-get install apt-transport-https ca-certificates curl gnupg2 software-properties-commoncurl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -sudo add-apt-repository \ “deb [arch=amd64] https://mirrors.tuna.tsinghua.edu.cn/docker-ce/linux/ubuntu \ $(lsb_release -cs) \ stable"sudo apt-get updatesudo apt-get install docker-ce其他看 https://mirror.tuna.tsinghua.edu.cn/help/docker-ce/中科大 https://mirrors.ustc.edu.cn/help/docker-ce.htmlDocker HubDocker 中国 https://www.docker-cn.com/registry-mirror中科大https://mirrors.ustc.edu.cn/help/dockerhub.html国内 docker 仓库镜像对比Linux对于使用 upstart 的系统（Ubuntu 14.04、Debian 7 Wheezy），在配置文件 /etc/default/docker 中的 DOCKER_OPTS 中配置Hub地址：DOCKER_OPTS=”–registry-mirror=https://docker.mirrors.ustc.edu.cn/“重新启动服务:sudo service docker restart对于使用 systemd 的系统（Ubuntu 16.04+、Debian 8+、CentOS 7）， 在配置文件 /etc/docker/daemon.json 中加入：{ “registry-mirrors”: [“https://docker.mirrors.ustc.edu.cn/”]}重新启动 dockerd：sudo systemctl restart dockermacOS打开 “Docker.app”进入偏好设置页面(快捷键 ⌘, )打开 “Daemon” 选项卡在 “Registry mirrors” 中添加 https://docker.mirrors.ustc.e…点击下方的 “Apply & Restart” 按钮Windows在系统右下角托盘 Docker 图标内右键菜单选择 Settings ，打开配置窗口后左侧导航菜单选择 Daemon 。在 Registry mirrors 一栏中填写地址 https://docker.mirrors.ustc.e… ，之后点击 Apply 保存后 Docker 就会重启并应用配置的镜像地址了。检查 Docker Hub 是否生效在命令行执行 docker info ，如果从结果中看到了如下内容，说明配置成功。Registry Mirrors: https://docker.mirrors.ustc.edu.cn/Anaconda下载安装清华 https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/中科大 https://mirrors.ustc.edu.cn/anaconda/archive/Condahttps://mirror.tuna.tsinghua.edu.cn/help/anaconda/https://mirrors.ustc.edu.cn/help/anaconda.htmlconda config –add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/conda config –add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/conda config –set show_channel_urls yesHomebrew清华https://mirror.tuna.tsinghua.edu.cn/help/homebrew/替换现有上游cd “$(brew –repo)“git remote set-url origin https://mirrors.tuna.tsinghua.edu.cn/git/homebrew/brew.gitcd “$(brew –repo)/Library/Taps/homebrew/homebrew-core"git remote set-url origin https://mirrors.tuna.tsinghua.edu.cn/git/homebrew/homebrew-core.gitbrew update使用homebrew-science或者homebrew-pythoncd “$(brew –repo)/Library/Taps/homebrew/homebrew-science"git remote set-url origin https://mirrors.tuna.tsinghua.edu.cn/git/homebrew/homebrew-science.gitcd “$(brew –repo)/Library/Taps/homebrew/homebrew-python"git remote set-url origin https://mirrors.tuna.tsinghua.edu.cn/git/homebrew/homebrew-python.gitbrew updatehttps://mirror.tuna.tsinghua.edu.cn/help/homebrew-bottles/该镜像是 Homebrew 二进制预编译包的镜像。本镜像站同时提供 Homebrew 的 formula 索引的镜像（即 brew update 时所更新内容）临时替换export HOMEBREW_BOTTLE_DOMAIN=https://mirrors.tuna.tsinghua.edu.cn/homebrew-bottles长期替换echo ’export HOMEBREW_BOTTLE_DOMAIN=https://mirrors.tuna.tsinghua.edu.cn/homebrew-bottles’ >> ~/.bash_profilesource /.bash_profile中科大https://mirrors.ustc.edu.cn/help/brew.git.htmlhttps://mirrors.ustc.edu.cn/help/homebrew-bottles.htmlhttps://mirrors.ustc.edu.cn/help/homebrew-core.git.htmlhttps://mirrors.ustc.edu.cn/help/homebrew-cask.git.htmlMaven/.m2/settings.xml<?xml version=“1.0” encoding=“UTF-8”?><settings xmlns=“http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=“http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"> <mirrors> <mirror> <id>alimaven</id> <name>aliyun maven</name> <url>https://maven.aliyun.com/repository/central</url> <mirrorOf>central</mirrorOf> </mirror> </mirrors></settings> 镜像站汇总清华大学开源软件镜像站 - Tsinghua Open Source Mirror中国科学技术大学开源软件镜像淘宝 NPM 镜像网易开源镜像站阿里巴巴开源镜像站搜狐开源镜像站Capital Online Mirror Site北京理工大学开源软件镜像站北京交通大学开源软件镜像站兰州大学开源社区镜像站浙江大学开源镜像站Northeastern University OpenSource Mirrors华中科技大学开源镜像站重庆大学开源软件镜像站东软信息学院自己的开源镜像站 ...