1 需要剖析

  • 公有开发源：开发团队须要不便的python公有包公布机制
  • 公有镜像源：自建官网源镜像，晋升访问速度，躲避偶尔网络问题，不便离线环境的私有化部署

2 应用Docker部署PypiServer服务器

2.1 下载PypiServer镜像

docker pull pypiserver/pypiserver

2.2 生成Auth信息

# 装置依赖apt-get install -y apache2-utilssudo pip3 install passlib # 生成 htpass 文件mkdir -p /opt/pypiserver/auth /opt/pypiserver/packages# 示意所有用户都能够读写但不能执行文件/文件夹chmod -R 666 /opt/pypiserver/packages# 会 prompt 明码输出，反复两遍一样的cd /opt/pypiserver/auth && htpasswd -sc .htaccess ${username}

2.3 容器部署

docker run -d \-p ${port}:8080 \ --restart=always \ --name=pypiserver \ -v /opt/pypiserver/packages/:/data/packages \-v /opt/pypiserver/auth:/data/auth/ \ pypiserver/pypiserver -P /data/auth/.htaccess -a update /data/packages

2.4 Nginx反向代理

  • 应用Docker部署Nginx服务，同时提供HTTPS反对

    echo 'server {listen 80;server_name ${sever_name];rewrite ^(.*)$ https://${server_name}$1 permanent;}server {listen 443 ssl;server_name ${server_name};#ssl证书文件地位(常见证书文件格式为：crt/pem)ssl_certificate /etc/nginx/ssl/ps-cert.pem;#ssl证书key地位ssl_certificate_key /etc/nginx/ssl/ps-cert.key;ssl_session_timeout 10m;ssl_protocols TLSv1 TLSv1.1 TLSv1.2;ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4;ssl_prefer_server_ciphers on;location / {proxy_set_header Host $host;proxy_set_header X-Forwarded-Proto $scheme;proxy_set_header X-Forwarded-For $host;proxy_set_header X-Real-IP $remote_addr;# 此处能够应用frp做穿透，将内网的服务映射到公网上proxy_pass http://${public_ip}:${port};}}' >> /opt/pypi/pypi.conf

  • 部署Nginx容器

    docker run -d \--restart always \-v /opt/pypi/pypi.conf:/etc/nginx/conf.d/pypi.conf \-v /opt/pypi/ssl/ps-cert.pem:/etc/nginx/ssl/ps-cert.pem \-v /opt/pypi/ssl/ps-cert.key:/etc/nginx/ssl/ps-cert.key \-p ${port}:80 \--name=pypi_nginxnginx

3 装置bandersnatch本地源同步工具

3.1 本机配置

配置文件

mkdir -p /opt/bandersnatch/log && touch /opt/bandersnatch/bandersnatch.conf /opt/bandersnatch/bandersnatch-log.confecho '[mirror]directory = /opt/bandersnatchjson = falserelease-files = truecleanup = falsemaster = https://pypi.orgtimeout = 10global-timeout = 1800workers = 3hash-index = falsestop-on-error = falsestorage-backend = filesystem;log-config = /opt/bandersnatch/bandersnatch-log.conf; root_uri = https://example.comverifiers = 3;keep_index_versions = 0;vim: set ft=cfg:;diff-file = /srv/pypi/mirrored-files;diff-append-epoch = true[plugins]enabled = all[blacklist]; https://bandersnatch.readthedocs.io/en/latest/filtering_configuration.html; https://pypi.org/stats/[whitelist]packages = cntk tensorflow-gpu tensorflow tensorflow-cpu torch' > /opt/bandersnatch/bandersnatch.conf \&& echo '[loggers]keys=root,file[handlers]keys=root,file[formatters]keys=common[logger_root]level=NOTSEThandlers=root[logger_file]level=INFOhandlers=filepropagate=1qualname=bandersnatch[formatter_common]format=%(asctime)s %(name)-12s: %(levelname)s %(message)s[handler_root]class=StreamHandlerlevel=DEBUGformatter=commonargs=(sys.stdout,) [handler_file]class=handlers.RotatingFileHandlerlevel=INFOformatter=commonargs=('/opt/bandersnatch/log/bandersnatch.log','D',1,'UTF-8')# will manage one file a day' > /opt/bandersnatch/bandersnatch-log.conf

部署容器

docker run -d \--restart=always \ --name=bandersnatch \ -v /opt/bandersnatch/bandersnatch.conf:/etc/bandersnatch.conf \ -v /opt/bandersnatch:/opt/bandersnatch \ pypa/bandersnatch bandersnatch mirror

3.2 nginx反向代理配置

应用Docker部署Nginx服务，nginx配置文件如下

server {listen 80;server_name ${server_name};rewrite ^(.*)$ https://${server_name}$1 permanent;}server {listen 443 ssl;server_name ${server_name};#ssl证书文件地位(常见证书文件格式为：crt/pem)ssl_certificate /etc/nginx/ssl/bs-cert.pem;#ssl证书key地位ssl_certificate_key /etc/nginx/ssl/bs-cert.key;ssl_session_timeout 10m;ssl_protocols TLSv1 TLSv1.1 TLSv1.2;ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4;ssl_prefer_server_ciphers on;location / {proxy_set_header Host $host;proxy_set_header X-Forwarded-Proto $scheme;proxy_set_header X-Forwarded-For $host;proxy_set_header X-Real-IP $remote_addr;# 此处能够应用frp做穿透，将内网的服务映射到公网上proxy_pass http://${public_ip}:${port};}}

5 参考

  • PypiServer

    • https://pypi.org/project/pypi...
    • https://github.com/pypiserver...
    • PypiServer Docker Hub

  • bandersnatch

    • https://hub.docker.com/r/pypa...
    • https://github.com/pypa/bande...
    • Mirror configuration