关于python:pip私有源部署

1 需要剖析

  • 公有开发源:开发团队须要不便的python公有包公布机制
  • 公有镜像源:自建官网源镜像,晋升访问速度,躲避偶尔网络问题,不便离线环境的私有化部署

2 应用Docker部署PypiServer服务器

2.1 下载PypiServer镜像

docker pull pypiserver/pypiserver

2.2 生成Auth信息

# 装置依赖
apt-get install -y apache2-utilssudo pip3 install passlib 
# 生成 htpass 文件
mkdir -p /opt/pypiserver/auth /opt/pypiserver/packages
# 示意所有用户都能够读写但不能执行文件/文件夹
chmod -R 666 /opt/pypiserver/packages
# 会 prompt 明码输出,反复两遍一样的
cd /opt/pypiserver/auth && htpasswd -sc .htaccess ${username}

2.3 容器部署

docker run -d \
-p ${port}:8080 \ 
--restart=always \ --name=pypiserver \ 
-v /opt/pypiserver/packages/:/data/packages \
-v /opt/pypiserver/auth:/data/auth/ \ 
pypiserver/pypiserver -P /data/auth/.htaccess -a update /data/packages

2.4 Nginx反向代理

  • 应用Docker部署Nginx服务,同时提供HTTPS反对

    echo 'server {
    listen 80;
    server_name ${sever_name];
    rewrite ^(.*)$ https://${server_name}$1 permanent;
    }
    server {
    listen 443 ssl;
    server_name ${server_name};
    #ssl证书文件地位(常见证书文件格式为:crt/pem)
    ssl_certificate /etc/nginx/ssl/ps-cert.pem;
    #ssl证书key地位
    ssl_certificate_key /etc/nginx/ssl/ps-cert.key;
    ssl_session_timeout 10m;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4;
    ssl_prefer_server_ciphers on;
    location / {
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-For $host;
    proxy_set_header X-Real-IP $remote_addr;
    # 此处能够应用frp做穿透,将内网的服务映射到公网上
    proxy_pass http://${public_ip}:${port};
    }
    }' >> /opt/pypi/pypi.conf
  • 部署Nginx容器

    【腾讯云】云产品限时秒杀,爆款1核2G云服务器,首年99元
    docker run -d \
    --restart always \
    -v /opt/pypi/pypi.conf:/etc/nginx/conf.d/pypi.conf \
    -v /opt/pypi/ssl/ps-cert.pem:/etc/nginx/ssl/ps-cert.pem \
    -v /opt/pypi/ssl/ps-cert.key:/etc/nginx/ssl/ps-cert.key \
    -p ${port}:80 \
    --name=pypi_nginx
    nginx

3 装置bandersnatch本地源同步工具

3.1 本机配置

配置文件

mkdir -p /opt/bandersnatch/log && touch /opt/bandersnatch/bandersnatch.conf /opt/bandersnatch/bandersnatch-log.conf

echo '[mirror]
directory = /opt/bandersnatchjson = false
release-files = true
cleanup = false
master = https://pypi.org
timeout = 10
global-timeout = 1800
workers = 3hash-index = false
stop-on-error = false
storage-backend = filesystem
;log-config = /opt/bandersnatch/bandersnatch-log.conf
; root_uri = https://example.comverifiers = 3
;keep_index_versions = 0
;vim: set ft=cfg:
;diff-file = /srv/pypi/mirrored-files
;diff-append-epoch = true
[plugins]
enabled = all
[blacklist]
; https://bandersnatch.readthedocs.io/en/latest/filtering_configuration.html
; https://pypi.org/stats/
[whitelist]
packages = 
cntk 
tensorflow-gpu 
tensorflow 
tensorflow-cpu 
torch' > /opt/bandersnatch/bandersnatch.conf \
&& echo '
[loggers]
keys=root,file
[handlers]
keys=root,file
[formatters]
keys=common
[logger_root]
level=NOTSEThandlers=root
[logger_file]
level=INFO
handlers=file
propagate=1qual
name=bandersnatch
[formatter_common]
format=%(asctime)s %(name)-12s: %(levelname)s %(message)s
[handler_root]
class=StreamHandlerlevel=DEBUGformatter=commonargs=(sys.stdout,) 
[handler_file]
class=handlers.Rotating
FileHandlerlevel=INFO
formatter=commonargs=('/opt/bandersnatch/log/bandersnatch.log','D',1,'UTF-8')
# will manage one file a day' > /opt/bandersnatch/bandersnatch-log.conf

部署容器

docker run -d \
--restart=always \ 
--name=bandersnatch \ 
-v /opt/bandersnatch/bandersnatch.conf:/etc/bandersnatch.conf \ 
-v /opt/bandersnatch:/opt/bandersnatch \ 
pypa/bandersnatch bandersnatch mirror

3.2 nginx反向代理配置

应用Docker部署Nginx服务,nginx配置文件如下

server {
listen 80;
server_name ${server_name};
rewrite ^(.*)$ https://${server_name}$1 permanent;
}
server {
listen 443 ssl;
server_name ${server_name};
#ssl证书文件地位(常见证书文件格式为:crt/pem)
ssl_certificate /etc/nginx/ssl/bs-cert.pem;
#ssl证书key地位
ssl_certificate_key /etc/nginx/ssl/bs-cert.key;
ssl_session_timeout 10m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4;
ssl_prefer_server_ciphers on;
location / {
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $host;
proxy_set_header X-Real-IP $remote_addr;
# 此处能够应用frp做穿透,将内网的服务映射到公网上
proxy_pass http://${public_ip}:${port};
}
}

5 参考

  • PypiServer

    • https://pypi.org/project/pypi…
    • https://github.com/pypiserver…
    • PypiServer Docker Hub
  • bandersnatch

    • https://hub.docker.com/r/pypa…
    • https://github.com/pypa/bande…
    • Mirror configuration

阿里云限时活动-2核2G-5M带宽-40-100G SSD服务器,特惠价86元/年(原价724元/年,限时99元续购三次),速抢

本文由乐趣区整理发布,转载请注明出处,谢谢。

You may also like...

发表评论

邮箱地址不会被公开。 必填项已用*标注

此站点使用Akismet来减少垃圾评论。了解我们如何处理您的评论数据