Docker Overlay网络的MTU

原文地址Docker Daemon生产环境配置提到了MTU设置，但是这只是针对于名为bridge的docker bridge network，对于overlay network是无效的。如果docker host machine的网卡MTU为1500，则不需要此步骤设置ingress和docker_gwbridge的MTU以下步骤得在swarm init或join之前做假设你有三个机器，manager、worker-1、worker-2，准备搞一个Docker swarm集群1) [manager] docker swarm init2) [manager] 获得docker_gwbridge的参数，注意Subnet$ docker network inspect docker_gwbridge[ { “Name”: “docker_gwbridge”, … “IPAM”: { … “Config”: [ { “Subnet”: “172.18.0.0/16”, … } ] }, … }]3) [manager] docker swarm leave –force4) [manager] 停掉docker sudo systemctl stop docker.service5) [manager] 删掉虚拟网卡docker_gwbridge$ sudo ip link set docker_gwbridge down$ sudo ip link del dev docker_gwbridge6) [manager] 启动docker sudo systemctl start docker.service7) [manager] 重建docker_gwbridge，记得设置之前得到的Subnet参数和正确的MTU值$ docker network rm docker_gwbridge$ docker network create \ –subnet 172.18.0.0/16 \ –opt com.docker.network.bridge.name=docker_gwbridge \ –opt com.docker.network.bridge.enable_icc=false \ –opt com.docker.network.bridge.enable_ip_masquerade=true \ –opt com.docker.network.driver.mtu=1450 \ docker_gwbridge再到worker-1和worker-2上执行相同的命令。8) [manager] docker swarm init9) [manager] 先观察ingress network的参数，注意Subnet和Gateway：$ docker network inspect ingress[ { “Name”: “ingress”, … “IPAM”: { … “Config”: [ { “Subnet”: “10.255.0.0/16”, “Gateway”: “10.255.0.1” } ] }, … }]10) [manager] 删除ingress network，docker network rm ingress。11) [manager] 重新创建ingress network，记得填写之前得到的Subnet和Gateway，以及正确的MTU值：$ docker network create \ –driver overlay \ –ingress \ –subnet=10.255.0.0/16 \ –gateway=10.255.0.1 \ –opt com.docker.network.driver.mtu=1450 \ ingress12) [worker-1] [worker-2] join docker swarm join …注意：新机器在join到swarm之前，得先执行第7步验证：1) 启动一个swarm service，docker service create -td –name busybox busybox2) 观察虚拟网卡发现MTU都是1450：$ ip link1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:002: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether fa:16:3e:71:09:f5 brd ff:ff:ff:ff:ff:ff3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default link/ether 02:42:6b🇩🇪95:71 brd ff:ff:ff:ff:ff:ff298: docker_gwbridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default link/ether 02:42:ae:7b💿b4 brd ff:ff:ff:ff:ff:ff309: veth7e0f9e5@if308: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master docker_gwbridge state UP mode DEFAULT group default link/ether 16:ca:8f:c7:d3:7f brd ff:ff:ff:ff:ff:ff link-netnsid 1311: vethcb94fec@if310: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master docker0 state UP mode DEFAULT group default link/ether 9a:aa🇩🇪7b:4f:d4 brd ff:ff:ff:ff:ff:ff link-netnsid 23) 观察容器内网卡网卡MTU也是1450：$ docker exec b.1.pdsdgghzyy5rhqkk5et59qa3o ip link1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00310: eth0@if311: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff自建overlay network的MTU方法一：在docker compose file设置…networks: my-overlay: driver: bridge driver_opts: com.docker.network.driver.mtu: 1450不过这样不好，因为这样就把docker compose file的内容和生产环境绑定了，换了个环境这个MTU值未必合适。方法二：外部创建时设置docker network create \ -d overlay \ –opt com.docker.network.driver.mtu=1450 \ –attachable \ my-overlay用法：在docker compose file里这样用：…networks: app-net: external: true name: my-overlaydocker run –network my-overlay …docker service create –network my-overlay …参考资料Use overlay networksDocker MTU issues and solutionsdocker network create