1 背景
在 k8s 中基于 alpine 做底层零碎的容器进行 git lfs push 操作时，发现报错无奈上传胜利
Fatal error: Server error: http://git.ops.xxxxx.com/xxxx/yyyy.git/gitlab-lfs/objects/b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e/3
error: failed to push some refs to 'http://git.ops.xxxx.com/xxxx/yyyy.git'而 git clone、commit 等操作均失常，并且 git lfs push 的操作在另一个机器 A 上能失常执行。
2 摸索思路
2.1 从报错动手
先依据报错信息搜寻，根本答复是此报错是 git server(gitlab) 侧产生谬误，不过能够应用如下命令来打印 trace 跟踪信息 (via: https://github.com/git-lfs/gi…
GIT_TRACE=1 GIT_TRANSFER_TRACE=1 GIT_CURL_VERBOSE=1 git push origin xxx/xxx的确打印了更多信息，包含与 Server 的申请过程，如下：
07:18:13.442791 trace git-lfs: HTTP: POST http://xxxx:xxxx@git.ops.yunnex.com/suncard/packages.git/info/lfs/objects/batch
> POST /suncard/packages.git/info/lfs/objects/batch HTTP/1.1
> Host: git.ops.yunnex.com
> Accept: application/vnd.git-lfs+json; charset=utf-8
> Content-Length: 158
> Content-Type: application/vnd.git-lfs+json; charset=utf-8
> User-Agent: git-lfs/2.5.1 (GitHub; linux amd64; go 1.11.4; git 7c940069fa)
>
{"operation":"upload","objects":[{"oid":"b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e","size":3}],"ref":{"name":"refs/heads/test-branch"}}07:18:13.618655 trace git-lfs: HTTP: 200
< HTTP/1.1 200 OK
< Transfer-Encoding: chunked
< Cache-Control: max-age=0, private, must-revalidate
< Connection: keep-alive
< Content-Type: application/json; charset=utf-8
< Date: Thu, 24 Sep 2020 07:18:13 GMT
< Etag: W/"0fb39c6aeeda2850fdcf244083961713"
< Server: nginx
< Strict-Transport-Security: max-age=31536000
< X-Content-Type-Options: nosniff
< X-Frame-Options: DENY
< X-Request-Id: 570154fd-b15f-4e8a-a735-b034e98dfbfd
< X-Runtime: 0.165423
< X-Ua-Compatible: IE=edge
< X-Xss-Protection: 1; mode=block
<
07:18:13.619071 trace git-lfs: HTTP: {"objects":[{"oid":"b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e","size":3,"actions":{"upload":{"href":"http://git.ops.yunnex.com/suncard/packages.git/gitlab-lfs/objects/b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e/3","header":{"Authorization":"Basic cHJvZHVjdGlvbjpVc2VfMTk5MA=="}}}}]}
{"objects":[{"oid":"b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e","size":3,"actions":{"upload":{"href":"http://git.ops.yunnex.com/suncard/packages.git/gitlab-lfs/objects/b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e/3","header":{"Authorization":"Basic cHJvZHVjdGlvbjpVc2VfMTk5MA=="}}}}]}07:18:13.619307 trace git-lfs: tq: starting transfer adapter "basic"
07:18:13.619345 trace git-lfs: xfer: adapter "basic" Begin() with 8 workers
07:18:13.619383 trace git-lfs: xfer: adapter "basic" started
07:18:13.619413 trace git-lfs: xfer: adapter "basic" worker 1 starting
07:18:13.619537 trace git-lfs: xfer: adapter "basic" worker 1 waiting for Auth
07:18:13.619549 trace git-lfs: xfer: adapter "basic" worker 5 starting
07:18:13.619585 trace git-lfs: xfer: adapter "basic" worker 5 waiting for Auth
07:18:13.619642 trace git-lfs: xfer: adapter "basic" worker 0 starting
07:18:13.619673 trace git-lfs: xfer: adapter "basic" worker 6 starting
07:18:13.619681 trace git-lfs: xfer: adapter "basic" worker 0 processing job for "b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e"
07:18:13.619702 trace git-lfs: xfer: adapter "basic" worker 7 starting
07:18:13.619722 trace git-lfs: xfer: adapter "basic" worker 7 waiting for Auth
07:18:13.619732 trace git-lfs: xfer: adapter "basic" worker 2 starting
07:18:13.619775 trace git-lfs: xfer: adapter "basic" worker 2 waiting for Auth
07:18:13.619723 trace git-lfs: xfer: adapter "basic" worker 4 starting
07:18:13.619775 trace git-lfs: xfer: adapter "basic" worker 3 starting
07:18:13.619805 trace git-lfs: xfer: adapter "basic" worker 3 waiting for Auth
07:18:13.619788 trace git-lfs: xfer: adapter "basic" worker 4 waiting for Auth
07:18:13.619697 trace git-lfs: xfer: adapter "basic" worker 6 waiting for Auth
07:18:13.621028 trace git-lfs: HTTP: PUT http://git.ops.yunnex.com/suncard/packages.git/gitlab-lfs/objects/b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e/3
> PUT /suncard/packages.git/gitlab-lfs/objects/b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e/3 HTTP/1.1
> Host: git.ops.yunnex.com
> Authorization: Basic * * * * *
> Content-Length: 3
> Content-Type: text/plain; charset=utf-8
> User-Agent: git-lfs/2.5.1 (GitHub; linux amd64; go 1.11.4; git 7c940069fa)
>
07:18:13.621492 trace git-lfs: xfer: adapter "basic" worker 7 auth signal received
dd
07:18:13.621527 trace git-lfs: xfer: adapter "basic" worker 5 auth signal received
07:18:13.621536 trace git-lfs: xfer: adapter "basic" worker 4 auth signal received
07:18:13.621563 trace git-lfs: xfer: adapter "basic" worker 2 auth signal received
07:18:13.621571 trace git-lfs: xfer: adapter "basic" worker 3 auth signal received
07:18:13.621579 trace git-lfs: xfer: adapter "basic" worker 6 auth signal received
07:18:13.621522 trace git-lfs: xfer: adapter "basic" worker 1 auth signal received
07:18:13.786463 trace git-lfs: HTTP: 500
< HTTP/1.1 500 Internal Server Error
< Content-Length: 2911
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Connection: keep-alive
< Content-Type: text/html; charset=utf-8
< Date: Thu, 24 Sep 2020 07:18:13 GMT
< Expires: Fri, 01 Jan 1990 00:00:00 GMT
< Pragma: no-cache
< Server: nginx
< X-Request-Id: 0574e8c0-8ee8-4667-8141-1982d67c0bdc
< X-Runtime: 0.162882
<
07:18:13.786829 trace git-lfs: xfer: adapter "basic" worker 0 finished job for "b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e"
07:18:13.786927 trace git-lfs: tq: retrying object b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e: Fatal error: Server error: http://git.ops.yunnex.com/suncard/packages.git/gitlab-lfs/objects/b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e/3
07:18:13.787030 trace git-lfs: tq: enqueue retry #1 for "b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e" (size: 3)
07:18:13.787091 trace git-lfs: tq: sending batch of size 1
07:18:13.787212 trace git-lfs: api: batch 1 files
07:18:13.787330 trace git-lfs: HTTP: POST http://xxxxx:xxxxx@git.ops.yunnex.com/suncard/packages.git/info/lfs/objects/batch
... 一直重试反复雷同日志 其中要害申请如下
200 POST /suncard/packages.git/info/lfs/objects/batch
500 PUT /suncard/packages.git/gitlab-lfs/objects/b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e/3同时到 gitlab server 执行 tail -f /var/log/gitlab//.log，日志如下
==> /var/log/gitlab/gitlab-rails/production.log <==
Started POST "/suncard/packages.git/info/lfs/objects/batch" for 10.10.51.195 at 2020-09-24 15:23:16 +0800
Processing by Projects::LfsApiController#batch as JSON
Parameters: {"operation"=>"upload", "objects"=>[{"oid"=>"b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e", "size"=>3}], "ref"=>{"name"=>"refs/heads/test-branch"}, "namespace_id"=>"suncard", "project_id"=>"packages.git", "lfs_api"=>{"operation"=>"upload", "objects"=>[{"oid"=>"b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e", "size"=>3}], "ref"=>{"name"=>"refs/heads/test-branch"}}}
Completed 200 OK in 138ms (Views: 0.3ms | ActiveRecord: 4.7ms)
Started PUT "/suncard/packages.git/gitlab-lfs/objects/b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e/3" for 10.10.51.195 at 2020-09-24 15:23:17 +0800
Processing by Projects::LfsStorageController#upload_finalize as HTML
Parameters: {"namespace_id"=>"suncard", "project_id"=>"packages.git", "oid"=>"b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e", "size"=>"3"}
Completed 500 Internal Server Error in 104ms (ActiveRecord: 2.6ms)
NoMethodError (undefined method `include?' for nil:NilClass):
app/controllers/projects/lfs_storage_controller.rb:55:in `tmp_filename'app/controllers/projects/lfs_storage_controller.rb:23:in `upload_finalize'
lib/gitlab/i18n.rb:45:in `with_locale'lib/gitlab/i18n.rb:51:in `with_user_locale'
app/controllers/application_controller.rb:294:in `set_locale'lib/gitlab/performance_bar/peek_performance_bar_with_rack_body.rb:16:in `call'
lib/gitlab/middleware/multipart.rb:93:in `call'lib/gitlab/request_profiler/middleware.rb:14:in `call'
lib/gitlab/middleware/go.rb:16:in `call'lib/gitlab/etag_caching/middleware.rb:11:in `call'
lib/gitlab/request_context.rb:18:in `call'lib/gitlab/metrics/connection_rack_middleware.rb:31:in `call'
... 一直重试反复雷同日志
==> /var/log/gitlab/nginx/gitlab_access.log <==
10.10.51.195 - - [24/Sep/2020:15:23:16 +0800] "GET /suncard/packages.git/info/refs?service=git-upload-pack HTTP/1.1" 200 701 "-" "git/2.20.2"
10.10.51.195 - production [24/Sep/2020:15:23:16 +0800] "POST /suncard/packages.git/info/lfs/locks/verify HTTP/1.1" 404 1489 "-" "git-lfs/2.5.1 (GitHub; linux amd64; go 1.11.4; git 7c940069fa)"
10.10.51.195 - production [24/Sep/2020:15:23:17 +0800] "POST /suncard/packages.git/info/lfs/objects/batch HTTP/1.1" 200 251 "-" "git-lfs/2.5.1 (GitHub; linux amd64; go 1.11.4; git 7c940069fa)"
10.10.51.195 - production [24/Sep/2020:15:23:17 +0800] "PUT /suncard/packages.git/gitlab-lfs/objects/b6f9dd313cde39ae1b87e63b9b457029bcea6e9520b5db5de20d3284e4c0259e/3 HTTP/1.1" 500 2911 "-" "git-lfs/2.5.1 (GitHub; linux amd64; go 1.11.4; git 7c940069fa)"
10.10.51.195 - production [24/Sep/2020:15:23:17 +0800] "POST /suncard/packages.git/info/lfs/objects/batch HTTP/1.1" 200 251 "-" "git-lfs/2.5.1 (GitHub; linux amd64; go 1.11.4; git 7c940069fa)" 可见与 client 对应，500 的报错信息如下，然而搜寻后发现没有相干线索
NoMethodError (undefined method `include?' for nil:NilClass):
app/controllers/projects/lfs_storage_controller.rb:55:in `tmp_filename'app/controllers/projects/lfs_storage_controller.rb:23:in `upload_finalize'
lib/gitlab/i18n.rb:45:in `with_locale'2.2 比照测试
线索中断，想到机器 A 能够失常 push，所以进行比照察看，执行雷同操作，client 日志如下
23:15:15.348869 trace git-lfs: HTTP: POST http://production:Use_1990@git.ops.yunnex.com/suncard/packages.git/info/lfs/objects/batch
> POST /suncard/packages.git/info/lfs/objects/batch HTTP/1.1
> Host: git.ops.yunnex.com
> Accept: application/vnd.git-lfs+json; charset=utf-8
> Content-Length: 158
> Content-Type: application/vnd.git-lfs+json; charset=utf-8
> User-Agent: git-lfs/2.4.2 (GitHub; linux amd64; go 1.9.4)
>
{"operation":"upload","objects":[{"oid":"9a92adbc0cee38ef658c71ce1b1bf8c65668f166bfb213644c895ccb1ad07a25","size":3}],"ref":{"name":"refs/heads/test-branch"}}23:15:15.525308 trace git-lfs: HTTP: 200
< HTTP/1.1 200 OK
< Transfer-Encoding: chunked
< Cache-Control: max-age=0, private, must-revalidate
< Connection: keep-alive
< Content-Type: application/json; charset=utf-8
< Date: Thu, 24 Sep 2020 15:15:15 GMT
< Etag: W/"486d174eca874b88d375fecca47236e2"
< Server: nginx
< Strict-Transport-Security: max-age=31536000
< X-Content-Type-Options: nosniff
< X-Frame-Options: DENY
< X-Request-Id: b87656c2-5593-4e19-bdc0-ea522657b9fc
< X-Runtime: 0.172955
< X-Ua-Compatible: IE=edge
< X-Xss-Protection: 1; mode=block
<
23:15:15.525557 trace git-lfs: HTTP: {"objects":[{"oid":"9a92adbc0cee38ef658c71ce1b1bf8c65668f166bfb213644c895ccb1ad07a25","size":3,"actions":{"upload":{"href":"http://git.ops.yunnex.com/suncard/packages.git/gitlab-lfs/objects/9a92adbc0cee38ef658c71ce1b1bf8c65668f166bfb213644c895ccb1ad07a25/3","header":{"Authorization":"Basic cHJvZHVjdGlvbjpVc2VfMTk5MA=="}}}}]}
{"objects":[{"oid":"9a92adbc0cee38ef658c71ce1b1bf8c65668f166bfb213644c895ccb1ad07a25","size":3,"actions":{"upload":{"href":"http://git.ops.yunnex.com/suncard/packages.git/gitlab-lfs/objects/9a92adbc0cee38ef658c71ce1b1bf8c65668f166bfb213644c895ccb1ad07a25/3","header":{"Authorization":"Basic cHJvZHVjdGlvbjpVc2VfMTk5MA=="}}}}]}23:15:15.525647 trace git-lfs: tq: starting transfer adapter "basic"
23:15:15.525662 trace git-lfs: xfer: adapter "basic" Begin() with 8 workers
23:15:15.525669 trace git-lfs: xfer: adapter "basic" started
23:15:15.525700 trace git-lfs: xfer: adapter "basic" worker 0 starting
23:15:15.525705 trace git-lfs: xfer: adapter "basic" worker 0 processing job for "9a92adbc0cee38ef658c71ce1b1bf8c65668f166bfb213644c895ccb1ad07a25"
23:15:15.525723 trace git-lfs: xfer: adapter "basic" worker 4 starting
23:15:15.525736 trace git-lfs: xfer: adapter "basic" worker 4 waiting for Auth
23:15:15.525759 trace git-lfs: xfer: adapter "basic" worker 6 starting
23:15:15.525770 trace git-lfs: xfer: adapter "basic" worker 6 waiting for Auth
23:15:15.525773 trace git-lfs: xfer: adapter "basic" worker 7 starting
23:15:15.525780 trace git-lfs: xfer: adapter "basic" worker 7 waiting for Auth
23:15:15.525777 trace git-lfs: xfer: adapter "basic" worker 5 starting
23:15:15.525789 trace git-lfs: xfer: adapter "basic" worker 5 waiting for Auth
23:15:15.525754 trace git-lfs: xfer: adapter "basic" worker 3 starting
23:15:15.525796 trace git-lfs: xfer: adapter "basic" worker 3 waiting for Auth
23:15:15.525731 trace git-lfs: xfer: adapter "basic" worker 2 starting
23:15:15.525803 trace git-lfs: xfer: adapter "basic" worker 2 waiting for Auth
23:15:15.525821 trace git-lfs: xfer: adapter "basic" worker 1 starting
23:15:15.525836 trace git-lfs: xfer: adapter "basic" worker 1 waiting for Auth
23:15:15.525855 trace git-lfs: HTTP: PUT http://git.ops.yunnex.com/suncard/packages.git/gitlab-lfs/objects/9a92adbc0cee38ef658c71ce1b1bf8c65668f166bfb213644c895ccb1ad07a25/3
> PUT /suncard/packages.git/gitlab-lfs/objects/9a92adbc0cee38ef658c71ce1b1bf8c65668f166bfb213644c895ccb1ad07a25/3 HTTP/1.1
> Host: git.ops.yunnex.com
> Authorization: Basic * * * * *
> Content-Length: 3
> Content-Type: application/octet-stream
> User-Agent: git-lfs/2.4.2 (GitHub; linux amd64; go 1.9.4)
>
23:15:15.525963 trace git-lfs: xfer: adapter "basic" worker 7 auth signal received
23:15:15.525968 trace git-lfs: xfer: adapter "basic" worker 6 auth signal received
23:15:15.525969 trace git-lfs: xfer: adapter "basic" worker 3 auth signal received
23:15:15.525970 trace git-lfs: xfer: adapter "basic" worker 2 auth signal received
23:15:15.525972 trace git-lfs: xfer: adapter "basic" worker 4 auth signal received
23:15:15.526006 trace git-lfs: xfer: adapter "basic" worker 5 auth signal received
23:15:15.526027 trace git-lfs: xfer: adapter "basic" worker 1 auth signal received
23:15:15.931456 trace git-lfs: HTTP: 200
< HTTP/1.1 200 OK
< Content-Length: 0
< Cache-Control: no-cache
< Connection: keep-alive
< Content-Type: text/html
< Date: Thu, 24 Sep 2020 15:15:15 GMT
< Server: nginx
< Strict-Transport-Security: max-age=31536000
< X-Content-Type-Options: nosniff
< X-Frame-Options: DENY
< X-Request-Id: e0ce367a-adce-44bd-b1ed-29a33f756bb5
< X-Runtime: 0.245919
< X-Ua-Compatible: IE=edge
< X-Xss-Protection: 1; mode=block
<
23:15:15.931592 trace git-lfs: xfer: adapter "basic" worker 0 finished job for "9a92adbc0cee38ef658c71ce1b1bf8c65668f166bfb213644c895ccb1ad07a25"
23:15:15.931664 trace git-lfs: xfer: adapter "basic" End()
23:15:15.931671 trace git-lfs: xfer: adapter "basic" worker 7 stopping
23:15:15.931675 trace git-lfs: xfer: adapter "basic" worker 2 stopping
23:15:15.931684 trace git-lfs: xfer: adapter "basic" worker 6 stopping
23:15:15.931685 trace git-lfs: xfer: adapter "basic" worker 3 stopping
23:15:15.931689 trace git-lfs: xfer: adapter "basic" worker 5 stopping
23:15:15.931692 trace git-lfs: xfer: adapter "basic" worker 1 stopping
23:15:15.931688 trace git-lfs: xfer: adapter "basic" worker 0 stopping
23:15:15.931682 trace git-lfs: xfer: adapter "basic" worker 4 stopping
23:15:15.931726 trace git-lfs: xfer: adapter "basic" stopped
23:15:15.931787 trace git-lfs: filepathfilter: rewrite ".git" as "**/.git/**"
23:15:15.931801 trace git-lfs: filepathfilter: rewrite "**/.git" as "**/.git"
23:15:15.931819 trace git-lfs: filepathfilter: rejecting "tmp" via []
23:15:15.931824 trace git-lfs: filepathfilter: accepting "tmp"
Uploading LFS objects: 100% (1/1), 3 B | 0 B/s, done其中要害申请如下，其中 500 申请已变成 200
200 POST /suncard/packages.git/info/lfs/objects/batch
200 PUT /suncard/packages.git/gitlab-lfs/objects/9a92adbc0cee38ef658c71ce1b1bf8c65668f166bfb213644c895ccb1ad07a25/3而 gitalb server 对应日志如下
==> /var/log/gitlab/gitlab-rails/production.log <==
Started POST "/suncard/packages.git/info/lfs/objects/batch" for 192.168.1.25 at 2020-09-23 18:36:27 +0800
Processing by Projects::LfsApiController#batch as JSON
Parameters: {"operation"=>"upload", "objects"=>[{"oid"=>"edeaaff3f1774ad2888673770c6d64097e391bc362d7d6fb34982ddf0efd18cb", "size"=>4}], "ref"=>{"name"=>"refs/heads/test-branch"}, "namespace_id"=>"suncard", "project_id"=>"packages.git", "lfs_api"=>{"operation"=>"upload", "objects"=>[{"oid"=>"edeaaff3f1774ad2888673770c6d64097e391bc362d7d6fb34982ddf0efd18cb", "size"=>4}], "ref"=>{"name"=>"refs/heads/test-branch"}}}
Completed 200 OK in 195ms (Views: 0.3ms | ActiveRecord: 6.1ms)
Started PUT "/suncard/packages.git/gitlab-lfs/objects/edeaaff3f1774ad2888673770c6d64097e391bc362d7d6fb34982ddf0efd18cb/4/authorize" for 192.168.1.25 at 2020-09-23 18:36:28 +0800
Processing by Projects::LfsStorageController#upload_authorize as HTML
Parameters: {"namespace_id"=>"suncard", "project_id"=>"packages.git", "oid"=>"edeaaff3f1774ad2888673770c6d64097e391bc362d7d6fb34982ddf0efd18cb", "size"=>"4"}
Completed 200 OK in 114ms (Views: 0.2ms | ActiveRecord: 3.0ms)
Started PUT "/suncard/packages.git/gitlab-lfs/objects/edeaaff3f1774ad2888673770c6d64097e391bc362d7d6fb34982ddf0efd18cb/4" for 192.168.1.25 at 2020-09-23 18:36:28 +0800
Processing by Projects::LfsStorageController#upload_finalize as HTML
Parameters: {"namespace_id"=>"suncard", "project_id"=>"packages.git", "oid"=>"edeaaff3f1774ad2888673770c6d64097e391bc362d7d6fb34982ddf0efd18cb", "size"=>"4"}
Completed 200 OK in 169ms (ActiveRecord: 51.1ms)
Updating statistics for project 32
==> /var/log/gitlab/nginx/gitlab_access.log <==
192.168.1.25 - production [23/Sep/2020:18:36:28 +0800] "POST /suncard/packages.git/info/lfs/objects/batch HTTP/1.1" 200 251 "-" "git-lfs/2.4.2 (GitHub; linux amd64; go 1.9.4)"
192.168.1.25 - production [23/Sep/2020:18:36:28 +0800] "PUT /suncard/packages.git/gitlab-lfs/objects/edeaaff3f1774ad2888673770c6d64097e391bc362d7d6fb34982ddf0efd18cb/4 HTTP/1.1" 200 0 "-" "git-lfs/2.4.2 (GitHub; linux amd64; go 1.9.4)"变动如下
# 多了 1 个 200 申请
PUT "/suncard/packages.git/gitlab-lfs/objects/edeaaff3f1774ad2888673770c6d64097e391bc362d7d6fb34982ddf0efd18cb/4/authorize"
# 下列申请从 500 变为 200
200 PUT "/suncard/packages.git/gitlab-lfs/objects/edeaaff3f1774ad2888673770c6d64097e391bc362d7d6fb34982ddf0efd18cb/4"gitlab server 雷同、操作的仓库分支雷同的状况下，这样的差别揣测是 client 客户端导致的了，先来看版本
git --version 1.8.3.1(失常) 2.20.4(异样)
git lfs version 2.4.2(失常) 2.5.1(异样) 其中失常的机器 A 是物理机 centos 用 yum 装置的 git 和 git-lfs；异样的是跑在 k8s 集群中基于 alpine 的 pod，是通过 apk 装置的，遂尝试 apk 降级 git 和 git-lfs 到机器 A 失常的版本。apk 降级须要到其市场查找指标版本 https://pkgs.alpinelinux.org/…。
再通过降通过降 repository 版本来装置低版本的软件包，原 alpine 是 v3.9，当初尝试改为 v3.8
apk add --no-cache --repository http://mirrors.aliyun.com/alpine/v3.8/main --repository http://mirrors.aliyun.com/alpine/v3.8/community 'git-lfs=2.4.2-r1'再次执行原 push 等操作，终于失常了！以防万一，再到本来失常的机器 A 把 git-lfs 降级一下，看是否也异样。
装置过程：mkdir /tmp/test
wget https://github.com/git-lfs/git-lfs/releases/download/v2.5.1/git-lfs-linux-amd64-v2.5.1.tar.gz
tar -xvf git-lfs-linux-amd64-v2.5.1.tar.gz
# 这样就能够在不影响原 git-lfs 装置的状况下进行测试，后续用 /tmp/test/git-lfs 进行 push 等操作
/tmp/test/git-lfs version果不其然，也重现了雷同异样。
3 论断
首先异样与 git 版本没有关系，只和 git-lfs 版本无关。之后又尝试了更低的 2.3.x、更高的 2.6.x~2.11.x。确定是从 2.5.x 开始的版本有异样。另外还须要关注 gitlab 的版本, 目前为 9.4.1
sudo gitlab-rake gitlab:env:info
System information
System:
Current User: git
Using RVM: no
Ruby Version: 2.3.3p222
Gem Version: 2.6.6
Bundler Version:1.13.7
Rake Version: 10.5.0
Redis Version: 3.2.5
Git Version: 2.13.0
Sidekiq Version:5.0.0
Go Version: unknown
GitLab information
Version: 9.4.1
Revision: 6678b4e
Directory: /opt/gitlab/embedded/service/gitlab-rails
DB Adapter: postgresql
URL: http://git.ops.yunnex.com
HTTP Clone URL: http://git.ops.yunnex.com/some-group/some-project.git
SSH Clone URL: git@git.ops.yunnex.com:some-group/some-project.git
Using LDAP: yes
Using Omniauth: no
GitLab Shell
Version: 5.3.1
Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories
Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks
Git: /opt/gitlab/embedded/bin/git目前只能揣测该 gitlab/lfs server 版本与 git lfs client push 不兼容，所以采取的计划是 lfs 降为 2.4.x，alpine 应用 v3.8 repository 重新安装 git-lfs。
“架构人生，迭代生命”——高深老夏，搜寻 summer_deep 微信公众号可获取更多帮忙