关于kubernetes:如何彻底删除EKS中一直卡在Terminating的Namespace

41次阅读

共计 8250 个字符,预计需要花费 21 分钟才能阅读完成。


作者:SRE 运维博客
博客地址:https://www.cnsre.cn/
文章地址:https://www.cnsre.cn/posts/220107906441/
相干话题:https://www.cnsre.cn/tags/eks/


介绍

最近始终在玩 EKS(Elastic Kubernetes Service — Amazon EKS)和 kubesphere。因为之前没有应用过 EKS 和 kubesphere 所以这个过程也是一个试错的过程,在我应用 kubesphere 的时候发现有一个日志服务,在好奇心的驱使下,我创立了它。

在我创立了日志服务(KubeSphere Logging System)当前,我发现我并不想应用它。(可能我只是想看看它到底是什么吧。)强迫症我的,就想把它给删除掉。于是我在我的 EKS 中对他进行了强制删除 kubectl delete ns kubesphere-logging-system --force --grace-period=0

让人难堪的是,这个 Namespace 并没有立马删除,我自我安慰到,可能 Namespace 下边有其余没有删除的资源在期待删除,我在等等。。。

过了半个小时,我去查看的时候

[root@ip-10-0-100-206 ~]# kubectl get  ns  kubesphere-logging-system
NAME                              STATUS        AGE
kubesphere-logging-system         Terminating   6d19h

它如同这地卡在了 Terminating 的状态。
我试着寻找解决办法,http://github.com/kubernetes/… 然而这中办法要通过 API 才能够实现。EKS 是托管在 AWS 中的,我基本没有方法去操作 eks 的后盾。

我开始悔恨我为什么要去装置它,我本能够不折腾的。然而回忆如果我不去装置它的话,我必定会像想在这样惆怅,因为我无奈理解它(KubeSphere Logging System)到底是什么样的。

终于我找到了 commented https://github.com/kubernetes…

如何彻底删除 namespace

获取 namespace 的详情信息并转为 json

kubectl get namespace kubesphere-logging-system -o json > kubesphere-logging-system.json

关上 json 文件编辑

{
    "apiVersion": "v1",
    "kind": "Namespace",
    "metadata": {
        "creationTimestamp": "2021-12-31T05:03:58Z",
        "deletionTimestamp": "2022-01-05T08:05:40Z",
        "labels": {
            "kubesphere.io/namespace": "kubesphere-logging-system",
            "kubesphere.io/workspace": "system-workspace"
        },
        "managedFields": [
            {
                "apiVersion": "v1",
                "fieldsType": "FieldsV1",
                "fieldsV1": {
                    "f:metadata": {
                        "f:labels": {".": {},
                            "f:kubesphere.io/namespace": {}},
                        "f:ownerReferences": {".": {},
                            "k:{\"uid\":\"6d535470-2592-4f3c-a155-eabc362c339d\"}": {".": {},
                                "f:apiVersion": {},
                                "f:blockOwnerDeletion": {},
                                "f:controller": {},
                                "f:kind": {},
                                "f:name": {},
                                "f:uid": {}}
                        }
                    }
                },
                "manager": "controller-manager",
                "operation": "Update",
                "time": "2021-12-31T05:04:01Z"
            },
            {
                "apiVersion": "v1",
                "fieldsType": "FieldsV1",
                "fieldsV1": {
                    "f:metadata": {
                        "f:labels": {"f:kubesphere.io/workspace": {}
                        }
                    },
                    "f:status": {"f:phase": {}
                    }
                },
                "manager": "kubectl",
                "operation": "Update",
                "time": "2021-12-31T05:04:01Z"
            },
            {
                "apiVersion": "v1",
                "fieldsType": "FieldsV1",
                "fieldsV1": {
                    "f:status": {
                        "f:conditions": {".": {},
                            "k:{\"type\":\"NamespaceContentRemaining\"}": {".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}},
                            "k:{\"type\":\"NamespaceDeletionContentFailure\"}": {".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}},
                            "k:{\"type\":\"NamespaceDeletionDiscoveryFailure\"}": {".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}},
                            "k:{\"type\":\"NamespaceDeletionGroupVersionParsingFailure\"}": {".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}},
                            "k:{\"type\":\"NamespaceFinalizersRemaining\"}": {".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}}
                        }
                    }
                },
                "manager": "kube-controller-manager",
                "operation": "Update",
                "time": "2022-01-05T08:05:47Z"
            }
        ],
        "name": "kubesphere-logging-system",
        "ownerReferences": [
            {
                "apiVersion": "tenant.kubesphere.io/v1alpha1",
                "blockOwnerDeletion": true,
                "controller": true,
                "kind": "Workspace",
                "name": "system-workspace",
                "uid": "6d535470-2592-4f3c-a155-eabc362c339d"
            }
        ],
        "resourceVersion": "7376520",
        "uid": "2b76e9b1-75f2-4a2e-a819-73b36aea188e"
    },
    "spec": {
        "finalizers": ["kubernetes" # 将此行删除]
    },
    "status": {
        "conditions": [
            {
                "lastTransitionTime": "2022-01-05T08:05:47Z",
                "message": "All resources successfully discovered",
                "reason": "ResourcesDiscovered",
                "status": "False",
                "type": "NamespaceDeletionDiscoveryFailure"
            },
            {
                "lastTransitionTime": "2022-01-05T08:05:47Z",
                "message": "All legacy kube types successfully parsed",
                "reason": "ParsedGroupVersions",
                "status": "False",
                "type": "NamespaceDeletionGroupVersionParsingFailure"
            },
            {
                "lastTransitionTime": "2022-01-05T08:05:47Z",
                "message": "All content successfully deleted, may be waiting on finalization",
                "reason": "ContentDeleted",
                "status": "False",
                "type": "NamespaceDeletionContentFailure"
            },
            {
                "lastTransitionTime": "2022-01-05T08:05:47Z",
                "message": "Some resources are remaining: fluentbits.logging.kubesphere.io has 1 resource instances",
                "reason": "SomeResourcesRemain",
                "status": "True",
                "type": "NamespaceContentRemaining"
            },
            {
                "lastTransitionTime": "2022-01-05T08:05:47Z",
                "message": "Some content in the namespace has finalizers remaining: fluentbit.logging.kubesphere.io in 1 resource instances",
                "reason": "SomeFinalizersRemain",
                "status": "True",
                "type": "NamespaceFinalizersRemaining"
            }
        ],
        "phase": "Terminating"
    }
}

找到 spec 将 finalizers 下的 kubernetes 删除
具体如下

{
    "apiVersion": "v1",
    "kind": "Namespace",
    "metadata": {
        "creationTimestamp": "2021-12-31T05:03:58Z",
        "deletionTimestamp": "2022-01-05T08:05:40Z",
        "labels": {
            "kubesphere.io/namespace": "kubesphere-logging-system",
            "kubesphere.io/workspace": "system-workspace"
        },
        "managedFields": [
            {
                "apiVersion": "v1",
                "fieldsType": "FieldsV1",
                "fieldsV1": {
                    "f:metadata": {
                        "f:labels": {".": {},
                            "f:kubesphere.io/namespace": {}},
                        "f:ownerReferences": {".": {},
                            "k:{\"uid\":\"6d535470-2592-4f3c-a155-eabc362c339d\"}": {".": {},
                                "f:apiVersion": {},
                                "f:blockOwnerDeletion": {},
                                "f:controller": {},
                                "f:kind": {},
                                "f:name": {},
                                "f:uid": {}}
                        }
                    }
                },
                "manager": "controller-manager",
                "operation": "Update",
                "time": "2021-12-31T05:04:01Z"
            },
            {
                "apiVersion": "v1",
                "fieldsType": "FieldsV1",
                "fieldsV1": {
                    "f:metadata": {
                        "f:labels": {"f:kubesphere.io/workspace": {}
                        }
                    },
                    "f:status": {"f:phase": {}
                    }
                },
                "manager": "kubectl",
                "operation": "Update",
                "time": "2021-12-31T05:04:01Z"
            },
            {
                "apiVersion": "v1",
                "fieldsType": "FieldsV1",
                "fieldsV1": {
                    "f:status": {
                        "f:conditions": {".": {},
                            "k:{\"type\":\"NamespaceContentRemaining\"}": {".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}},
                            "k:{\"type\":\"NamespaceDeletionContentFailure\"}": {".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}},
                            "k:{\"type\":\"NamespaceDeletionDiscoveryFailure\"}": {".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}},
                            "k:{\"type\":\"NamespaceDeletionGroupVersionParsingFailure\"}": {".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}},
                            "k:{\"type\":\"NamespaceFinalizersRemaining\"}": {".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}}
                        }
                    }
                },
                "manager": "kube-controller-manager",
                "operation": "Update",
                "time": "2022-01-05T08:05:47Z"
            }
        ],
        "name": "kubesphere-logging-system",
        "ownerReferences": [
            {
                "apiVersion": "tenant.kubesphere.io/v1alpha1",
                "blockOwnerDeletion": true,
                "controller": true,
                "kind": "Workspace",
                "name": "system-workspace",
                "uid": "6d535470-2592-4f3c-a155-eabc362c339d"
            }
        ],
        "resourceVersion": "7376520",
        "uid": "2b76e9b1-75f2-4a2e-a819-73b36aea188e"
    },
    "spec": {"finalizers": []
    },
    "status": {
        "conditions": [
            {
                "lastTransitionTime": "2022-01-05T08:05:47Z",
                "message": "All resources successfully discovered",
                "reason": "ResourcesDiscovered",
                "status": "False",
                "type": "NamespaceDeletionDiscoveryFailure"
            },
            {
                "lastTransitionTime": "2022-01-05T08:05:47Z",
                "message": "All legacy kube types successfully parsed",
                "reason": "ParsedGroupVersions",
                "status": "False",
                "type": "NamespaceDeletionGroupVersionParsingFailure"
            },
            {
                "lastTransitionTime": "2022-01-05T08:05:47Z",
                "message": "All content successfully deleted, may be waiting on finalization",
                "reason": "ContentDeleted",
                "status": "False",
                "type": "NamespaceDeletionContentFailure"
            },
            {
                "lastTransitionTime": "2022-01-05T08:05:47Z",
                "message": "Some resources are remaining: fluentbits.logging.kubesphere.io has 1 resource instances",
                "reason": "SomeResourcesRemain",
                "status": "True",
                "type": "NamespaceContentRemaining"
            },
            {
                "lastTransitionTime": "2022-01-05T08:05:47Z",
                "message": "Some content in the namespace has finalizers remaining: fluentbit.logging.kubesphere.io in 1 resource instances",
                "reason": "SomeFinalizersRemain",
                "status": "True",
                "type": "NamespaceFinalizersRemaining"
            }
        ],
        "phase": "Terminating"
    }
}

执行清理命令

当初咱们只须要一条命令 就能够彻底删除这个 Namespace

 kubectl replace --raw "/api/v1/namespaces/kubesphere-logging-system/finalize" -f ./kubesphere-logging-system.json 

执行完当前,你须要期待一会
再次执行命令查看 namespaces

 kubectl replace --raw "/api/v1/namespaces/kubesphere-logging-system/finalize" -f ./kubesphere-logging-system.json 

最初的查看

kubectl get  ns  kubesphere-logging-system
Error from server (NotFound): namespaces "kubesphere-logging-system" not found

[root@ip-10-0-100-206 ~]# kubectl get  ns 
NAME                              STATUS   AGE
default                           Active   23d
kubesphere-controls-system        Active   9d
kubesphere-devops-system          Active   9d
kubesphere-devops-worker          Active   16h
kubesphere-monitoring-federated   Active   9d
kubesphere-monitoring-system      Active   9d
kubesphere-sample-dev             Active   8d
kubesphere-system                 Active   9d

在此查看的时候,它曾经不存在了。


作者:SRE 运维博客
博客地址:https://www.cnsre.cn/
文章地址:https://www.cnsre.cn/posts/220107906441/
相干话题:https://www.cnsre.cn/tags/eks/


正文完
 0