Kubernetes能够应用一个lock对象,进行多实例的选举:
- 抢到lock对象更新权的实例即为leader;
lock对象能够是:
- Lease
- ConfigMap
- Endpoints
controller-runtime罕用于operator的开发,其封装了client-go中leader选举的细节,leader选举的场景为:
- 某一时刻只有一个实例能够reconcile,其它实例处于standby状态;
- 一旦leader实例挂掉,standby实例能够顶上,继续执行reconcile;
一.operator
Operator开发中,应用controller-runtime实现leader选举非常简单,调用时传入election参数即可:
func main() { flag.BoolVar(&enableLeaderElection, "enableLeaderElection", false, "default false, if enabled the cronHPA would be in primary and standby mode.") flag.Parse() mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{ LeaderElection: enableLeaderElection, LeaderElectionID: "kubernetes-cronhpa-controller", MetricsBindAddress: metricsAddr, }) ...}
多实例抢锁时,应用的资源对象为configmap:
# kubectl get cm -n kube-system kubernetes-cronhpa-controller -oyamlapiVersion: v1kind: ConfigMapmetadata: annotations: control-plane.alpha.kubernetes.io/leader: '{"holderIdentity":"kubernetes-cronhpa-controller-6bcbdf9844-d2z4q_c36ac963-3e9c-4e32-ac14-ab48b6f2633b","leaseDurationSeconds":15,"acquireTime":"2023-04-08T16:14:30Z","renewTime":"2023-04-10T07:55:15Z","leaderTransitions":126}' creationTimestamp: "2022-10-13T06:20:42Z" managedFields: - apiVersion: v1 fieldsType: FieldsV1 ... manager: kubernetes-cronhpa-controller operation: Update time: "2022-10-13T06:20:42Z" name: kubernetes-cronhpa-controller namespace: kube-system resourceVersion: "76431408" uid: 5099216c-2fd6-452a-afeb-bd39047c2cbb
二.controller-runtime
若应用底层的client-go进行实例选举,通常有以下步骤:
- 首先,创立resourcelock,指定锁的资源类型;
- 而后,创立leaderElector对象,调用leaderElector.Run();
controller-runtime也是这么做的。
1. 创立resourcelock
创立ControllerManager的时候,创立resourcelock:
// controller-runtime/pkg/manager/manager.gofunc New(config *rest.Config, options Options) (Manager, error) { ... resourceLock, err := options.newResourceLock(leaderConfig, recorderProvider, leaderelection.Options{ LeaderElection: options.LeaderElection, LeaderElectionID: options.LeaderElectionID, LeaderElectionNamespace: options.LeaderElectionNamespace, }) if err != nil { return nil, err } return &controllerManager{ config: config, ... resourceLock: resourceLock, }}
创立resourcelock的细节:
- 能够看出,这里创立的是ConfigMap类型的resourcelock;
// controller-runtime/pkg/leaderelection/leader_election.gofunc NewResourceLock(config *rest.Config, recorderProvider recorder.Provider, options Options) (resourcelock.Interface, error) { ... return resourcelock.New(resourcelock.ConfigMapsResourceLock, options.LeaderElectionNamespace, options.LeaderElectionID, client.CoreV1(), client.CoordinationV1(), resourcelock.ResourceLockConfig{ Identity: id, EventRecorder: recorderProvider.GetEventRecorderFor(id), })}
2. 创立LeaderElector对象,调用LeaderElector.Run():
首先,是ControllerManager.Start():
// controller-manager/pkg/manager/internal.gofunc (cm *controllerManager) Start(stop <-chan struct{}) (err error) { ... go func() { if cm.resourceLock != nil { //须要选举 err := cm.startLeaderElection() if err != nil { cm.errChan <- err } } else { //不须要选举 // Treat not having leader election enabled the same as being elected. close(cm.elected) go cm.startLeaderElectionRunnables() } }() select { case <-stop: // We are done return nil case err := <-cm.errChan: // Error starting or running a runnable return err }}
若须要选举:
- 创立LeaderElector对象;
- 调用LeaderElector.Run();
- 选举胜利后,执行startLeaderElectionRunnables(),这里的runables即是Reconcile();
// controller-runtime/pkg/manager/internal.gofunc (cm *controllerManager) startLeaderElection() (err error) { ... l, err := leaderelection.NewLeaderElector(leaderelection.LeaderElectionConfig{ Lock: cm.resourceLock, LeaseDuration: cm.leaseDuration, RenewDeadline: cm.renewDeadline, RetryPeriod: cm.retryPeriod, Callbacks: leaderelection.LeaderCallbacks{ OnStartedLeading: func(_ context.Context) { close(cm.elected) cm.startLeaderElectionRunnables() // 选举胜利后,执行Reconcile() }, OnStoppedLeading: cm.onStoppedLeading, }, }) ... // Start the leader elector process go l.Run(ctx) return nil}
若不要选举,则间接执行startLeaderElectionRunnables(),即执行Reconcile()。
参考:
1.https://itnext.io/leader-election-in-kubernetes-using-client-...