一.背景
在kubernetes中,hpa(HorizontalPodAutoscaler)能够依据工作负载的指标(cpu/mem等),主动对其进行伸缩,即主动减少、缩小其pod数量。
cronhpa是一个定时扩缩容的组件,它反对依照Crontab表达式的策略,定时地对workload进行扩缩容,这里应用aliyun的kubernetes-cronhpa-controller。
cronhpa的例子如下:
apiVersion: autoscaling.alibabacloud.com/v1beta1kind: CronHorizontalPodAutoscalermetadata: name: cronhpa-samplespec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: nginx // 对deploy: nginx进行定时扩缩容; jobs: - name: "scale-down" // 在每分钟的30s,将deploy缩容到1个正本; schedule: "30 */1 * * * *" targetSize: 1 - name: "scale-up" // 在每分钟的0s,将deploy扩容到3个正本; schedule: "0 */1 * * * *" targetSize: 3
问题
如果CronHPA和HPA,同时操作一个scaleTargetRef,就会呈现replicas被CronHPA和HPA轮番批改的状况,比方:
- HPA依据指标,要将正本扩容为5个;
- CronHPA依据crontab规定,将正本扩容为3个;
- 两个controller独立工作,两个扩容操作会互相笼罩;
二.计划
kubernetes-cronhpa-controller对该问题的计划:
- 将CronHPA的scaleTargetRef设置为HPA;
- 再设置HPA的target为deployment,从而让CronHPA感知HPA的以后状态;
最终:
- CronHPA能够感知到HPA的minReplicas/maxReplicas/desiredReplicas;
- CronHPA能够通过HPA的target,晓得deploy的currentReplicas;
CronHPA通过调整hpa.minReplicas/hpa.maxReplicas/deploy.replicas的形式,与HPA一起实现扩缩容操作。
策略调整
在kubernetes-cronhpa-controller计划的根底上,能够依据本人的需要进行调整,比方:
- 规定cronhpa不批改hpa的参数;
- 规定cronhpa在伸缩时,伸缩范畴为[hpa.minReplica, hpa.maxReplica],不能超过范畴;
三.源码
看一下kubernetes-cronhpa-controller对cronhpa和hpa共存伸缩的源码。
1. 源码入口
在执行定时工作的时候,对cronhpa.TargetRef.Kind="HPA"独自解决:
// pkg/controller/cronjob.gofunc (ch *CronJobHPA) Run() (msg string, err error) { startTime := time.Now() times := 0 for { now := time.Now() ... // hpa compatible if ch.TargetRef.RefKind == "HorizontalPodAutoscaler" { // target=HPA msg, err = ch.ScaleHPA() if err == nil { break } } else { msg, err = ch.ScalePlainRef() // target=Deploy/Sts if err == nil { break } } ... } return msg, err}
CronHPA对HPA伸缩时,进行两个操作:
- 批改HPA的min/maxReplicas;
- 对HPA.target(deploy/statefulset)进行Scale;
CronHPA对HPA伸缩时,根据的参数:
- min: hpa.minReplicas
- max: hpa.maxReplicas
- deploy: hpa.Status.CurrentReplicas
- target: cronhpa.job.TargetSize
2. 伸缩逻辑
ScaleHPA()做的事件:
- 根据上述不同的场景,更新hpa.minReplicas和maxReplicas;
伸缩hpa.targetRef:
- 若hpa.targetRef正本数 >= cronhpa.target,则不须要伸缩,间接返回;
- 否则,将hpa.targetRef正本数,伸缩至cronhpa.target;
// pkg/controller/cronjob.gofunc (ch *CronJobHPA) ScaleHPA() (msg string, err error) { ... // 查问HPA对象 hpa := &autoscalingapi.HorizontalPodAutoscaler{} err = ch.client.Get(ctx, types.NamespacedName{Namespace: ch.HPARef.Namespace, Name: ch.TargetRef.RefName}, hpa) ... mappings, err := ch.mapper.RESTMappings(targetGK) // 查问HPA的target,即scale对象 for _, mapping := range mappings { targetGR = mapping.Resource.GroupResource() scale, err = ch.scaler.Scales(ch.TargetRef.RefNamespace).Get(context.Background(), targetGR, targetRef.Name, v1.GetOptions{}) if err == nil { found = true break } } // 依据不同场景,批改hpa.minReplicas/maxReplicas updateHPA := false if ch.DesiredSize > hpa.Spec.MaxReplicas { hpa.Spec.MaxReplicas = ch.DesiredSize updateHPA = true } if ch.DesiredSize < *hpa.Spec.MinReplicas { *hpa.Spec.MinReplicas = ch.DesiredSize updateHPA = true } if hpa.Status.CurrentReplicas == *hpa.Spec.MinReplicas && ch.DesiredSize < hpa.Status.CurrentReplicas { *hpa.Spec.MinReplicas = ch.DesiredSize updateHPA = true } if hpa.Status.CurrentReplicas < ch.DesiredSize { *hpa.Spec.MinReplicas = ch.DesiredSize updateHPA = true } // 将hpa的批改长久化 if updateHPA { err = ch.client.Update(ctx, hpa) if err != nil { return "", err } } // 若指标对象不须要伸缩,则间接返回 if hpa.Status.CurrentReplicas >= ch.DesiredSize { // skip change replicas and exit return fmt.Sprintf("Skip scale replicas because HPA %s current replicas:%d >= desired replicas:%d.", hpa.Name, scale.Spec.Replicas, ch.DesiredSize), nil } // 否则,伸缩指标对象到target正本数 scale.Spec.Replicas = int32(ch.DesiredSize) _, err = ch.scaler.Scales(ch.TargetRef.RefNamespace).Update(context.Background(), targetGR, scale, metav1.UpdateOptions{}) return msg, nil}
参考
1.https://help.aliyun.com/document_detail/151557.html
2.https://github.com/AliyunContainerService/kubernetes-cronhpa-controller