New to KubeDB? Please start here.

Kafka Vertical Scaling

This guide will give an overview on how KubeDB Ops-manager operator updates the resources(for example CPU and Memory etc.) of the Kafka.

Before You Begin

• You should be familiar with the following KubeDB concepts:
  • Kafka
  • KafkaOpsRequest

How Vertical Scaling Process Works

The following diagram shows how KubeDB Ops-manager operator updates the resources of the Kafka. Open the image in a new tab to see the enlarged version.

  

The vertical scaling process consists of the following steps:

1. At first, a user creates a Kafka Custom Resource (CR).

2. KubeDB Provisioner operator watches the Kafka CR.

3. When the operator finds a Kafka CR, it creates required number of PetSets and related necessary stuff like secrets, services, etc.

4. Then, in order to update the resources(for example CPU, Memory etc.) of the Kafka cluster, the user creates a KafkaOpsRequest CR with desired information.

5. KubeDB Ops-manager operator watches the KafkaOpsRequest CR.

6. When it finds a KafkaOpsRequest CR, it halts the Kafka object which is referred from the KafkaOpsRequest. So, the KubeDB Provisioner operator doesn’t perform any operations on the Kafka object during the vertical scaling process.

7. Then the KubeDB Ops-manager operator will update resources of the PetSet Pods to reach desired state.

8. After the successful update of the resources of the PetSet’s replica, the KubeDB Ops-manager operator updates the Kafka object to reflect the updated state.

9. After the successful update of the Kafka resources, the KubeDB Ops-manager operator resumes the Kafka object so that the KubeDB Provisioner operator resumes its usual operations.

Vertical Scaling Modes

KubeDB actuates vertical scaling in one of two modes, selected through the spec.verticalScaling.mode field of the KafkaOpsRequest:

• Restart (default): The operator patches the PetSet with the new resources and restarts the Pods (one at a time, honoring the database’s failover rules) so they come back with the updated CPU and Memory. This works on every Kubernetes cluster.
• InPlace: The operator resizes the running containers in place using the Kubernetes in-place Pod resize (pods/resize subresource) — no Pod restart, so scaling happens without downtime or failover. If a Node cannot accommodate the new resources (the resize is reported Infeasible), the operator automatically falls back to the Restart behavior for that Pod.

If spec.verticalScaling.mode is omitted, it defaults to Restart.

Note: InPlace mode relies on the Kubernetes InPlacePodVerticalScaling feature gate, which is enabled by default from Kubernetes v1.33. On older clusters, or when the feature gate is disabled, use Restart mode.

In the next docs, we are going to show a step by step guide on updating resources of Kafka database using KafkaOpsRequest CRD.