AppsCode
  • KubeDB

    Run Production-Grade Databases on Kubernetes

    Stash

    Backup and Recovery Solution for Kubernetes

    KubeVault

    Run Production-Grade Vault on Kubernetes

    Voyager

    Secure Ingress Controller for Kubernetes

    ConfigSyncer

    Kubernetes Configuration Syncer

    Guard

    Kubernetes Authentication WebHook Server

    KubeDB

    KubeDB simplifies Provisioning, Upgrading, Scaling, Volume Expansion, Monitor, Backup, Restore for various Databases in Kubernetes on any Public & Private Cloud

    • check-boxLower administrative burden
    • check-boxNative Kubernetes Support
    • check-boxPerformance
    • check-boxAvailability and durability
    • check-boxManageability
    • check-boxCost-effectiveness
    • check-boxSecurity
    Stash

    A complete Kubernetes native disaster recovery solution for backup and restore your volumes and databases in Kubernetes on any public and private clouds.

    • check-boxDeclarative API
    • check-boxBackup Kubernetes Volumes
    • check-boxBackup Database
    • check-boxMultiple Storage Support
    • check-boxDeduplication
    • check-boxData Encryption
    • check-boxVolume Snapshot
    • check-boxPolicy Based Backup
    KubeVault

    KubeVault is a Git-Ops ready, production-grade solution for deploying and configuring Hashicorp's Vault on Kubernetes.

    • check-boxVault Kubernetes Deployment
    • check-boxAuto Initialization & Unsealing
    • check-boxVault Backup & Restore
    • check-boxConsume KubeVault Secrets with CSI
    • check-boxManage DB Users Privileges
    • check-boxStorage Backend
    • check-boxAuthentication Method
    • check-boxDatabase Secret Engine
    Voyager

    Secure Ingress Controller for Kubernetes

    • check-boxHTTP & TCP
    • check-boxSSL
    • check-boxPlatform support
    • check-boxHAProxy
    • check-boxPrometheus
    • check-boxLet's Encrypt
    configsyncer

    Kubernetes Configuration Syncer

    • check-boxConfiguration Syncer
    Guard

    Kubernetes Authentication WebHook Server

    • check-boxIdentity Providers
    • check-boxCLI
    • check-boxRBAC
  • RESOURCES
  • Webinar New
CONTACT SALES
KubeDB
github-icon twitterx-icon
TRY NOW FREE

New to KubeDB? Please start here.

Kafka Topology Volume Expansion

This guide will show you how to use KubeDB Ops-manager operator to expand the volume of a Kafka Topology Cluster.

Before You Begin

  • At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster.

  • You must have a StorageClass that supports volume expansion.

  • Install KubeDB Provisioner and Ops-manager operator in your cluster following the steps here.

  • You should be familiar with the following KubeDB concepts:

To keep everything isolated, we are going to use a separate namespace called demo throughout this tutorial.

$ kubectl create ns demo
namespace/demo created

Note: The yaml files used in this tutorial are stored in docs/examples/kafka folder in GitHub repository kubedb/docs.

Expand Volume of Topology Kafka Cluster

Here, we are going to deploy a Kafka topology using a supported version by KubeDB operator. Then we are going to apply KafkaOpsRequest to expand its volume.

Prepare Kafka Topology Cluster

At first verify that your cluster has a storage class, that supports volume expansion. Let’s check,

$ kubectl get storageclass
NAME                 PROVISIONER            RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
standard (default)   kubernetes.io/gce-pd   Delete          Immediate           true                   2m49s

We can see from the output the standard storage class has ALLOWVOLUMEEXPANSION field as true. So, this storage class supports volume expansion. We can use it.

Now, we are going to deploy a Kafka combined cluster with version 3.6.1.

Deploy Kafka

In this section, we are going to deploy a Kafka topology cluster for broker and controller with 1GB volume. Then, in the next section we will expand its volume to 2GB using KafkaOpsRequest CRD. Below is the YAML of the Kafka CR that we are going to create,

apiVersion: kubedb.com/v1
kind: Kafka
metadata:
  name: kafka-prod
  namespace: demo
spec:
  version: 3.6.1
  topology:
    broker:
      replicas: 2
      storage:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi
        storageClassName: standard
    controller:
      replicas: 2
      storage:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi
        storageClassName: standard
  storageType: Durable
  deletionPolicy: WipeOut

Let’s create the Kafka CR we have shown above,

$ kubectl create -f https://github.com/kubedb/docs/raw/v2024.8.21/docs/examples/kafka/volume-expansion/kafka-topology.yaml
kafka.kubedb.com/kafka-prod created

Now, wait until kafka-prod has status Ready. i.e,

$ kubectl get kf -n demo -w
NAME          TYPE            VERSION   STATUS         AGE
kafka-prod    kubedb.com/v1   3.6.1     Provisioning   0s
kafka-prod    kubedb.com/v1   3.6.1     Provisioning   9s
.
.
kafka-prod    kubedb.com/v1   3.6.1     Ready          2m10s

Let’s check volume size from petset, and from the persistent volume,

$ kubectl get petset -n demo kafka-prod-broker -o json | jq '.spec.volumeClaimTemplates[].spec.resources.requests.storage'
"1Gi"

$ kubectl get petset -n demo kafka-prod-controller -o json | jq '.spec.volumeClaimTemplates[].spec.resources.requests.storage'
"1Gi"

$ kubectl get pv -n demo                                       
NAME                   CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS     CLAIM                                           STORAGECLASS   REASON   AGE
pvc-3f177a92721440bb   1Gi        RWO            Delete           Bound      demo/kafka-prod-data-kafka-prod-controller-0    standard                106s
pvc-86ff354122324b1c   1Gi        RWO            Delete           Bound      demo/kafka-prod-data-kafka-prod-broker-1        standard                78s
pvc-9fa35d773aa74bd0   1Gi        RWO            Delete           Bound      demo/kafka-prod-data-kafka-prod-controller-1    standard                75s
pvc-ccf50adf179e4162   1Gi        RWO            Delete           Bound      demo/kafka-prod-data-kafka-prod-broker-0        standard                106s

You can see the petsets have 1GB storage, and the capacity of all the persistent volumes are also 1GB.

We are now ready to apply the KafkaOpsRequest CR to expand the volume of this database.

Volume Expansion

Here, we are going to expand the volume of the kafka topology cluster.

Create KafkaOpsRequest

In order to expand the volume of the database, we have to create a KafkaOpsRequest CR with our desired volume size. Below is the YAML of the KafkaOpsRequest CR that we are going to create,

apiVersion: ops.kubedb.com/v1alpha1
kind: KafkaOpsRequest
metadata:
  name: kf-volume-exp-topology
  namespace: demo
spec:
  type: VolumeExpansion  
  databaseRef:
    name: kafka-prod
  volumeExpansion:
    broker: 3Gi
    controller: 2Gi
    mode: Online

Here,

  • spec.databaseRef.name specifies that we are performing volume expansion operation on kafka-prod.
  • spec.type specifies that we are performing VolumeExpansion on our database.
  • spec.volumeExpansion.broker specifies the desired volume size for broker node.
  • spec.volumeExpansion.controller specifies the desired volume size for controller node.

If you want to expand the volume of only one node, you can specify the desired volume size for that node only.

Let’s create the KafkaOpsRequest CR we have shown above,

$ kubectl apply -f https://github.com/kubedb/docs/raw/v2024.8.21/docs/examples/kafka/volume-expansion/kafka-volume-expansion-topology.yaml
kafkaopsrequest.ops.kubedb.com/kf-volume-exp-topology created

Verify Kafka Topology volume expanded successfully

If everything goes well, KubeDB Ops-manager operator will update the volume size of Kafka object and related PetSets and Persistent Volumes.

Let’s wait for KafkaOpsRequest to be Successful. Run the following command to watch KafkaOpsRequest CR,

$ kubectl get kafkaopsrequest -n demo
NAME                     TYPE              STATUS       AGE
kf-volume-exp-topology   VolumeExpansion   Successful   3m1s

We can see from the above output that the KafkaOpsRequest has succeeded. If we describe the KafkaOpsRequest we will get an overview of the steps that were followed to expand the volume of kafka.

$ kubectl describe kafkaopsrequest -n demo kf-volume-exp-topology   
Name:         kf-volume-exp-topology
Namespace:    demo
Labels:       <none>
Annotations:  <none>
API Version:  ops.kubedb.com/v1alpha1
Kind:         KafkaOpsRequest
Metadata:
  Creation Timestamp:  2024-07-31T04:44:17Z
  Generation:          1
  Resource Version:    149682
  UID:                 e0e19d97-7150-463c-9a7d-53eff05ea6c4
Spec:
  Apply:  IfReady
  Database Ref:
    Name:  kafka-prod
  Type:    VolumeExpansion
  Volume Expansion:
    Broker:      3Gi
    Controller:  2Gi
    Mode:        Online
Status:
  Conditions:
    Last Transition Time:  2024-07-31T04:44:17Z
    Message:               Kafka ops-request has started to expand volume of kafka nodes.
    Observed Generation:   1
    Reason:                VolumeExpansion
    Status:                True
    Type:                  VolumeExpansion
    Last Transition Time:  2024-07-31T04:44:25Z
    Message:               get pet set; ConditionStatus:True
    Observed Generation:   1
    Status:                True
    Type:                  GetPetSet
    Last Transition Time:  2024-07-31T04:44:25Z
    Message:               is petset deleted; ConditionStatus:True
    Observed Generation:   1
    Status:                True
    Type:                  IsPetsetDeleted
    Last Transition Time:  2024-07-31T04:44:45Z
    Message:               successfully deleted the petSets with orphan propagation policy
    Observed Generation:   1
    Reason:                OrphanPetSetPods
    Status:                True
    Type:                  OrphanPetSetPods
    Last Transition Time:  2024-07-31T04:44:50Z
    Message:               get pvc; ConditionStatus:True
    Observed Generation:   1
    Status:                True
    Type:                  GetPvc
    Last Transition Time:  2024-07-31T04:44:50Z
    Message:               is pvc patched; ConditionStatus:True
    Observed Generation:   1
    Status:                True
    Type:                  IsPvcPatched
    Last Transition Time:  2024-07-31T04:44:55Z
    Message:               compare storage; ConditionStatus:True
    Observed Generation:   1
    Status:                True
    Type:                  CompareStorage
    Last Transition Time:  2024-07-31T04:45:10Z
    Message:               successfully updated controller node PVC sizes
    Observed Generation:   1
    Reason:                UpdateControllerNodePVCs
    Status:                True
    Type:                  UpdateControllerNodePVCs
    Last Transition Time:  2024-07-31T04:45:35Z
    Message:               successfully updated broker node PVC sizes
    Observed Generation:   1
    Reason:                UpdateBrokerNodePVCs
    Status:                True
    Type:                  UpdateBrokerNodePVCs
    Last Transition Time:  2024-07-31T04:45:42Z
    Message:               successfully reconciled the Kafka resources
    Observed Generation:   1
    Reason:                UpdatePetSets
    Status:                True
    Type:                  UpdatePetSets
    Last Transition Time:  2024-07-31T04:45:47Z
    Message:               PetSet is recreated
    Observed Generation:   1
    Reason:                ReadyPetSets
    Status:                True
    Type:                  ReadyPetSets
    Last Transition Time:  2024-07-31T04:45:47Z
    Message:               Successfully completed volumeExpansion for kafka
    Observed Generation:   1
    Reason:                Successful
    Status:                True
    Type:                  Successful
  Observed Generation:     1
  Phase:                   Successful
Events:
  Type     Reason                                   Age   From                         Message
  ----     ------                                   ----  ----                         -------
  Normal   Starting                                 116s  KubeDB Ops-manager Operator  Start processing for KafkaOpsRequest: demo/kf-volume-exp-topology
  Normal   Starting                                 116s  KubeDB Ops-manager Operator  Pausing Kafka databse: demo/kafka-prod
  Normal   Successful                               116s  KubeDB Ops-manager Operator  Successfully paused Kafka database: demo/kafka-prod for KafkaOpsRequest: kf-volume-exp-topology
  Warning  get pet set; ConditionStatus:True        108s  KubeDB Ops-manager Operator  get pet set; ConditionStatus:True
  Warning  is petset deleted; ConditionStatus:True  108s  KubeDB Ops-manager Operator  is petset deleted; ConditionStatus:True
  Warning  get pet set; ConditionStatus:True        103s  KubeDB Ops-manager Operator  get pet set; ConditionStatus:True
  Warning  get pet set; ConditionStatus:True        98s   KubeDB Ops-manager Operator  get pet set; ConditionStatus:True
  Warning  is petset deleted; ConditionStatus:True  98s   KubeDB Ops-manager Operator  is petset deleted; ConditionStatus:True
  Warning  get pet set; ConditionStatus:True        93s   KubeDB Ops-manager Operator  get pet set; ConditionStatus:True
  Normal   OrphanPetSetPods                         88s   KubeDB Ops-manager Operator  successfully deleted the petSets with orphan propagation policy
  Warning  get pvc; ConditionStatus:True            83s   KubeDB Ops-manager Operator  get pvc; ConditionStatus:True
  Warning  is pvc patched; ConditionStatus:True     83s   KubeDB Ops-manager Operator  is pvc patched; ConditionStatus:True
  Warning  get pvc; ConditionStatus:True            78s   KubeDB Ops-manager Operator  get pvc; ConditionStatus:True
  Warning  compare storage; ConditionStatus:True    78s   KubeDB Ops-manager Operator  compare storage; ConditionStatus:True
  Warning  get pvc; ConditionStatus:True            73s   KubeDB Ops-manager Operator  get pvc; ConditionStatus:True
  Warning  is pvc patched; ConditionStatus:True     73s   KubeDB Ops-manager Operator  is pvc patched; ConditionStatus:True
  Warning  get pvc; ConditionStatus:True            68s   KubeDB Ops-manager Operator  get pvc; ConditionStatus:True
  Warning  compare storage; ConditionStatus:True    68s   KubeDB Ops-manager Operator  compare storage; ConditionStatus:True
  Normal   UpdateControllerNodePVCs                 63s   KubeDB Ops-manager Operator  successfully updated controller node PVC sizes
  Warning  get pvc; ConditionStatus:True            58s   KubeDB Ops-manager Operator  get pvc; ConditionStatus:True
  Warning  is pvc patched; ConditionStatus:True     58s   KubeDB Ops-manager Operator  is pvc patched; ConditionStatus:True
  Warning  get pvc; ConditionStatus:True            53s   KubeDB Ops-manager Operator  get pvc; ConditionStatus:True
  Warning  compare storage; ConditionStatus:True    53s   KubeDB Ops-manager Operator  compare storage; ConditionStatus:True
  Warning  get pvc; ConditionStatus:True            48s   KubeDB Ops-manager Operator  get pvc; ConditionStatus:True
  Warning  is pvc patched; ConditionStatus:True     48s   KubeDB Ops-manager Operator  is pvc patched; ConditionStatus:True
  Warning  get pvc; ConditionStatus:True            43s   KubeDB Ops-manager Operator  get pvc; ConditionStatus:True
  Warning  compare storage; ConditionStatus:True    43s   KubeDB Ops-manager Operator  compare storage; ConditionStatus:True
  Normal   UpdateBrokerNodePVCs                     38s   KubeDB Ops-manager Operator  successfully updated broker node PVC sizes
  Normal   UpdatePetSets                            31s   KubeDB Ops-manager Operator  successfully reconciled the Kafka resources
  Warning  get pet set; ConditionStatus:True        26s   KubeDB Ops-manager Operator  get pet set; ConditionStatus:True
  Warning  get pet set; ConditionStatus:True        26s   KubeDB Ops-manager Operator  get pet set; ConditionStatus:True
  Normal   ReadyPetSets                             26s   KubeDB Ops-manager Operator  PetSet is recreated
  Normal   Starting                                 26s   KubeDB Ops-manager Operator  Resuming Kafka database: demo/kafka-prod
  Normal   Successful                               26s   KubeDB Ops-manager Operator  Successfully resumed Kafka database: demo/kafka-prod for KafkaOpsRequest: kf-volume-exp-topology

Now, we are going to verify from the Petset, and the Persistent Volumes whether the volume of the database has expanded to meet the desired state, Let’s check,

$ kubectl get petset -n demo kafka-prod-broker -o json | jq '.spec.volumeClaimTemplates[].spec.resources.requests.storage'
"3Gi"

$ kubectl get petset -n demo kafka-prod-controller -o json | jq '.spec.volumeClaimTemplates[].spec.resources.requests.storage'
"2Gi"

$ kubectl get pv -n demo                                       
NAME                   CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS     CLAIM                                           STORAGECLASS   REASON   AGE
pvc-3f177a92721440bb   1Gi        RWO            Delete           Bound      demo/kafka-prod-data-kafka-prod-controller-0    standard                5m25s
pvc-86ff354122324b1c   1Gi        RWO            Delete           Bound      demo/kafka-prod-data-kafka-prod-broker-1        standard                4m51s
pvc-9fa35d773aa74bd0   1Gi        RWO            Delete           Bound      demo/kafka-prod-data-kafka-prod-controller-1    standard                5m1s
pvc-ccf50adf179e4162   1Gi        RWO            Delete           Bound      demo/kafka-prod-data-kafka-prod-broker-0        standard                5m30s

The above output verifies that we have successfully expanded the volume of the Kafka.

Cleaning Up

To clean up the Kubernetes resources created by this tutorial, run:

kubectl delete kafkaopsrequest -n demo kf-volume-exp-topology
kubectl delete kf -n demo kafka-prod
kubectl delete ns demo

Next Steps

What's on this Page

Search
Improve This Page