New to KubeDB? Please start here.

Kafka Branch Configuration

These optional examples reuse kafka-source from the same-cluster tutorial. Keep the source and namespaces before that tutorial’s cleanup. Each Branch below has a distinct target name.

Storage and resource overrides

target.storageClassName selects the clone’s StorageClass; it must resolve the source snapshot on the same backend. target.resources configures the target database containers independently of source resource requests. The manifests below set both explicitly. See snapshot configuration for driver and backend requirements.

Select a snapshot class

Create a compatible ceph-rbd-snapclass before this example. volumeSnapshotClassName selects the per-volume/import snapshot class; the compatible VolumeGroupSnapshotClass is discovered separately. Individual fallback snapshots do not provide an atomic snapshot across volumes.

Save the following as kafka-snapshot.yaml:

apiVersion: courier.kubedb.com/v1alpha1
kind: Branch
metadata:
  name: kafka-snapshot
  namespace: demo
spec:
  source:
    databaseRef:
      apiGroup: kubedb.com
      kind: Kafka
      name: kafka-source
    namespace: demo
  target:
    namespace: dev
    name: kafka-snapshot
    storageClassName: ceph-rbd
    resources:
      requests:
        cpu: 500m
        memory: 1200Mi
      limits:
        memory: 1200Mi
  volumeSnapshotClassName: ceph-rbd-snapclass
  deletionPolicy: Delete
kubectl -n demo apply -f kafka-snapshot.yaml
kubectl -n demo wait branch/kafka-snapshot --for=condition=Ready --timeout=60m
kubectl -n demo get branch kafka-snapshot
kubectl -n demo get volumesnapshots,volumegroupsnapshots
kubectl -n dev get volumesnapshots

For cross-namespace or cross-cluster imports, the named VolumeSnapshotClass must exist wherever it is used. Both clusters must resolve the snapshot handle against the same Ceph backend.

Deletion policies

PolicyWhen the Branch is deleted
Delete (default)Remove the managed target database, cloned storage, action Jobs, and branch snapshot resources.
OrphanStop branch management and refresh, retain the target database and required storage/credentials, and clean up branch snapshot resources.

The source database and namespaces remain under either policy. Keep the controllers and storage available until finalizer cleanup completes.

Retain a branch with Orphan

Save the following as kafka-kept.yaml:

apiVersion: courier.kubedb.com/v1alpha1
kind: Branch
metadata:
  name: kafka-kept
  namespace: demo
spec:
  source:
    databaseRef:
      apiGroup: kubedb.com
      kind: Kafka
      name: kafka-source
    namespace: demo
  target:
    namespace: dev
    name: kafka-kept
    storageClassName: ceph-rbd
    resources:
      requests:
        cpu: 500m
        memory: 1200Mi
      limits:
        memory: 1200Mi
  deletionPolicy: Orphan
kubectl -n demo apply -f kafka-kept.yaml
kubectl -n demo wait branch/kafka-kept --for=condition=Ready --timeout=60m
kubectl -n demo get branch kafka-kept
kubectl -n demo delete branch kafka-kept --wait=true --timeout=30m
kubectl -n dev get kafka kafka-kept
kubectl -n dev get pvc,secrets -l app.kubernetes.io/instance=kafka-kept

Verify that the retained database still serves its copied data:

From your terminal, open a pod shell:

kubectl -n dev exec -it kafka-kept-0 -c kafka -- bash

Run the following Kafka commands inside this Bash shell. The operator has written the broker pod’s SASL settings to /opt/kafka/config/clientauth.properties. Use the headless Service for bootstrap; Kafka advertises the individual broker addresses after connection.

kafka-get-offsets.sh --bootstrap-server kafka-kept-pods.dev.svc.cluster.local:9092 \
  --command-config /opt/kafka/config/clientauth.properties --topic customers --time latest
customers:0:6
kafka-console-consumer.sh --bootstrap-server kafka-kept-pods.dev.svc.cluster.local:9092 \
  --consumer.config /opt/kafka/config/clientauth.properties --topic customers --from-beginning --max-messages 6 --timeout-ms 20000
{"id": 1, "full_name": "Customer 1", "email": "[email protected]"}
{"id": 2, "full_name": "Customer 2", "email": "[email protected]"}
{"id": 3, "full_name": "Customer 3", "email": "[email protected]"}
{"id": 4, "full_name": "Customer 4", "email": "[email protected]"}
{"id": 5, "full_name": "Customer 5", "email": "[email protected]"}
SOURCE-ONLY

Run exit to return to your terminal.

The results above assume you completed the source edit in the same-cluster tutorial. Preserve the retained auth Secret; the database still needs its credential. To remove the retained deployment and its data explicitly:

kubectl -n dev patch kafka kafka-kept --type=merge -p '{"spec":{"deletionPolicy":"WipeOut"}}'
kubectl -n dev delete kafka kafka-kept --wait=true --timeout=30m

TLS-enabled sources

Configure a TLS-enabled source following the Kafka TLS guide before creating this optional Branch. Use that TLS-enabled source’s name in source.databaseRef.name below if it differs from kafka-source. Prepare cert-manager and a target Issuer branch-ca in dev, including its signing Secret. Courier delivers database resources; it does not create this issuer.

Save the following as kafka-tls.yaml:

apiVersion: courier.kubedb.com/v1alpha1
kind: Branch
metadata:
  name: kafka-tls
  namespace: demo
spec:
  source:
    databaseRef:
      apiGroup: kubedb.com
      kind: Kafka
      name: kafka-source
    namespace: demo
  target:
    namespace: dev
    name: kafka-tls
    storageClassName: ceph-rbd
    resources:
      requests:
        cpu: 500m
        memory: 1200Mi
      limits:
        memory: 1200Mi
    issuerRef:
      apiGroup: cert-manager.io
      kind: Issuer
      name: branch-ca
  deletionPolicy: Delete
kubectl -n demo apply -f kafka-tls.yaml
kubectl -n demo wait branch/kafka-tls --for=condition=Ready --timeout=60m
kubectl -n demo get branch kafka-tls
kubectl -n dev get certificates
kubectl -n dev get kafka kafka-tls -o yaml

Confirm that target certificates are ready and use branch-specific names. A TLS source’s broker client file uses SASL_SSL and a truststore. Use the target broker’s generated /opt/kafka/config/clientauth.properties for verification. A separate action pod needs its own matching TLS client configuration and mounted trust material; the plaintext post-action file is not sufficient.

Use kind: ClusterIssuer for a cluster-scoped issuer. For OCM, the issuer must exist on the target cluster, and a namespaced Issuer and its supporting Secret must be in the target namespace.

Cleanup optional examples

kubectl -n demo delete branch kafka-snapshot kafka-tls --ignore-not-found --wait=true --timeout=30m

Return to the same-cluster cleanup when all dependent Branches are gone.