New to KubeDB? Please start here.

ZooKeeper

What is ZooKeeper

ZooKeeper is a Kubernetes Custom Resource Definitions (CRD). It provides declarative configuration for ZooKeeper in a Kubernetes native way. You only need to describe the desired database configuration in a ZooKeeper object, and the KubeDB operator will create Kubernetes objects in the desired state for you.

ZooKeeper Spec

As with all other Kubernetes objects, a ZooKeeper needs apiVersion, kind, and metadata fields. It also needs a .spec section. Below is an example ZooKeeper object.

apiVersion: kubedb.com/v1alpha2
kind: ZooKeeper
metadata:
  name: zk-ensemble
  namespace: demo
spec:
  version: 3.9.1
  replicas: 3
  disableAuth: false
  adminServerPort: 8080
  authSecret:
    name: zk-auth
    externallyManaged: false
  storage:
    storageClassName: "standard"
    accessModes:
      - ReadWriteOnce
    resources:
      requests:
        storage: 1Gi
  monitor:
    agent: prometheus.io/operator
    prometheus:
      serviceMonitor:
        labels:
          app: kubedb
        interval: 10s
  configSecret:
    name: zk-custom-config
  podTemplate:
    metadata:
      annotations:
        passMe: ToDatabasePod
    controller:
      annotations:
        passMe: ToStatefulSet
    spec:
      serviceAccountName: my-service-account
      schedulerName: my-scheduler
      nodeSelector:
        disktype: ssd
      imagePullSecrets:
        - name: myregistrykey
  serviceTemplates:
    - alias: primary
      metadata:
        annotations:
          passMe: ToService
      spec:
        type: NodePort
        ports:
          - name:  http
            port:  9200
  terminationPolicy: Halt
  halted: false
  healthChecker:
    periodSeconds: 15
    timeoutSeconds: 10
    failureThreshold: 2
    disableWriteCheck: false

spec.version

spec.version is a required field specifying the name of the ZooKeeperVersion crd where the docker images are specified. Currently, when you install KubeDB, it creates the following ZooKeeperVersion crds,

• 3.7.2
• 3.8.3
• 3.9.1

spec.disableAuth

spec.disableAuth is an optional field that decides whether ZooKeeper instance will be secured by auth or no.

spec.authSecret

spec.authSecret is an optional field that points to a Secret used to hold credentials for zookeeper superuser. If not set, KubeDB operator creates a new Secret {zookeeper-object-name}-auth for storing the password for zookeeper superuser.

We can use this field in 3 mode.

1. Using an external secret. In this case, You need to create an auth secret first with required fields, then specify the secret name when creating the ZooKeeper object using spec.authSecret.name & set spec.authSecret.externallyManaged to true.

authSecret:
  name: <your-created-auth-secret-name>
  externallyManaged: true

2. Specifying the secret name only. In this case, You need to specify the secret name when creating the ZooKeeper object using spec.authSecret.name. externallyManaged is by default false.

authSecret:
  name: <intended-auth-secret-name>

3. Let KubeDB do everything for you. In this case, no work for you.

AuthSecret contains a username key and a password key which contains the username and password respectively for zookeeper superuser.

Example:

$ kubectl create secret generic zk-auth -n demo \
--from-literal=username=jhon-doe \
--from-literal=password=6q8u_2jMOW-OOZXk
secret "zk-auth" created

apiVersion: v1
data:
  password: NnE4dV8yak1PVy1PT1pYaw==
  username: amhvbi1kb2U=
kind: Secret
metadata:
  name: zk-auth
  namespace: demo
type: Opaque

Secrets provided by users are not managed by KubeDB, and therefore, won’t be modified or garbage collected by the KubeDB operator (version 0.13.0 and higher).

spec.storage

If you set spec.storageType: to Durable, then spec.storage is a required field that specifies the StorageClass of PVCs dynamically allocated to store data for the database. This storage spec will be passed to the StatefulSet created by KubeDB operator to run database pods. You can specify any StorageClass available in your cluster with appropriate resource requests.

• spec.storage.storageClassName is the name of the StorageClass used to provision PVCs. PVCs don’t necessarily have to request a class. A PVC with its storageClassName set equal to "" is always interpreted to be requesting a PV with no class, so it can only be bound to PVs with no class (no annotation or one set equal to “”). A PVC with no storageClassName is not quite the same and is treated differently by the cluster depending on whether the DefaultStorageClass admission plugin is turned on.
• spec.storage.accessModes uses the same conventions as Kubernetes PVCs when requesting storage with specific access modes.
• spec.storage.resources can be used to request specific quantities of storage. This follows the same resource model used by PVCs.

To learn how to configure spec.storage, please visit the links below:

• https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims

spec.monitor

ZooKeeper managed by KubeDB can be monitored with builtin-Prometheus and Prometheus operator out-of-the-box.

spec.configSecret

spec.configSecret is an optional field that allows users to provide custom configuration for ZooKeeper. This field accepts a VolumeSource. So you can use any Kubernetes supported volume source such as configMap, secret, azureDisk etc.

spec.podTemplate

KubeDB allows providing a template for database pod through spec.podTemplate. KubeDB operator will pass the information provided in spec.podTemplate to the StatefulSet created for ZooKeeper server.

KubeDB accept following fields to set in spec.podTemplate:

• metadata:
  • annotations (pod’s annotation)
• controller:
  • annotations (statefulset’s annotation)
• spec:
  • args
  • env
  • resources
  • initContainers
  • imagePullSecrets
  • nodeSelector
  • affinity
  • serviceAccountName
  • schedulerName
  • tolerations
  • priorityClassName
  • priority
  • securityContext
  • livenessProbe
  • readinessProbe
  • lifecycle

You can check out the full list here. Uses of some field of spec.podTemplate is described below,

spec.podTemplate.spec.args

spec.podTemplate.spec.args is an optional field. This can be used to provide additional arguments to database installation.

spec.podTemplate.spec.env

spec.podTemplate.spec.env is an optional field that specifies the environment variables to pass to the ZooKeeper docker image.

spec.podTemplate.spec.imagePullSecret

KubeDB provides the flexibility of deploying ZooKeeper server from a private Docker registry.

spec.podTemplate.spec.nodeSelector

spec.podTemplate.spec.nodeSelector is an optional field that specifies a map of key-value pairs. For the pod to be eligible to run on a node, the node must have each of the indicated key-value pairs as labels (it can have additional labels as well). To learn more, see here .

spec.podTemplate.spec.serviceAccountName

serviceAccountName is an optional field supported by KubeDB Operator (version 0.13.0 and higher) that can be used to specify a custom service account to fine tune role based access control.

If this field is left empty, the KubeDB operator will create a service account name matching ZooKeeper crd name. Role and RoleBinding that provide necessary access permissions will also be generated automatically for this service account.

If a service account name is given, but there’s no existing service account by that name, the KubeDB operator will create one, and Role and RoleBinding that provide necessary access permissions will also be generated for this service account.

If a service account name is given, and there’s an existing service account by that name, the KubeDB operator will use that existing service account. Since this service account is not managed by KubeDB, users are responsible for providing necessary access permissions manually.

spec.podTemplate.spec.resources

spec.podTemplate.spec.resources is an optional field. This can be used to request compute resources required by the database pods. To learn more, visit here.

spec.serviceTemplates

You can also provide a template for the services created by KubeDB operator for ZooKeeper server through spec.serviceTemplates. This will allow you to set the type and other properties of the services.

KubeDB allows following fields to set in spec.serviceTemplates:

• alias represents the identifier of the service. It has the following possible value:

  • primary is used for the primary service identification.
  • standby is used for the secondary service identification.
  • stats is used for the exporter service identification.

• metadata:

  • annotations

• spec:

  • type
  • ports
  • clusterIP
  • externalIPs
  • loadBalancerIP
  • loadBalancerSourceRanges
  • externalTrafficPolicy
  • healthCheckNodePort
  • sessionAffinityConfig

See here to understand these fields in detail.

spec.terminationPolicy

terminationPolicy gives flexibility whether to nullify(reject) the delete operation of ZooKeeper crd or which resources KubeDB should keep or delete when you delete ZooKeeper crd. KubeDB provides following four termination policies:

• DoNotTerminate
• Halt
• Delete (Default)
• WipeOut

When terminationPolicy is DoNotTerminate, KubeDB takes advantage of ValidationWebhook feature in Kubernetes 1.9.0 or later clusters to implement DoNotTerminate feature. If admission webhook is enabled, DoNotTerminate prevents users from deleting the database as long as the spec.terminationPolicy is set to DoNotTerminate.

Following table show what KubeDB does when you delete ZooKeeper crd for different termination policies,

spec.halted

Indicates that the database is halted and all offshoot Kubernetes resources except PVCs are deleted.

spec.healthChecker

It defines the attributes for the health checker.

• spec.healthChecker.periodSeconds specifies how often to perform the health check.
• spec.healthChecker.timeoutSeconds specifies the number of seconds after which the probe times out.
• spec.healthChecker.failureThreshold specifies minimum consecutive failures for the healthChecker to be considered failed.
• spec.healthChecker.disableWriteCheck specifies whether to disable the writeCheck or not.

Know details about KubeDB Health checking from this blog post.

Next Steps

• Learn how to use KubeDB to run a ZooKeeper server here.
• Want to hack on KubeDB? Check our contribution guidelines.