Monitoring Database with KubeDB

KubeDB has native support for monitoring via Prometheus. You can use builtin Prometheus scraper or CoreOS Prometheus Operator to monitor KubeDB managed databases. This tutorial will show you how database monitoring works with KubeDB and how to configure Database crd to enable monitoring.

Overview

KubeDB uses Prometheus exporter images to export Prometheus metrics for respective databases. Following diagram shows the logical flow of database monitoring with KubeDB.

  Database Monitoring Flow

When a user creates a database crd with spec.monitor section configured, KubeDB operator provisions the respective database and injects an exporter image as sidecar to the database pod. It also creates a dedicated stats service with name {database-crd-name}-stats for monitoring. Prometheus server can scrape metrics using this stats service.

Configure Monitoring

In order to enable monitoring for a database, you have to configure spec.monitor section. KubeDB provides following options to configure spec.monitor section:

FieldTypeUses
spec.monitor.agentRequiredType of the monitoring agent that will be used to monitor this database. It can be prometheus.io/builtin or prometheus.io/coreos-operator.
spec.monitor.prometheus.namespaceOptionalNamespace where the Prometheus server is running or will be deployed. For agent type prometheus.io/coreos-operator, ServiceMonitor crd will be created in this namespace.
spec.monitor.prometheus.labelsOptionalLabels for ServiceMonitor crd.
spec.monitor.prometheus.portOptionalPort number where the exporter side car will serve metrics.
spec.monitor.prometheus.intervalOptionalInterval at which metrics should be scraped.
spec.monitor.argsOptionalArguments to pass to the exporter sidecar.
spec.monitor.envOptionalList of environment variables to set in the exporter sidecar container.
spec.monitor.resourcesOptionalResources required by exporter sidecar container.
spec.monitor.securityContextOptionalSecurity options the exporter should run with.

Sample Configuration

A sample YAML for Redis crd with spec.monitor section configured to enable monitoring with CoreOS prometheus-operator is shown below.

apiVersion: kubedb.com/v1alpha1
kind: Redis
metadata:
  name: sample-redis
  namespace: databases
spec:
  version: "4.0-v1"
  terminationPolicy: WipeOut
  configSource: # configure Redis to use password for authentication
    configMap:
      name: redis-config
  storageType: Durable
  storage:
    storageClassName: default
    accessModes:
    - ReadWriteOnce
    resources:
      requests:
        storage: 5Gi
  monitor:
    agent: prometheus.io/coreos-operator
    prometheus:
      namespace: monitoring
      labels:
        k8s-app: prometheus
    args:
    - --redis.password=$(REDIS_PASSWORD)
    env:
    - name: REDIS_PASSWORD
      valueFrom:
        secretKeyRef:
          name: _name_of_secret_with_redis_password
          key: password # key with the password
    resources:
      requests:
        memory: 512Mi
        cpu: 200m
      limits:
        memory: 512Mi
        cpu: 250m
    securityContext:
      runAsUser: 2000
      allowPrivilegeEscalation: false

Assume that above Redis is configured to use basic authentication. So, exporter image also need to provide password to collect metrics. We have provided it through spec.monitor.args field.

Here, we have specified that we are going to monitor this server using CoreOS prometheus-operator through spec.monitor.agent: prometheus.io/coreos-operator. KubeDB will create a ServiceMonitor crd in monitoring namespace and this ServiceMonitor will have k8s-app: prometheus label.

Next Steps