You are looking at the documentation of a prior release. To read the documentation of the latest release, please
visit here.
New to KubeDB? Please start here.
KubeDB Elasticsearch supports multi-node database cluster.
At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using minikube.
Now, install KubeDB cli on your workstation and KubeDB operator in your cluster following the steps here.
To keep things isolated, this tutorial uses a separate namespace called demo
throughout this tutorial.
$ kubectl create ns demo
namespace "demo" created
$ kubectl get ns demo
NAME STATUS AGE
demo Active 5s
Note: Yaml files used in this tutorial are stored in docs/examples/elasticsearch folder in github repository kubedb/cli.
Elasticsearch can be created with multiple nodes. If you want to create Elasticsearch cluster with three nodes, you need to set spec.replicas
to 3
.
In this case, all of these three nodes will act as master, data and client.
Check following Elasticsearch object
apiVersion: kubedb.com/v1alpha1
kind: Elasticsearch
metadata:
name: multi-node-es
namespace: demo
spec:
version: 5.6
replicas: 3
storage:
storageClassName: "standard"
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 50Mi
Here,
spec.replicas
is the number of nodes in the Elasticsearch cluster. Here, we are creating a three node Elasticsearch cluster.Note: If
spec.topology
is set,spec.replicas
has no effect.
Create example above with following command
$ kubedb create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.2/docs/examples/elasticsearch/clustering/multi-node-es.yaml
validating "https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.2/docs/examples/elasticsearch/clustering/multi-node-es.yaml"
elasticsearch "multi-node-es" created
Lets describe Elasticsearch object multi-node-es
while Running
$ kubedb describe es -S=false -W=false -n demo multi-node-es
Name: multi-node-es
Namespace: demo
CreationTimestamp: Tue, 20 Feb 2018 14:36:03 +0600
Status: Running
Replicas: 3 total
Volume:
StorageClass: standard
Capacity: 50Mi
Access Modes: RWO
StatefulSet: multi-node-es
Service: multi-node-es, multi-node-es-master
Secrets: multi-node-es-auth, multi-node-es-cert
Topology:
Type Pod StartTime Phase
---- --- --------- -----
master|client|data multi-node-es-0 2018-02-20 14:36:13 +0600 +06 Running
master|client|data multi-node-es-1 2018-02-20 14:36:24 +0600 +06 Running
master|client|data multi-node-es-2 2018-02-20 14:36:45 +0600 +06 Running
No Snapshots.
Events:
FirstSeen LastSeen Count From Type Reason Message
--------- -------- ----- ---- -------- ------ -------
12m 12m 1 Elasticsearch operator Normal Successful Successfully patched Elasticsearch
13m 13m 1 Elasticsearch operator Normal Successful Successfully patched StatefulSet
13m 13m 1 Elasticsearch operator Normal Successful Successfully created Elasticsearch
13m 13m 1 Elasticsearch operator Normal Successful Successfully created StatefulSet
14m 14m 1 Elasticsearch operator Normal Successful Successfully created Service
14m 14m 1 Elasticsearch operator Normal Successful Successfully created Service
Here, we can see in Topology section that all three Pods are acting as master, data and client.
If you want to use separate node for master, data and client role, you need to configure spec.topology
In this tutorial, we will create following Elasticsearch with topology
apiVersion: kubedb.com/v1alpha1
kind: Elasticsearch
metadata:
name: topology-es
namespace: demo
spec:
version: 5.6
topology:
master:
prefix: master
replicas: 1
data:
prefix: data
replicas: 2
client:
prefix: client
replicas: 2
storage:
storageClassName: "standard"
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 50Mi
Here,
spec.topology
point to the number of pods we want as dedicated master
, client
and data
nodes and also specify prefix for their StatefulSet nameLets create this Elasticsearch object
$ kubedb create -f https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.2/docs/examples/elasticsearch/clustering/topology-es.yaml
validating "https://raw.githubusercontent.com/kubedb/cli/0.8.0-beta.2/docs/examples/elasticsearch/clustering/topology-es.yaml"
elasticsearch "topology-es" created
When this object is created, Elasticsearch database has started with 5 pods under 3 different StatefulSets.
$ kubectl get statefulset -n demo --show-labels --selector="kubedb.com/kind=Elasticsearch,kubedb.com/name=topology-es"
NAME DESIRED CURRENT AGE LABELS
client-topology-es 2 2 6m kubedb.com/kind=Elasticsearch,kubedb.com/name=topology-es,node.role.client=set
data-topology-es 2 2 2m kubedb.com/kind=Elasticsearch,kubedb.com/name=topology-es,node.role.data=set
master-topology-es 1 1 2m kubedb.com/kind=Elasticsearch,kubedb.com/name=topology-es,node.role.master=set
Three StatefulSets are created
client-topology-es
spec:
topology:
client:
prefix: client
replicas: 2
This configuration creates a StatefulSet named client-topology-es
for client node
spec.replicas
is set to 2
. Two dedicated nodes is created as client.node.role.client: set
is added in Podsdata-topology-es
spec:
topology:
data:
prefix: data
replicas: 2
This configuration creates a StatefulSet named data-topology-es
for data node
spec.replicas
is set to 2
. Two dedicated nodes is created for data.master-topology-es
spec:
topology:
master:
prefix: master
replicas: 1
This configuration creates a StatefulSet named data-topology-es
for master node
spec.replicas
is set to 1
. One dedicated node is created as master.node.role.master: set
is added in PodsNote: StatefulSet name format:
{topology-prefix}-{elasticsearch-name}
Lets describe this Elasticsearch
$ kubedb describe es -S=false -W=false -n demo topology-es
Name: topology-es
Namespace: demo
CreationTimestamp: Tue, 20 Feb 2018 16:34:43 +0600
Status: Running
Replicas: 0 total
Volume:
StorageClass: standard
Capacity: 50Mi
Access Modes: RWO
StatefulSet: client-topology-es, data-topology-es, master-topology-es
Service: topology-es, topology-es-master
Secrets: topology-es-auth, topology-es-cert
Topology:
Type Pod StartTime Phase
---- --- --------- -----
client client-topology-es-0 2018-02-20 16:34:50 +0600 +06 Running
client client-topology-es-1 2018-02-20 16:38:23 +0600 +06 Running
data data-topology-es-0 2018-02-20 16:39:12 +0600 +06 Running
data data-topology-es-1 2018-02-20 16:39:40 +0600 +06 Running
master master-topology-es-0 2018-02-20 16:38:44 +0600 +06 Running
No Snapshots.
Events:
FirstSeen LastSeen Count From Type Reason Message
--------- -------- ----- ---- -------- ------ -------
23m 23m 1 Elasticsearch operator Normal Successful Successfully patched Elasticsearch
24m 24m 1 Elasticsearch operator Normal Successful Successfully patched StatefulSet
24m 24m 1 Elasticsearch operator Normal Successful Successfully patched StatefulSet
24m 24m 1 Elasticsearch operator Normal Successful Successfully patched StatefulSet
24m 24m 1 Elasticsearch operator Normal Successful Successfully created Elasticsearch
25m 25m 1 Elasticsearch operator Normal Successful Successfully created StatefulSet
26m 26m 1 Elasticsearch operator Normal Successful Successfully created StatefulSet
26m 26m 1 Elasticsearch operator Normal Successful Successfully created StatefulSet
30m 30m 1 Elasticsearch operator Normal Successful Successfully created Service
30m 30m 1 Elasticsearch operator Normal Successful Successfully created Service
We can see in Topology section that
Two Pods are dedicated as client
Topology:
Type Pod StartTime Phase
---- --- --------- -----
client client-topology-es-0 2018-02-20 16:34:50 +0600 +06 Running
client client-topology-es-1 2018-02-20 16:38:23 +0600 +06 Running
Two Pods for data node
Topology:
Type Pod StartTime Phase
---- --- --------- -----
data data-topology-es-0 2018-02-20 16:39:12 +0600 +06 Running
data data-topology-es-1 2018-02-20 16:39:40 +0600 +06 Running
And one Pod as master node
Topology:
Type Pod StartTime Phase
---- --- --------- -----
master master-topology-es-0 2018-02-20 16:38:44 +0600 +06 Running
Two services are also created for this Elasticsearch object.
Service quick-elasticsearch
targets all Pods which are acting as client node
Service quick-elasticsearch-master
targets all Pods which are acting as master node
To cleanup the Kubernetes resources created by this tutorial, run:
$ kubedb delete es,drmn,snap -n demo --all --force
$ kubectl delete ns demo
namespace "demo" deleted
out-of-the-box
builtin-Prometheus.out-of-the-box
CoreOS Prometheus Operator.