New to KubeDB? Please start here.
Restart Etcd
KubeDB supports restarting an Etcd cluster through an EtcdOpsRequest. Restarting is useful if
some member pods are stuck, if a node has to be drained, or if you simply want the members to come
up fresh. This tutorial shows you how to use it.
Unlike a plain kubectl delete pod, this restart is Raft aware. Evicting the current Raft
leader forces an election, and an election makes the whole cluster unavailable for writes until a
new leader is elected. So the operator:
- Resolves which pod currently holds the Raft leadership.
- Restarts every follower first, one at a time, waiting for each pod to become
Readyand for the cluster to regain quorum before moving to the next one. - Restarts the leader last, and only after transferring its leadership to another healthy
voting member with etcd’s
MoveLeaderRPC — a transfer is near instantaneous, an election is not.
A single member cluster (spec.replicas: 1) has nowhere to hand the leadership to, so it is simply
restarted like any other pod.
Before You Begin
At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using kind.
Now, install KubeDB cli on your workstation and KubeDB operator in your cluster following the steps here. Etcd support is behind an alpha feature gate, so make sure the KubeDB Provisioner and Ops-manager operators are installed with
featureGates.Etcd=true.You should be familiar with the following
KubeDBconcepts:To keep things isolated, this tutorial uses a separate namespace called
demothroughout this tutorial.
$ kubectl create ns demo
namespace/demo created
Note: YAML files used in this tutorial are stored in docs/examples/etcd folder in GitHub repository kubedb/docs.
Deploy Etcd
In this section, we are going to deploy a 3 member Etcd cluster using KubeDB.
apiVersion: kubedb.com/v1alpha2
kind: Etcd
metadata:
name: etcd-quickstart
namespace: demo
spec:
version: "3.6.4"
replicas: 3
storageType: Durable
storage:
storageClassName: "standard"
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
deletionPolicy: WipeOut
Let’s create the Etcd CR we have shown above,
$ kubectl create -f https://github.com/kubedb/docs/raw/v2026.11.15/docs/examples/etcd/restart/etcd.yaml
etcd.kubedb.com/etcd-quickstart created
Now, wait until etcd-quickstart has status Ready. i.e,
$ kubectl get etcd -n demo
NAME VERSION STATUS AGE
etcd-quickstart 3.6.4 Ready 3m
$ kubectl get pods -n demo -l app.kubernetes.io/instance=etcd-quickstart
NAME READY STATUS RESTARTS AGE
etcd-quickstart-0 1/1 Running 0 3m
etcd-quickstart-1 1/1 Running 0 2m
etcd-quickstart-2 1/1 Running 0 2m
Apply Restart opsRequest
apiVersion: ops.kubedb.com/v1alpha1
kind: EtcdOpsRequest
metadata:
name: etcd-restart
namespace: demo
spec:
type: Restart
databaseRef:
name: etcd-quickstart
timeout: 5m
apply: Always
spec.typespecifies the type of the ops request. Here it isRestart.spec.databaseRefholds the name of theEtcdobject. The database must be in the same namespace as the ops request.spec.timeoutbounds how long a single step (evicting a pod, waiting for it to be ready, waiting for quorum) may keep retrying before the request is failed.- The meaning of
spec.timeout&spec.applyfields is described here.
Note: Restarting a single member cluster and a multi member cluster is done exactly the same way. All you need is to point
spec.databaseRef.nameat the correspondingEtcdobject.
Let’s create the EtcdOpsRequest CR we have shown above,
$ kubectl create -f https://github.com/kubedb/docs/raw/v2026.11.15/docs/examples/etcd/restart/ops.yaml
etcdopsrequest.ops.kubedb.com/etcd-restart created
Track the progress
Now the Ops-manager operator will pause the Etcd object, restart the member pods one at a time —
followers first, leader last — and finally resume the object.
$ kubectl get etcdopsrequest -n demo
NAME TYPE STATUS AGE
etcd-restart Restart Progressing 35s
While it runs, you can watch the pods being recreated one at a time. The cluster keeps quorum throughout, because the operator never evicts the next pod until the previous one is back in the Raft quorum:
$ kubectl get pods -n demo -l app.kubernetes.io/instance=etcd-quickstart -w
NAME READY STATUS RESTARTS AGE
etcd-quickstart-0 1/1 Terminating 0 8m
etcd-quickstart-1 1/1 Running 0 8m
etcd-quickstart-2 1/1 Running 0 7m
kubectl describe gives you the step-by-step view, including the events the operator records for
each retryable step:
$ kubectl describe etcdopsrequest -n demo etcd-restart
Name: etcd-restart
Namespace: demo
Labels: <none>
Annotations: <none>
API Version: ops.kubedb.com/v1alpha1
Kind: EtcdOpsRequest
Metadata:
Creation Timestamp: 2026-02-10T09:22:57Z
Generation: 1
Resource Version: 1072309
UID: 6091d9fa-1c2b-4734-bdd1-1ace91460bea
Spec:
Apply: Always
Database Ref:
Name: etcd-quickstart
Timeout: 5m
Type: Restart
Status:
Conditions:
Last Transition Time: 2026-02-10T09:22:57Z
Message: Restart is in progress
Observed Generation: 1
Reason: Running
Status: True
Type: Running
Last Transition Time: 2026-02-10T09:23:10Z
Message: check pod ready; ConditionStatus:True; PodName:etcd-quickstart-0
Observed Generation: 1
Status: True
Type: CheckPodReady--etcd-quickstart-0
Observed Generation: 1
Phase: Progressing
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal PauseDatabase 35s KubeDB Ops-manager Operator Pausing Etcd demo/etcd-quickstart
Warning check pod ready; ConditionStatus:False; PodName:etcd-quickstart-0 30s KubeDB Ops-manager Operator check pod ready; ConditionStatus:False; PodName:etcd-quickstart-0
Warning check pod ready; ConditionStatus:True; PodName:etcd-quickstart-0 5s KubeDB Ops-manager Operator check pod ready; ConditionStatus:True; PodName:etcd-quickstart-0
Once every member has been recreated, the request reaches Successful:
$ kubectl get etcdopsrequest -n demo
NAME TYPE STATUS AGE
etcd-restart Restart Successful 4m
Here is the full object after a successful run. In this example the Raft leader happened to be
etcd-quickstart-1, so that pod was restarted last, after its leadership had been moved away:
$ kubectl get etcdopsrequest -n demo etcd-restart -o yaml
apiVersion: ops.kubedb.com/v1alpha1
kind: EtcdOpsRequest
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"ops.kubedb.com/v1alpha1","kind":"EtcdOpsRequest","metadata":{"annotations":{},"name":"etcd-restart","namespace":"demo"},"spec":{"apply":"Always","databaseRef":{"name":"etcd-quickstart"},"timeout":"5m","type":"Restart"}}
creationTimestamp: "2026-02-10T09:22:57Z"
generation: 1
name: etcd-restart
namespace: demo
resourceVersion: "1072471"
uid: 6091d9fa-1c2b-4734-bdd1-1ace91460bea
spec:
apply: Always
databaseRef:
name: etcd-quickstart
maxRetries: 1
timeout: 5m
type: Restart
status:
conditions:
- lastTransitionTime: "2026-02-10T09:22:57Z"
message: Restart is in progress
observedGeneration: 1
reason: Running
status: "True"
type: Running
- lastTransitionTime: "2026-02-10T09:23:35Z"
message: check pod ready; ConditionStatus:True; PodName:etcd-quickstart-0
observedGeneration: 1
status: "True"
type: CheckPodReady--etcd-quickstart-0
- lastTransitionTime: "2026-02-10T09:23:40Z"
message: etcd cluster healthy; ConditionStatus:True; PodName:etcd-quickstart-0
observedGeneration: 1
status: "True"
type: EtcdClusterHealthy--etcd-quickstart-0
- lastTransitionTime: "2026-02-10T09:24:15Z"
message: check pod ready; ConditionStatus:True; PodName:etcd-quickstart-2
observedGeneration: 1
status: "True"
type: CheckPodReady--etcd-quickstart-2
- lastTransitionTime: "2026-02-10T09:24:20Z"
message: etcd cluster healthy; ConditionStatus:True; PodName:etcd-quickstart-2
observedGeneration: 1
status: "True"
type: EtcdClusterHealthy--etcd-quickstart-2
- lastTransitionTime: "2026-02-10T09:24:30Z"
message: move leader; ConditionStatus:True; PodName:etcd-quickstart-1
observedGeneration: 1
status: "True"
type: MoveLeader--etcd-quickstart-1
- lastTransitionTime: "2026-02-10T09:25:05Z"
message: check pod ready; ConditionStatus:True; PodName:etcd-quickstart-1
observedGeneration: 1
status: "True"
type: CheckPodReady--etcd-quickstart-1
- lastTransitionTime: "2026-02-10T09:25:10Z"
message: etcd cluster healthy; ConditionStatus:True; PodName:etcd-quickstart-1
observedGeneration: 1
status: "True"
type: EtcdClusterHealthy--etcd-quickstart-1
- lastTransitionTime: "2026-02-10T09:25:15Z"
message: Successfully restarted all the etcd members
observedGeneration: 1
reason: RestartEtcdPods
status: "True"
type: RestartEtcdPods
- lastTransitionTime: "2026-02-10T09:25:16Z"
message: Successfully restarted the etcd cluster
observedGeneration: 1
reason: Successful
status: "True"
type: Successful
observedGeneration: 1
phase: Successful
Reading the conditions:
Runningis set as soon as the request leaves thePendingphase.RestartEtcdPodstracks the whole rolling sequence. It is first written withstatus: "False"when the sequence starts and flips to"True"once every member has been recreated.MoveLeader--<pod>is the leadership handoff. It appears for exactly one pod — whichever one held the Raft leadership when the request started — and always before that pod is evicted. On a single member cluster it never appears, because there is no other voter to hand the leadership to.CheckPodReady--<pod>andEtcdClusterHealthy--<pod>are the two gates the operator waits on after evicting each pod: the pod must be recreated andReady, and the cluster must have quorum again (healthy members >= N/2 + 1), before the next member is touched.Successfulis the terminal condition, written after theEtcdobject is resumed and handed back to the Provisioner operator.
Note: the per-pod conditions are not just retry bookkeeping — they are how the operator remembers where it got to. A completed step writes a
Truecondition unconditionally, soGetLeader--<pod>,EvictPod--<pod>,CheckPodReady--<pod>andEtcdClusterHealthy--<pod>are present for every pod the restart touched, on every run, whether or not anything had to be retried. A step that is still waiting shows up as the same condition withStatus: False. Some of these are elided from the output above for brevity.
If you want to transfer the Raft leadership on its own, without restarting anything, use the
dedicated MoveLeader ops request type instead — it records an EtcdLeaderMoved condition and
optionally takes spec.moveLeader.newLeader to name the target member.
Cleaning up
To cleanup the Kubernetes resources created by this tutorial, run:
kubectl delete etcdopsrequest -n demo etcd-restart
kubectl delete etcd -n demo etcd-quickstart
kubectl delete ns demo
Next Steps
- Detail concepts of Etcd object.
- Detail concepts of EtcdOpsRequest object.
- Change the etcd tuning knobs of a running cluster with Reconfigure.
- Rotate the etcd credentials with Rotate Authentication.
- Want to hack on KubeDB? Check our contribution guidelines.

































