You are looking at the documentation of a prior release. To read the documentation of the latest release, please visit here.

New to KubeDB? Please start here.

Continuous Archiving to S3

WAL-G is used to continuously archive PostgreSQL WAL files. Please refer to continuous archiving in KubeDB to learn more about it.

Before You Begin

At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using kind.

Now, install KubeDB cli on your workstation and KubeDB operator in your cluster following the steps here.

Note: YAML files used in this tutorial are stored in docs/examples/postgres folder in GitHub repository kubedb/docs.

To keep things isolated, this tutorial uses a separate namespace called demo throughout this tutorial.

$ kubectl create ns demo
namespace/demo created

Create PostgreSQL with Continuous Archiving

For archiving, we need storage Secret, and storage backend information. Below is a Postgres object created with Continuous Archiving support to backup WAL files to Amazon S3.

kind: Postgres
  name: wal-postgres
  namespace: demo
  version: "11.1-v3"
  replicas: 2
    storageClassName: "standard"
    - ReadWriteOnce
        storage: 1Gi
      storageSecretName: s3-secret
        bucket: kubedb


  • specifies storage information that will be used by WAL-G
    • storage.storageSecretName points to the Secret containing the credentials for cloud storage destination.
    • storage.s3 points to s3 storage configuration.
    • storage.s3.bucket points to the bucket name used to store continuous archiving data.

Archiver Storage Secret

Storage Secret should contain credentials that will be used to access storage destination.

Storage Secret for WAL-G is needed with the following 2 keys:

AWS_ACCESS_KEY_IDRequired. AWS access key ID
AWS_SECRET_ACCESS_KEYRequired. AWS secret access key
$ echo -n '<your-aws-access-key-id-here>' > AWS_ACCESS_KEY_ID
$ echo -n '<your-aws-secret-access-key-here>' > AWS_SECRET_ACCESS_KEY
$ kubectl create secret -n demo generic s3-secret \
    --from-file=./AWS_ACCESS_KEY_ID \
secret "s3-secret" created
$ kubectl get secret -n demo s3-secret -o yaml
apiVersion: v1
kind: Secret
  creationTimestamp: 2018-02-06T09:12:37Z
  name: s3-secret
  namespace: demo
  resourceVersion: "59225"
  selfLink: /api/v1/namespaces/demo/secrets/s3-secret
  uid: dfbe6b06-0b1d-11e8-9fb9-42010a800064
type: Opaque

Archiver Storage Backend

To configure s3 backend, following parameters are available:

ParameterDescription Name of Bucket Path prefix into bucket where snapshot will be stores

Now create this Postgres object with continuous archiving support.

$ kubectl create -f created

When database is ready, WAL-G takes a base backup and uploads it to the cloud storage defined by storage backend.

Archived data is stored in a folder called {bucket}/{prefix}/kubedb/{namespace}/{postgres-name}/archive/.

You can see continuous archiving data stored in S3 bucket.


From the above image, you can see that the archived data is stored in a folder kubedb/kubedb/demo/wal-postgres/archive.

Termination Policy

If termination policy of this wal-postgres is set to WipeOut or, If Spec.WipeOut of dormant database is set to true, then the data in cloud backend will be deleted.

The data will be intact in other scenarios.

Cleaning up

To cleanup the Kubernetes resources created by this tutorial, run:

kubectl patch -n demo pg/wal-postgres -p '{"spec":{"terminationPolicy":"WipeOut"}}' --type="merge"
kubectl delete -n demo pg/wal-postgres

kubectl delete -n demo secret/s3-secret
kubectl delete ns demo

Next Steps