Home Explore Blog CI



kubernetes

5th chunk of `content/en/docs/tasks/administer-cluster/configure-upgrade-etcd.md`
27940df84a58b834a32d9d174092e094587b6cbcaea64e5e0000000100000fc8
ETCDCTL_API=3 etcdctl --endpoints $ENDPOINT snapshot save snapshot.db
```

Verify the snapshot:

{{< tabs name="etcd_verify_snapshot" >}}
{{% tab name="Use etcdutl" %}}
   The below example depicts the usage of the `etcdutl` tool for verifying a snapshot:

   ```shell
   etcdutl --write-out=table snapshot status snapshot.db 
   ```

   This should generate an output resembling the example provided below:

   ```console
   +----------+----------+------------+------------+
   |   HASH   | REVISION | TOTAL KEYS | TOTAL SIZE |
   +----------+----------+------------+------------+
   | fe01cf57 |       10 |          7 | 2.1 MB     |
   +----------+----------+------------+------------+
   ```

{{% /tab %}}
{{% tab name="Use etcdctl (Deprecated)" %}}

   {{< note >}}
   The usage of `etcdctl snapshot status` has been **deprecated** since etcd v3.5.x and is slated for removal from etcd v3.6.
   It is recommended to utilize [`etcdutl`](https://github.com/etcd-io/etcd/blob/main/etcdutl/README.md) instead.
   {{< /note >}}

   The below example depicts the usage of the `etcdctl` tool for verifying a snapshot:

   ```shell
   export ETCDCTL_API=3
   etcdctl --write-out=table snapshot status snapshot.db
   ```

   This should generate an output resembling the example provided below:

   ```console
   Deprecated: Use `etcdutl snapshot status` instead.

   +----------+----------+------------+------------+
   |   HASH   | REVISION | TOTAL KEYS | TOTAL SIZE |
   +----------+----------+------------+------------+
   | fe01cf57 |       10 |          7 | 2.1 MB     |
   +----------+----------+------------+------------+
   ```


{{% /tab %}}
{{< /tabs >}}

### Volume snapshot

If etcd is running on a storage volume that supports backup, such as Amazon
Elastic Block Store, back up etcd data by creating a snapshot of the storage
volume.

### Snapshot using etcdctl options

We can also create the snapshot using various options given by etcdctl. For example: 

```shell
ETCDCTL_API=3 etcdctl -h 
``` 

will list various options available from etcdctl. For example, you can create a snapshot by specifying
the endpoint, certificates and key as shown below:

```shell
ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 \
  --cacert=<trusted-ca-file> --cert=<cert-file> --key=<key-file> \
  snapshot save <backup-file-location>
```
where `trusted-ca-file`, `cert-file` and `key-file` can be obtained from the description of the etcd Pod.

## Scaling out etcd clusters

Scaling out etcd clusters increases availability by trading off performance.
Scaling does not increase cluster performance nor capability. A general rule
is not to scale out or in etcd clusters. Do not configure any auto scaling
groups for etcd clusters. It is strongly recommended to always run a static
five-member etcd cluster for production Kubernetes clusters at any officially
supported scale.

A reasonable scaling is to upgrade a three-member cluster to a five-member
one, when more reliability is desired. See
[etcd reconfiguration documentation](https://etcd.io/docs/current/op-guide/runtime-configuration/#remove-a-member)
for information on how to add members into an existing cluster.

## Restoring an etcd cluster

{{< caution >}}
If any API servers are running in your cluster, you should not attempt to
restore instances of etcd. Instead, follow these steps to restore etcd:

- stop *all* API server instances
- restore state in all etcd instances
- restart all API server instances

The Kubernetes project also recommends restarting Kubernetes components (`kube-scheduler`,
`kube-controller-manager`, `kubelet`) to ensure that they don't rely on some
stale data. In practice the restore takes a bit of time.  During the
restoration, critical components will lose leader lock and restart themselves.
{{< /caution >}}

etcd supports restoring from snapshots that are taken from an etcd process of
the [major.minor](https://semver.org/) version. Restoring a version from a
different patch version of etcd is also supported. A restore operation is

Title: Verifying Snapshots, Volume Snapshots, and Scaling/Restoring etcd Clusters
Summary
This section provides examples of how to verify etcd snapshots using both `etcdutl` and the deprecated `etcdctl` tool. It then discusses using volume snapshots for etcd backup, particularly on storage volumes like Amazon EBS. The use of etcdctl options to create snapshots by specifying endpoints, certificates, and keys is also shown. Finally, it advises against scaling etcd clusters and recommends running a static five-member cluster for production Kubernetes. It also outlines the process for restoring an etcd cluster from a snapshot, emphasizing the need to stop all API servers before restoring and restart them afterward, along with other Kubernetes components.