Home Explore Blog CI



kubernetes

7th chunk of `content/en/docs/tasks/administer-cluster/configure-upgrade-etcd.md`
4f1e25d53005c85ac497f41d597ca82ae5e7723afeb7f4140000000100000d84
   use the `--data-dir` option to specify to which folder the cluster should be restored:

   ```shell
   etcdutl --data-dir <data-dir-location> snapshot restore snapshot.db
   ```
   where `<data-dir-location>` is a directory that will be created during the restore process.

{{% /tab %}}
{{% tab name="Use etcdctl (Deprecated)" %}}

   {{< note >}}
   The usage of `etcdctl` for restoring has been **deprecated** since etcd v3.5.x and is slated for removal from etcd v3.6.
   It is recommended to utilize [`etcdutl`](https://github.com/etcd-io/etcd/blob/main/etcdutl/README.md) instead.
   {{< /note >}}

   The below example depicts the usage of the `etcdctl` tool for the restore operation:

   ```shell
   export ETCDCTL_API=3
   etcdctl --data-dir <data-dir-location> snapshot restore snapshot.db
   ```

   If `<data-dir-location>` is the same folder as before, delete it and stop the etcd process before restoring the cluster. 
   Otherwise, change etcd configuration and restart the etcd process after restoration to have it use the new data directory:
   first change  `/etc/kubernetes/manifests/etcd.yaml`'s `volumes.hostPath.path` for `name: etcd-data`  to `<data-dir-location>`,
   then execute `kubectl -n kube-system delete pod <name-of-etcd-pod>` or `systemctl restart kubelet.service` (or both).

{{% /tab %}}
{{< /tabs >}}

For more information and examples on restoring a cluster from a snapshot file, see
[etcd disaster recovery documentation](https://etcd.io/docs/current/op-guide/recovery/#restoring-a-cluster).

If the access URLs of the restored cluster are changed from the previous
cluster, the Kubernetes API server must be reconfigured accordingly. In this
case, restart Kubernetes API servers with the flag
`--etcd-servers=$NEW_ETCD_CLUSTER` instead of the flag
`--etcd-servers=$OLD_ETCD_CLUSTER`. Replace `$NEW_ETCD_CLUSTER` and
`$OLD_ETCD_CLUSTER` with the respective IP addresses. If a load balancer is
used in front of an etcd cluster, you might need to update the load balancer
instead.

If the majority of etcd members have permanently failed, the etcd cluster is
considered failed. In this scenario, Kubernetes cannot make any changes to its
current state. Although the scheduled pods might continue to run, no new pods
can be scheduled. In such cases, recover the etcd cluster and potentially
reconfigure Kubernetes API servers to fix the issue.


## Upgrading etcd clusters

{{< caution >}}
Before you start an upgrade, back up your etcd cluster first.
{{< /caution >}}

For details on etcd upgrade, refer to the [etcd upgrades](https://etcd.io/docs/latest/upgrades/) documentation.

## Maintaining etcd clusters

For more details on etcd maintenance, please refer to the [etcd maintenance](https://etcd.io/docs/latest/op-guide/maintenance/) documentation.

### Cluster defragmentation

{{% thirdparty-content single="true" %}}

Defragmentation is an expensive operation, so it should be executed as infrequently
as possible. On the other hand, it's also necessary to make sure any etcd member
will not exceed the storage quota. The Kubernetes project recommends that when
you perform defragmentation, you use a tool such as [etcd-defrag](https://github.com/ahrtr/etcd-defrag).

You can also run the defragmentation tool as a Kubernetes CronJob, to make sure that
defragmentation happens regularly. See [`etcd-defrag-cronjob.yaml`](https://github.com/ahrtr/etcd-defrag/blob/main/doc/etcd-defrag-cronjob.yaml)
for details.

Title: Restoring, Upgrading, and Maintaining etcd Clusters
Summary
This section provides details on restoring etcd clusters from snapshots using `etcdutl` and the deprecated `etcdctl`, including command examples and instructions for updating the Kubernetes API server if the restored cluster's access URLs change. It emphasizes the importance of reconfiguring the API server with the new etcd cluster address and updating the `/etc/kubernetes/manifests/etcd.yaml` file if the data directory changes. Additionally, it covers upgrading etcd clusters and maintaining them, particularly focusing on cluster defragmentation using tools like `etcd-defrag` to avoid exceeding storage quotas, recommending the use of a Kubernetes CronJob for regular defragmentation.