Skip to content

Commit

Permalink
Add druid kubestash backup docs
Browse files Browse the repository at this point in the history
Signed-off-by: Tapajit Chandra Paul <[email protected]>
  • Loading branch information
tapojit047 committed Sep 17, 2024
1 parent 2fe7040 commit ccd80f0
Show file tree
Hide file tree
Showing 5 changed files with 131 additions and 0 deletions.
10 changes: 10 additions & 0 deletions docs/guides/druid/backup/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: Backup & Restore Druid | KubeStash
menu:
docs_{{ .version }}:
identifier: guides-druid-backup-stashv2
name: KubeStash (aka Stash 2.0)
parent: guides-druid-backup
weight: 50
menu_name: docs_{{ .version }}
---
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
21 changes: 21 additions & 0 deletions docs/guides/druid/backup/overview/images/kubedb_plus_kubestash.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
98 changes: 98 additions & 0 deletions docs/guides/druid/backup/overview/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
---
title: Backup & Restore Druid Overview
menu:
docs_{{ .version }}:
identifier: guides-druid-backup-kubestash-overview-stashv2
name: Overview
parent: guides-druid-backup-stashv2
weight: 10
menu_name: docs_{{ .version }}
section_menu_id: guides
---

> New to KubeDB? Please start [here](/docs/README.md).
{{< notice type="warning" message="Please install [KubeStash](https://kubestash.com/docs/latest/setup/install/kubestash/) to try this feature. Database backup with KubeStash is already included in the KubeDB license. So, you don't need a separate license for KubeStash." >}}

# Druid Backup & Restore Overview

KubeDB also uses [KubeStash](https://kubestash.com) to backup and restore databases. KubeStash by AppsCode is a cloud native data backup and recovery solution for Kubernetes workloads and databases. KubeStash utilizes [restic](https://github.com/restic/restic) to securely backup stateful applications to any cloud or on-prem storage backends (for example, S3, GCS, Azure Blob storage, Minio, NetApp, Dell EMC etc.).

<figure align="center">
  <img alt="KubeDB + KubeStash" src="/docs/guides/druid/backup/kubestash/overview/images/kubedb_plus_kubestash.svg">
<figcaption align="center">Fig: Backup KubeDB Databases Using KubeStash</figcaption>
</figure>

## How Backup Works

The following diagram shows how KubeStash takes backup of a `Druid` database. Open the image in a new tab to see the enlarged version.

<figure align="center">
  <img alt="Druid Backup Overview" src="/docs/guides/druid/backup/kubestash/overview/images/backup_overview.svg">
<figcaption align="center">Fig: Druid Backup Overview</figcaption>
</figure>

The backup process consists of the following steps:

1. At first, a user creates a `Secret`. This secret holds the credentials to access the backend where the backed up data will be stored.

2. Then, she creates a `BackupStorage` custom resource that specifies the backend information, along with the `Secret` containing the credentials needed to access the backend.

3. KubeStash operator watches for `BackupStorage` custom resources. When it finds a `BackupStorage` object, it initializes the `BackupStorage` by uploading the `metadata.yaml` file to the specified backend.

4. Next, she creates a `BackupConfiguration` custom resource that specifies the target database, addon information (including backup tasks), backup schedules, storage backends for storing the backup data, and other additional settings.

5. KubeStash operator watches for `BackupConfiguration` objects.

6. Once the KubeStash operator finds a `BackupConfiguration` object, it creates `Repository` with the information specified in the `BackupConfiguration`.

7. KubeStash operator watches for `Repository` custom resources. When it finds the `Repository` object, it Initializes `Repository` by uploading `repository.yaml` file into the `spec.sessions[*].repositories[*].directory` path specified in `BackupConfiguration`.

8. Then, it creates a `CronJob` for each session with the schedule specified in `BackupConfiguration` to trigger backup periodically.

9. KubeStash operator triggers an instant backup as soon as the `BackupConfiguration` is ready. Backups are otherwise triggered by the `CronJob` based on the specified schedule.

10. KubeStash operator watches for `BackupSession` custom resources.

11. When it finds a `BackupSession` object, it creates a `Snapshot` custom resource for each `Repository` specified in the `BackupConfiguration`.

12. Then it resolves the respective `Addon` and `Function` and prepares backup `Job` definition.

13. Then, it creates the `Job` to backup the targeted `Druid` database.

14. The backup `Job` reads necessary information (e.g. auth secret, port) to connect with the database from the `AppBinding` CR. It also reads backend information and access credentials from `BackupStorage` CR, Storage `Secret` and `Repository` path respectively.

15. Then, the `Job` dumps the targeted `Druid` database and uploads the output to the backend. KubeStash pipes the output of dump command to uploading process. Hence, backup `Job` does not require a large volume to hold the entire dump output.

16. After the backup process is completed, the backup `Job` updates the `status.components[dump]` field of the `Snapshot` resources with backup information of the target `Druid` database.

## How Restore Process Works

The following diagram shows how KubeStash restores backed up data into a `Druid` database. Open the image in a new tab to see the enlarged version.

<figure align="center">
  <img alt="Database Restore Overview" src="/docs/guides/druid/backup/kubestash/overview/images/restore_overview.svg">
<figcaption align="center">Fig: Druid Restore Process Overview</figcaption>
</figure>

The restore process consists of the following steps:

1. At first, a user creates a `Druid` database where the data will be restored or the user can use the same `Druid` database.

2. Then, she creates a `RestoreSession` custom resource that specifies the target database where the backed-up data will be restored, addon information (including restore tasks), the target snapshot to be restored, the `Repository` containing that snapshot, and other additional settings.

3. KubeStash operator watches for `RestoreSession` custom resources.

4. When it finds a `RestoreSession` custom resource, it resolves the respective `Addon` and `Function` and prepares a restore `Job` definition.

5. Then, it creates the `Job` to restore the target.

6. The `Job` reads necessary information to connect with the database from respective `AppBinding` CR. It also reads backend information and access credentials from `Repository` CR and storage `Secret` respectively.

7. Then, the `Job` downloads the backed up data from the backend and injects into the desired database. KubeStash pipes the downloaded data to the respective database tool to inject into the database. Hence, restore `Job` does not require a large volume to download entire backup data inside it.

8. Finally, when the restore process is completed, the `Job` updates the `status.components[*]` field of the `RestoreSession` with restore information of the target database.

## Next Steps

- Backup a `Druid` database using KubeStash by following the guides from [here](/docs/guides/druid/backup/kubestash/logical/index.md).

0 comments on commit ccd80f0

Please sign in to comment.