Skip to content

feat: add Docker backup and restore workflow - #233

Open
Optic00 wants to merge 2 commits into
Windshiftapp:mainfrom
Optic00:codex/backup-restore
Open

feat: add Docker backup and restore workflow#233
Optic00 wants to merge 2 commits into
Windshiftapp:mainfrom
Optic00:codex/backup-restore

Conversation

@Optic00

@Optic00 Optic00 commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add a conservative host-side backup and restore script for the stock Docker Compose SQLite and PostgreSQL deployments
  • archive the complete data volume, create a logical PostgreSQL dump when applicable, and record a compatibility fingerprint for the effective SSO_SECRET/SESSION_SECRET
  • verify and privately stage restore inputs, serialize operations with a Compose-project lock, create rollback artifacts before mutation, and delete rollback artifacts only after readiness succeeds for a previously running service
  • document the supported deployment shape, secret handling, failure behavior, and manual emergency recovery
  • add a Docker-free contract test for the guarded success and failure paths

Supported scope

This intentionally rejects replicas, external PostgreSQL connection strings, SSH-enabled persistence, command-line path overrides, and configured persistent paths outside the data volume. Operators must use the same Windshift image/schema generation and PostgreSQL major version for restore unless a separate migration plan has been verified.

Validation

  • bash -n deploy/backup.sh
  • bash -n deploy/test-backup.sh
  • env -u TMPDIR bash deploy/test-backup.sh
  • TMPDIR=/tmp bash deploy/test-backup.sh
  • env PATH=/usr/bin:/bin:/usr/sbin:/sbin /bin/bash deploy/test-backup.sh
  • git diff --check origin/main...HEAD
  • Linux Docker E2E against an immutable digest of the current main image:
    • SQLite: real API data and a /data marker were backed up, mutated, and restored; the baseline returned and post-backup data disappeared
    • PostgreSQL 17: the same API and /data rollback passed, with a nonempty logical database.dump
    • restore without --force and restore from a tampered backup were rejected without changing the live test data
    • health checks, archive checksums, rollback and staging cleanup, Compose cleanup, and absence of stale operation locks passed

Refs #195

@stefan-ernst

Copy link
Copy Markdown
Contributor

Hey @Optic00 - nice! I will take the md file and put it on the official docs (which I still have to provide the official repo for)

One minor thing:

docker compose exec -T "$POSTGRES_SERVICE" sh -ec 'pg_restore --clean --if-exists --no-owner --no-privileges --single-transaction --exit-on-error -U "$POSTGRES_USER" -d "$POSTGRES_DB" "$1"' sh "$remote_dump"

I dont think $1 resolves to anything here?

@Optic00

Optic00 commented Aug 29, 2026

Copy link
Copy Markdown
Contributor Author

Good catch. The previous form did resolve $1: with sh -ec '...' sh "$remote_dump", the first sh becomes $0 and the dump path becomes $1. That is valid but unnecessarily obscure, so I replaced it in 70454c82 with an explicit REMOTE_DUMP variable passed through docker compose exec -e. Thanks for flagging it.

@stefan-ernst

Copy link
Copy Markdown
Contributor

All good, do you want this to be merged? The PR is still in Draft state

@Optic00

Optic00 commented Aug 29, 2026

Copy link
Copy Markdown
Contributor Author

Yes, please merge it. I kept it in draft until the real Linux Docker validation was complete.

That E2E run has now passed for both SQLite and PostgreSQL 17 using an immutable digest of the current main image:

  • real API data and a /data marker were backed up, mutated, and restored successfully
  • the PostgreSQL backup included a nonempty logical dump
  • restore without --force and restore from a tampered backup were rejected without changing live test data
  • health checks, checksum validation, cleanup, and absence of stale locks all passed

I will mark the PR ready now.

One related question: CONTRIBUTING.md mentions the adjacent private core-tests repository. If you are open to external contributions there, I would be happy to help expand the backup and restore coverage. Would read access and a contribution workflow be possible, or would you prefer that we keep additional black-box tests in the public core repository?

@Optic00
Optic00 marked this pull request as ready for review August 29, 2026 13:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants