Skip to content

Replace MinIO with S3-compatible storage (Garage)#90

Merged
usnavy13 merged 5 commits into
devfrom
feat/migrate-minio-to-garage-s3
May 7, 2026
Merged

Replace MinIO with S3-compatible storage (Garage)#90
usnavy13 merged 5 commits into
devfrom
feat/migrate-minio-to-garage-s3

Conversation

@usnavy13
Copy link
Copy Markdown
Owner

@usnavy13 usnavy13 commented May 7, 2026

Summary

  • Replace MinIO with Garage S3-compatible storage — swaps the MinIO client/config for a standard boto3 S3 client, making the storage layer provider-agnostic (currently backed by Garage)
  • Add original filename support — preserves pre-sanitization filenames in file metadata and aligns Unicode sanitization with LibreChat's approach (emoji, NFC, two-pass)
  • Harden tmpfs security — adds noexec,nosuid,nodev to /tmp tmpfs mounts and relocates empty_proc to /var/lib/code-interpreter for better sandbox isolation

Changes

  • New S3Config (replaces MinIOConfig) with standard S3 env vars (S3_ENDPOINT, S3_ACCESS_KEY, etc.)
  • Refactored FileService and StateArchivalService to use boto3 S3 client
  • Updated health checks to use S3 status commands
  • Added garage.toml for local dev Garage configuration
  • Updated Docker Compose files (dev + prod) for Garage service
  • Added original_filename field to FileInfo model and plumbed through upload/list flows
  • Aligned sanitize_filename with LibreChat's Unicode handling (NFC normalize, emoji preservation)
  • Updated tests to match new S3 and output handling behavior

Test plan

  • Unit tests pass (pytest tests/unit/)
  • Integration tests pass with Garage backend (pytest tests/integration/)
  • Verify file upload/download works end-to-end with Garage S3
  • Verify state archival/restore works with new S3 client
  • Confirm health check reports Garage status correctly
  • Test Unicode/emoji filenames are preserved correctly

🤖 Generated with Claude Code

usnavy13 added 5 commits May 6, 2026 20:20
- Updated configuration and environment variables to transition from MinIO to S3 storage, including changes to .env.example and Docker Compose files.
- Introduced a new S3Config class for managing S3 settings and removed the MinIO configuration.
- Refactored file management and state archival services to utilize the S3 client, ensuring compatibility with S3 operations.
- Adjusted health checks and service dependencies to reflect the new S3 storage integration.
- Updated documentation and comments throughout the codebase to replace references to MinIO with S3.
- Changed S3 access and secret keys in .env.example and test configuration to new values.
- Updated Docker Compose files to reflect the new S3 access keys and added default bucket environment variable.
- Modified health check command in Docker Compose to use the new status command for better service monitoring.
- Added RPC settings in garage.toml for improved service configuration.
- Enhanced functional tests to verify that edits to mounted files produce new outputs with unique file_ids instead of in-place overwrites.
- Updated test descriptions for clarity on expected behavior regarding modified files.
- Introduced a helper function to locate modified files based on the original file_id, ensuring accurate assertions in test cases.
…etup

- Added a temporary filesystem configuration for /tmp with size and mode settings in both Docker Compose files.
- Changed the directory for empty_proc from /tmp to /var/lib/code-interpreter in the Dockerfile and related service files.
- Updated the sandbox execution commands to reflect the new empty_proc path and incorporated dynamic tmpfs size settings.
- Updated tmpfs mount options for /tmp in Docker Compose files to include noexec, nosuid, and nodev for improved security.
- Refactored sandbox execution commands to apply the new tmpfs settings consistently across service files.
- Introduced dynamic handling of skill dependencies with updated mount options to enhance security and isolation.
@usnavy13 usnavy13 merged commit 0fc9cc3 into dev May 7, 2026
9 checks passed
@usnavy13 usnavy13 deleted the feat/migrate-minio-to-garage-s3 branch May 7, 2026 02:20
usnavy13 added a commit that referenced this pull request May 7, 2026
…tale files

The repo had unnecessary docker-compose.prod.yml and docker-compose.local-test.yml
files. docker-compose.yml is now the single production-ready base (pulls published
GHCR image by default), and docker-compose.override.example.yml handles local dev
overrides. All MinIO references across 10+ docs updated to S3/Garage to match the
migration completed in #90. Removed stale Reference/ directory and placeholder
AGENTS.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant