Skip to content

Latest commit

 

History

History
427 lines (342 loc) · 15 KB

File metadata and controls

427 lines (342 loc) · 15 KB

S5 Node Configuration Reference

This document describes the toml configuration file format for S5 nodes. The same S5NodeConfig schema is read by both front-ends:

  • vup — default ~/.config/s5/config.toml, generated by vup onboard.
  • s5 — default ~/.config/s5/local.toml (other nodes: ~/.config/s5/nodes/<name>.toml), scaffolded by s5 config init.

Most fields cross-reference each other by name — a vault names a [store.*] and a [key.*], a task names a [vault.*] and a [source.*], and so on. Dangling references are caught at load time and reported, so a typo fails loudly instead of being silently ignored.

Structure

Top-level fields

# Node-wide default store: any vault without an explicit `data_store` uses
# this. When unset AND exactly one [store.*] entry exists, that entry is the
# implied default (architecture decision D1).
default_store = "local"

There is no top-level name or registry_path field — the registry lives in [registry.<name>] (see below), and nodes are identified by their cryptographic identity, not a human name.

[identity]

Configures the node's cryptographic identity.

[identity]
# Path to the node's secret-key file. Relative paths resolve next to the
# config file; absolute paths are also accepted. Age-encrypted at rest when
# `encrypted_with` names a key with an identity file.
secret_key_file = "node.key.age"

# OR provide the secret key directly as a hex string (not recommended).
# secret_key = "..."

# Name of a [key.*] entry the identity file is age-encrypted with.
encrypted_with = "main"

# Path to the warm master signing key seed (D17 cold/warm split). Generated
# on first boot if missing; age-encrypted at rest when [key.main] is set.
master_key_file = "master.key"

# Path to the cold-pointer anchor entry binding this DID to its warm key.
# Written by `vup onboard`/`vup recover`; defaults to a sibling of the warm
# key named `identity_anchor.entry` when unset.
anchor_entry_file = "identity_anchor.entry"

# Name of the [store.*] the daemon publishes the bootstrap vaults into at
# startup — the durable, recoverable store `vup recover` reads them back from.
# Set to a remote store (indexd/s3); a local-only store cannot anchor paper
# recovery. Unset = bootstrap vaults are not published (recovery unavailable).
bootstrap_store = "sia"

# Optional path to the per-device keyset file (device_keyset.cbor.age).
# Defaults to a sibling of `secret_key_file` named device_keyset.cbor.age.
# keyset_file = "device_keyset.cbor.age"

[key.<name>]

An age key used for vault encryption and for the published-snapshot recipient set. Vaults and tasks reference these by name.

[key.main]
# Age public key (recipient string).
public_key = "age1abc..."
# Optional path to an age identity file for decryption. A key without an
# identity file is encrypt-only (e.g. a yubikey/paper recipient this device
# cannot itself decrypt with).
identity_file = "main.age"

[key.recovery]
# Encrypt-only recovery recipient derived from the paper mnemonic.
public_key = "age1recovery..."

[store.<name>]

Defines a blob storage backend. You can define multiple stores. Each entry has a type plus a few wrapper-level toggles.

Backend types: local, s3, indexd (Sia via an indexd service), sia_renterd (direct renterd), memory, fjall, local_links.

Wrapper-level toggles (all backends):

# Write Bao outboard data alongside blobs ≥ 64 KiB — only useful for verified
# streaming to untrusted peers. Default: false.
outboard = false
# Optional in-RAM read-through cache above this store, in bytes. Default: off.
read_cache_bytes = 268435456
# Friend-hosted-storage push ACL: [friend.*] nicknames allowed to push blobs
# into this store when you host it for them. Currently UNENFORCED and not
# settable via the CLI; kept for config-format stability until friend-hosted
# blob serving consumes it.
# allow = ["alice"]

Local filesystem

[store.local]
type = "local"
base_path = "/home/user/.local/share/s5/blobs"

S3-compatible

[store.s3]
type = "s3"
endpoint = "https://s3.amazonaws.com"   # required S3-compatible endpoint
region = "us-east-1"                    # optional; some providers ignore it
bucket_name = "my-bucket"
access_key = "..."
secret_key = "..."

Sia via indexd

The backend vup store add sia <name> provisions. Every field needed to open the store lives inline (like the S3 backend's credentials).

[store.sia]
type = "indexd"
indexer_url = "https://sia.storage"
# The registered 32-byte AppKey, hex-encoded.
app_key = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
# Local index + capability cache (rebuildable; device-local, never synced).
cache_path = "/home/user/.local/share/s5/indexd-cache"
# Optional account label on this indexer ("" = the primary account).
account = ""
# Optional upload concurrency knob (device RAM/throughput). Default 8.
# max_inflight = 8

# Optional: raise the whole-pack upload deadline (seconds) if a very slow uplink
# can't finish a big backup within the default (1200 = 20 min). This is the only
# indexd knob; the enumeration and download timeouts, their retries, and the
# multi-device sync interval use defaults that carry every use case.
# upload_timeout_secs = 1200

Syncing store configs across devices

Store configs are not synced by a name convention — they are synced through the master-anchored config vault (see special-vaults.md). A store added with vup store add on one device can be mirrored into the config vault so a fresh device (or a paper vup recover) rematerialises the same [store.<name>] block — inline credentials and all — with no re-auth. The config vault holds each store entry in exactly the [store.<name>] TOML shape; vup store ls / vup store info show the effective set.

[registry.<name>]

Named registry backends. At least one entry named default is expected.

Types: local/redb (local Redb DB), store (a StoreRegistry over a named [store.*]), store_local (a StoreRegistry over a local directory), multi (fan-out to N backends), memory.

# Local, single-machine (what a local-only onboard generates):
[registry.default]
type = "redb"
path = "/home/user/.config/s5/registry"
# Durable multi-registry (what a Sia onboard generates): fan writes out to a
# fast local redb AND a StoreRegistry over the remote store, so every HEAD is
# recoverable by `vup recover` from paper alone.
[registry.default]
type = "multi"
write_policy = "all"          # "all" (default), "any", or "quorum:N"

[[registry.default.backends]]
type = "redb"
path = "/home/user/.config/s5/registry"

[[registry.default.backends]]
type = "store"
store = "sia"
prefix = "registry"

[source.<name>]

Declares a local directory that s5 is allowed to read. This is a security boundary — nothing outside a declared source can be ingested, regardless of what remote orchestration requests. Tasks and vaults reference sources by name.

[source.documents]
# Filesystem paths to include (one or many).
paths = ["/home/user/Documents"]

# Glob patterns to exclude.
exclude = ["*.tmp", "*.log", "**/.cache", "**/node_modules"]

# Include cache dirs (CACHEDIR.TAG, node_modules, target/, ...). Default: false.
include_caches = false

# Skip hidden files/dirs (dotfiles). Default: false — include them.
skip_hidden = false

# Respect .gitignore / .ignore rules. Default: false.
respect_ignore_files = false

# Stay on one filesystem — do not cross mount boundaries. Default: false.
one_file_system = false

# Follow symlinks (import target content instead of storing the link).
# Default: false.
follow_symlinks = false

# Detect deletions: tombstone snapshot entries whose source file no longer
# exists, so the snapshot mirrors the source instead of growing forever.
# Default: false (append-only archival).
detect_deletions = false

# Max files processed concurrently during ingest. Unset = default (8).
# max_concurrent_ops = 8

[vault.<name>]

An FS5 vault — the canonical local metadata state, created by vup vault create <name>. Each vault has its own FS5 root at root_path. All vaults are encrypted, so key is required. Unknown fields are rejected.

[vault.docs]
# Absolute path to the FS5 root directory (contains root.fs5.cbor). Metadata
# is always local.
root_path = "/home/user/.local/share/s5/vaults/docs"

# [key.*] name used to encrypt this vault's local state (required; this node
# must hold the identity file for it).
key = "main"

# Primary store for content blobs — the upload target for snapshots and the
# first read source. Unset = the node-level `default_store` (D1).
data_store = "local"

# Primary store for published meta blobs (encrypted Transparent Node, exports).
# Unset = resolves to `data_store`.
# meta_store = "local-ssd"

# [key.*] names forming the publish recipient set — every published snapshot is
# age-encrypted to all of them. Empty = local-only (cannot publish).
recipients = ["main", "recovery"]

# [source.*] names that feed this vault on backup. Empty = paths passed
# explicitly per run.
sources = ["documents"]

# Optional FS5 pipeline preset.
# preset = "e2ee_prolly_chunked_zstd_dict_default_chacha20"

# --- Sharing / membership (D11) ---
# Identities authorised to participate. Each entry is "self" or a [friend.*]
# nickname. `members` is the read set; `writers` (a subset) additionally get
# registry-write authority. Managed by `vup grant`/`vup revoke`.
# members = ["self", "alice"]
# writers = ["alice"]

# --- Automation shortcuts (see also [task.*] below) ---
# Use filesystem events to snapshot immediately on change. Default: false.
# watch = true
# Scheduled snapshot every N seconds (skipped when `watch = true`).
# snap_interval_secs = 3600

# --- Publish-and-distribute (public publisher / read-only consumers) ---
# Store tree nodes in plaintext (content-store interop). Default: false.
# plaintext_tree = true
# Publish the Transparent Node in plaintext instead of age-encrypting it.
# plaintext_published_tn = true
# On a read-only consumer, mirror the publisher's 32-byte hex vault_id.
# vault_id = "..."

# --- Published-history bound + cold-store GC (publisher-only) ---
# Keep only the last N history entries in the published TN chain.
# tn_history_keep = 30
# gc_enabled = true            # spawn the cold-store GC task
# gc_store = "s3-cold"         # the [store.*] the GC prunes (the cold tier)
# gc_interval_secs = 86400     # default 24 h
# gc_min_age_secs = 604800     # grace before a blob is deletable; default 7 d
# gc_dry_run = true            # report-only; flip false after one clean cycle

Per-key pipeline routing (first match wins) can be declared with repeated [[vault.<name>.pipelines]] tables:

[[vault.docs.pipelines]]
glob = "segments/**/*.seg"
pipeline = { compression = { type = "zstd", level = 9 } }
chunking = { type = "fixed", chunk_size = 8388608 }

[[vault.docs.pipelines]]
glob = "{ledger,rindex}/**"
pipeline = { compression = { type = "uncompressed" } }
chunking = { type = "none" }

[task.<name>]

A named task — the unit of work. Each task runs in its own ephemeral FS5 working tree, produces a snapshot, and merges it into the target vault. Tasks are run on demand (vup backup …, vup restore …, vup copy …) or fired by the daemon's automation engine (see the trigger fields below).

type selects the task kind; the remaining fields depend on it. All references (vault, source, blob_store, keys) name declared config entries.

# ingest — scan a source, import blobs, merge a snapshot into the vault.
[task.ingest-docs]
type = "ingest"
vault = "docs"
source = "documents"
blob_store = "local"
# target_path = "documents"        # optional path prefix in the FS5 tree

# publish — diff, replicate meta blobs, publish the snapshot to the registry.
[task.publish-docs]
type = "publish"
vault = "docs"
keys = ["main", "recovery"]        # recipients the published state is encrypted to

# backup — ingest + publish in one task.
[task.backup-docs]
type = "backup"
vault = "docs"
source = "documents"
blob_store = "local"
keys = ["main", "recovery"]
then = ["publish-docs"]            # tasks to fire on successful completion

# restore — restore a snapshot into a local directory.
[task.restore-docs]
type = "restore"
vault = "docs"
target_path = "/home/user/restore"
# blob_store = "local"             # optional read-store override
# snapshot = "3"                   # optional: revision number, ISO-8601 ts, or hash prefix
# subtree = "reports"              # optional: restore only this subtree

# copy — copy a vault (or subtree) into another vault (the D21 sharing primitive).
[task.copy-docs]
type = "copy"
src_vault = "docs"
dst_vault = "docs-share"
blob_store = "local"
keys = ["main", "alice"]
# src_path = "reports"             # optional source subtree
# deep = false                     # true = re-encrypt under the destination's keys

Automation triggers

Every task carries three automation fields (all default to a plain manual task, so existing tasks are unaffected). When a task is not manual, the daemon's automation engine reconciles it from [task.*] and keeps a live loop running. Managed interactively by vup automate add|list|show|pause|resume|rm.

[task.nightly-backup]
type = "backup"
vault = "docs"
source = "documents"
blob_store = "local"
keys = ["main", "recovery"]

# trigger: "manual" (default), "watch", or "every".
#   manual — runs only on explicit request.
#   watch  — the daemon watches the source paths and snaps on change
#            (requires a backup task).
#   every  — the daemon re-runs the task on a cadence (requires interval_secs).
trigger = "every"

# Cadence for trigger = "every", in seconds. Required for "every"; ignored
# otherwise.
interval_secs = 86400

# A paused automation stays configured but is not spawned. `automate
# pause`/`resume` flip this. No effect on manual tasks.
paused = false

[friend.<name>]

A paired peer identified by its did:s5: reference. Vault members/writers lists reference friends by this local nickname; the (currently unenforced) store push-ACL (allow) does too. Managed by vup friend pair / vup friend forget.

[friend.alice]
# did:s5:b<multibase(0xed || pubkey)> — the friend's DID.
id = "did:s5:b..."

# Optional 64-char hex iroh transport pubkey. Seeds a direct dial for
# bootstrap-from-cold-cache; without it the daemon waits for the identity
# bundle to arrive out of band.
# iroh_pubkey_hex = "..."

Retired / removed tables

  • [sync.<name>] (shared-secret folder sync) — removed. Continuous backup is now a watch/every automation on a [task.*] (or vault.watch / vault.snap_interval_secs); multi-party sharing is the identity/ACL vault model (vup friend pair + vup grant).
  • [peer.<name>] (endpoint-id + [peer.*.blobs] ACL) — removed, replaced by [friend.<name>] (above). ACL is now vault membership + per-store allow.
  • [fuse.<name>] (auto-mount table) — removed. Mount a vault at runtime with vup mount <vault>: <dir> (add --rw for a writable overlay mount).