Skip to content

Commit 37dd347

Browse files
kriszypclaudedawsontothcodex
authored
docs(security): secrets store, secrets operations, and deploy_component registryAuth (#581)
* docs(security): document the secrets store, operations, and deploy_component registryAuth Adds v5 reference documentation for the encrypted secrets store (#1550/#1554/ #1582) and the deploy_component private-registry auth path (#1717), none of which was previously documented. - reference/security/secrets.md (new): store model, custody (Pro) vs OSS core, the two delivery tiers (process.env global vs scoped grants), the config.yaml `env:` declaration block, the `import { secrets } from 'harper'` accessor, full client-side `enc:v1:` envelope format + Node reference client, and the threat model. Added to the Security sidebar. - reference/operations-api/operations.md: new Secrets section (set_secret, grant_secret, revoke_secret, list_secrets, delete_secret, get_secrets_public_key) plus a registryAuth subsection under deploy_component. - reference/components/javascript-environment.md: add `secrets` to the Harper API globals list. - reference/environment-variables/overview.md: cross-link to the secrets store as the production alternative to a committed .env. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(security): secrets change subscription and live scoped accessor (#584) * style(operations-api): apply prettier table alignment Pre-existing format:check violation on the kris/secrets-docs branch; the repo-wide prettier gate must be clean for CI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(security): document secrets change subscription and live scoped accessor Documents harper#1787 (live secret change detection) on the new secrets reference page: - secrets.subscribe(name) — async iterable yielding the current value then every change; shown as read-now + fire-and-forget background subscribe so module load isn't blocked (the practical hot-swap-on-rotation pattern). - Live scoped-tier accessor: a fresh secrets.NAME read reflects the latest value; a destructure is now called out as a point-in-time copy. - Reworked 'Healing after changes' into a per-tier 'How each tier sees a change' table (scoped = live; global/process.env = reload-only) and corrected the 'frozen' accessor description to the read-only live view, with subscribe as a non-enumerable member. Ships in v5.2.0 alongside the rest of the secrets store, so no differential VersionBadge (the page is uniformly 5.2.0 surface). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(security): dedupe the subscribe example's startup rebuild Address review on #584: secrets.subscribe() replays the current value first, so guard on key change to skip the redundant client rebuild on startup, and note that semantic inline. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(security): guard the subscribe example against unhandled rejections Mirror the studio review fix (HarperFast/studio#1501): the fire-and-forget subscribe loop is copy-paste code, so wrap it in try/catch and console.error the failure so a stream error can't become an unhandled promise rejection. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(security): lead with the live accessor; subscription is the exception Per feedback: reading secrets.NAME (undestructured) returns the live value on every read, so most components just use the secret where they need it — e.g. per request inside a resource — and get the current value with no subscription. Only a destructure takes a snapshot. Reframe subscribe() as the case for long-lived objects built from a secret (client/pool/signer) that must be rebuilt on rotation, and make the loader note's per-loader boundary explicit (vm/compartment read live anywhere; native resolves only during load). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * docs(security): rename registryAuth → credentials; document git-host deploy auth deploy_component's registryAuth field was renamed to a general-purpose `credentials` array (harper#1797) that now carries both npm-registry (`registry`) and git-host (`host`) credential entries; the git-over-HTTPS path (harper#1792) serves the token to git from memory via a credential helper. Update the operations reference and the secrets page accordingly, including the derived secret-name conventions and a rename note. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(security): mention Studio UI for secrets; address review nits - Note that Harper Studio offers a graphical interface for managing secrets (Dawson's suggestion) on both the Secrets page and the operations section. - Clarify that a string env value is stored plaintext in config.yaml (VCS). - Add a list_secrets response example. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Fix operations API table formatting Co-Authored-By: Codex <noreply@openai.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Dawson Toth <dawson@harperdb.io> Co-authored-by: Codex <noreply@openai.com>
1 parent c7e80a7 commit 37dd347

5 files changed

Lines changed: 481 additions & 0 deletions

File tree

‎reference/components/javascript-environment.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -133,6 +133,18 @@ See [Logging API](../logging/api.md) for full reference.
133133

134134
A `Map` of all resources registered on this Harper server, keyed by their URL path. Each entry contains the resource class, path, and routing metadata. Use this to look up or enumerate registered resources programmatically.
135135

136+
### `secrets`
137+
138+
A read-only, name→value map of the scoped secrets granted to the current component (plus any `process.env`-resolved values it declared). Read secrets from the encrypted [secrets store](../security/secrets.md) without them landing in `process.env`:
139+
140+
```javascript
141+
import { secrets } from 'harper';
142+
143+
const { STRIPE_KEY } = secrets;
144+
```
145+
146+
A module-top-level destructure is the recommended idiom — it binds correctly under every component loader. See [Secrets](../security/secrets.md) for full reference.
147+
136148
### `config`
137149

138150
The configuration object for the current application, as provided by the component's configuration (e.g. `config.yaml`). Defaults to an empty object if no configuration is provided.

‎reference/environment-variables/overview.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,10 @@ Harper supports loading environment variables in Harper applications `process.en
1313
If you are looking for information on how to configure your Harper installation using environment variables, see [Configuration](../configuration/overview.md) section for more information.
1414
:::
1515

16+
:::tip
17+
For production credentials, prefer the encrypted, replicated [secrets store](../security/secrets.md) over a committed `.env` file. Secrets are stored as ciphertext, delivered to components via `process.env` or a per-component `secrets` accessor, and never appear in operation logs or replication payloads. The same `enc:v1:` envelope format also lets you encrypt individual `.env` values at rest.
18+
:::
19+
1620
## Configuration
1721

1822
| Option | Type | Required | Description |

‎reference/operations-api/operations.md‎

Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -606,6 +606,55 @@ Additional parameters:
606606

607607
- `urlPath` — override the HTTP URL path the component is mounted at (e.g. `"/api/v2"`)
608608
- `install_allow_scripts` — set to `true` to allow npm pre/post install scripts (disabled by default)
609+
- `credentials` — credentials for installing a component from a private npm registry or private git repository (see below)
610+
611+
#### Deploy credentials (`credentials`)
612+
613+
When a component is installed from a private source, `credentials` supplies the authentication. It is an array of entries; each entry is one of two kinds, identified by its key:
614+
615+
- **npm registry auth** — an entry with a `registry` key, applied to a private npm registry.
616+
- **git host auth** — an entry with a `host` key, applied to a private git repository fetched by reference (e.g. `package: "github:my-org/my-app#semver:v1.2.3"`).
617+
618+
An entry provides its credential exactly one of two ways — a literal `token`, or a `secret` reference:
619+
620+
| Field | Kind | Description |
621+
| ---------- | ---- | ------------------------------------------------------------------------------------------------------------------- |
622+
| `registry` | npm | The registry URL or host the credential applies to. **Required** for an npm entry. |
623+
| `scope` | npm | Optional npm `@scope` (e.g. `"@my-org"`) the entry applies to; omit to set the default registry. |
624+
| `host` | git | The bare git host the credential applies to (e.g. `"github.com"`). **Required** for a git entry. |
625+
| `username` | git | Optional git HTTPS username. Defaults to `x-access-token` (GitHub); GitLab uses `oauth2`, Bitbucket `x-token-auth`. |
626+
| `token` | both | A literal auth token, **or** |
627+
| `secret` | both | The name of an [`hdb_secret`](../security/secrets.md) row to resolve the token from. |
628+
629+
A provided **`token`** is not treated as ephemeral: Harper ingests it into the encrypted [secrets store](../security/secrets.md) and references it everywhere, so package-reference deploys keep working through rollback, reboot, and new peers joining — without re-supplying the token. The token is encrypted at rest, stripped from the operation before replication and from the operations log, and only ever crosses the cluster as ciphertext. A git-host token is additionally served to git **from memory** (via a credential helper) — it is never written to a file or into a URL. Using a **`secret`** reference names an existing store row directly. Ingesting a token requires custody on the deploying node; on OSS core without custody, a literal token falls back to a transient, this-node-only credential (not persisted or replicated).
630+
631+
Ingested tokens are stored under a derived name granted to the component — `deploy.<component>.<registry>` for a registry entry, `deploy.<component>.git.<host>` for a git entry — so re-deploying with a rotated token idempotently updates the same row.
632+
633+
Private npm registry:
634+
635+
```json
636+
{
637+
"operation": "deploy_component",
638+
"project": "my-app",
639+
"package": "npm:@my-org/my-app@1.2.3",
640+
"credentials": [{ "registry": "https://registry.my-org.com", "scope": "@my-org", "token": "npm_..." }]
641+
}
642+
```
643+
644+
Private git repository (token resolved from an existing secret):
645+
646+
```json
647+
{
648+
"operation": "deploy_component",
649+
"project": "my-app",
650+
"package": "github:my-org/my-app#semver:v1.2.3",
651+
"credentials": [{ "host": "github.com", "secret": "deploy.my-app.git.github_com" }]
652+
}
653+
```
654+
655+
:::note
656+
`credentials` replaces the earlier `registryAuth` field (renamed while the feature was in alpha, before it grew to carry git-host credentials). `registryAuth` is now rejected with an error directing you to `credentials`.
657+
:::
609658

610659
The response includes a `deployment_id` that can be used to query the deployment record:
611660

@@ -724,6 +773,117 @@ Adds an SSH key (must be ed25519) for authenticating deployments from private re
724773

725774
---
726775

776+
## Secrets
777+
778+
Operations for managing the encrypted [secrets store](../security/secrets.md) (`system.hdb_secret`). All secret operations are **`super_user` only**. Values are never returned or logged by any of these operations.
779+
780+
Detailed documentation: [Secrets](../security/secrets.md)
781+
782+
:::tip
783+
Prefer a UI? [Harper Studio](../studio/overview.md) provides a graphical interface for creating, granting, and rotating secrets — it drives these operations for you, so you don't have to hand-craft the request bodies below.
784+
:::
785+
786+
| Operation | Description | Role Required |
787+
| ------------------------ | -------------------------------------------------------------- | ------------- |
788+
| `set_secret` | Creates or updates a secret and chooses its delivery tier | super_user |
789+
| `grant_secret` | Adds a component to a scoped secret's grants (idempotent) | super_user |
790+
| `revoke_secret` | Removes a component from a scoped secret's grants (idempotent) | super_user |
791+
| `list_secrets` | Lists secret metadata — never envelopes or values | super_user |
792+
| `delete_secret` | Deletes a secret row | super_user |
793+
| `get_secrets_public_key` | Returns the cluster public key for client-side encryption | super_user |
794+
795+
### `set_secret`
796+
797+
Creates or updates a secret. Supply exactly one of `value` (plaintext, encrypted on ingest — requires custody on this node) or `envelope` (an `enc:v1:` ciphertext produced client-side against `get_secrets_public_key`). The delivery tier is `processEnv: true` **or** `grants` — the two are mutually exclusive. On update, tier and metadata default to the stored row, so a value rotation preserves the tier without re-specifying it.
798+
799+
| Parameter | Type | Description |
800+
| ------------ | -------- | ------------------------------------------------------------------------------- |
801+
| `name` | string | Secret name (word characters, dots, dashes). **Required.** |
802+
| `value` | string | Plaintext value; encrypted immediately, then discarded. Requires custody. |
803+
| `envelope` | string | `enc:v1:` ciphertext (alternative to `value`). |
804+
| `processEnv` | boolean | `true` delivers the secret via `process.env` (global tier). |
805+
| `grants` | string[] | Components allowed to read the secret via the `secrets` accessor (scoped tier). |
806+
| `metadata` | object | Optional free-form label object (not a payload store). |
807+
808+
```json
809+
{
810+
"operation": "set_secret",
811+
"name": "STRIPE_KEY",
812+
"value": "sk_live_...",
813+
"grants": ["payments-service"]
814+
}
815+
```
816+
817+
Response:
818+
819+
```json
820+
{ "name": "STRIPE_KEY", "kid": "<hex fingerprint>", "created": true }
821+
```
822+
823+
### `grant_secret` / `revoke_secret`
824+
825+
Add or remove a component from a scoped secret's grants list. Both are idempotent. A `processEnv` (global) secret cannot be granted — convert it with `set_secret` `processEnv: false` first.
826+
827+
```json
828+
{ "operation": "grant_secret", "name": "STRIPE_KEY", "component": "payments-service" }
829+
```
830+
831+
Response includes the updated `grants` array and a `changed` flag (`false` when the call was a no-op).
832+
833+
### `list_secrets`
834+
835+
Returns metadata for every secret — **never** envelopes or values. Each entry includes `name`, `kid`, `grants`, `processEnv`, `metadata`, `unverified`, `updated_by`, timestamps, and `kid_matches_custody` (so a stale row on a cloned/rekeyed node is immediately visible). The response also carries the node's `custody_fingerprint` (`null` when no custody is held).
836+
837+
```json
838+
{ "operation": "list_secrets" }
839+
```
840+
841+
Response:
842+
843+
```json
844+
{
845+
"secrets": [
846+
{
847+
"name": "STRIPE_KEY",
848+
"kid": "a1b2c3d4...",
849+
"grants": ["payments-service"],
850+
"processEnv": false,
851+
"metadata": {},
852+
"unverified": false,
853+
"updated_by": "admin",
854+
"__createdtime__": 1700000000000,
855+
"__updatedtime__": 1700000000000,
856+
"kid_matches_custody": true
857+
}
858+
],
859+
"custody_fingerprint": "a1b2c3d4..."
860+
}
861+
```
862+
863+
### `delete_secret`
864+
865+
Removes a secret row by `name`. Not cryptographic erasure — audit/transaction logs and backups retain the encrypted envelope.
866+
867+
```json
868+
{ "operation": "delete_secret", "name": "STRIPE_KEY" }
869+
```
870+
871+
### `get_secrets_public_key`
872+
873+
Returns the cluster secrets public key for client-side envelope encryption. Requires custody on the node.
874+
875+
```json
876+
{ "operation": "get_secrets_public_key" }
877+
```
878+
879+
Response:
880+
881+
```json
882+
{ "public_key": "-----BEGIN PUBLIC KEY-----\n...", "fingerprint": "<hex sha256>" }
883+
```
884+
885+
---
886+
727887
## Replication & Clustering
728888

729889
Operations for configuring and managing Harper cluster replication.

0 commit comments

Comments
 (0)