Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/self-hosted/oel/keto/changelog/v26.2.12.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
No changelog entries found for keto/oel in versions v26.2.12
134 changes: 134 additions & 0 deletions docs/self-hosted/oel/kratos/changelog/v26.2.12.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
## v26.2.12

### Jsonnet worker is fully isolated from the filesystem

The hidden `jsonnet` subcommand that Kratos re-execs to evaluate Jsonnet mappers (OIDC claim mappers, courier templates,
identity-schema transforms) now applies an empty [Landlock](https://docs.kernel.org/userspace-api/landlock.html) layer at startup.
The worker only ever needs the inherited stdin/stdout/stderr — every path-based filesystem access from inside the worker is now
denied by the kernel, even if a Jsonnet snippet were to bypass the in-process import barrier. Already-open file descriptors keep
working, so the parent ↔ worker IPC is unaffected. This applies to Kratos OSS, Cloud, and OEL on Linux 5.13 and later, and is a
no-op on other platforms.

This is not configurable, for simplicity and security: there is no legitimate use case to allow the Jsonnet VM to read from disk.

### Landlock filesystem sandbox for kratos serve (https://docs.kernel.org/userspace-api/landlock.html)

#### Filesystem sandbox for kratos serve (OEL)

Kratos OEL activates the [Landlock](https://docs.kernel.org/userspace-api/landlock.html) sandbox for the main `kratos serve`
process after initialization. The process is then restricted to the files it needs at runtime:

- `/dev/null` (subprocess plumbing)
- `/etc/resolv.conf` and `/etc/hosts` (Go's pure-Go DNS resolver)
- the running Kratos binary itself, with read+execute, so `jsonnetsecure` can re-exec it as a sandboxed worker
- TLS certificates and keys for the public and admin listeners
- the courier template directory (courier templates are loaded at runtime)
- SMTP client certificate and key files (`courier.smtp.client_cert_path`, `courier.smtp.client_key_path`, and the equivalents
under `courier.channels[].smtp_config`)
- **every file referenced from the config via a `file://` URI.** Kratos walks the loaded config once at startup and allows any
value that begins with `file://` — identity schemas, OIDC mapper templates, `web_hook` body templates, courier HTTP body
templates, session tokenizer mappers and JWKS files, and anything added to the config schema later. Operators do not need to
duplicate these paths under `security.landlock.allowed_paths`.
- the directory containing the SQLite database — covers the database file itself and any `-journal` / `-wal` / `-shm` / transient
`-mj-XXXXX` siblings
- in the case of SQLite: `/tmp`, `/var/tmp` and `/usr/tmp` since they are needed by SQLite.
- any paths listed in `security.landlock.allowed_paths`

The system trust store at `/etc/ssl` is **not** allowed. Operators who need to trust an additional CA must point `SSL_CERT_FILE` /
`SSL_CERT_DIR` at the file and list it under `security.landlock.allowed_paths`. All other filesystem access is denied by the
kernel after activation. This includes the `/proc` and `/sys` virtual filesystems.

The sandbox degrades gracefully on older kernels and non-Linux platforms, so no action is required for deployments that don't
support Landlock. To opt out completely (not recommended in production), set `security.landlock.disabled: true`.

Configuration hot-reload still works as before with one caveat: Landlock restrictions are irrevocable for the lifetime of the OS
process. Hot-reloading a config that flips `security.landlock.disabled` from `false` to `true` will not lift the sandbox — the
process must be restarted for the change to take effect. Other config changes (allow list entries, courier templates, etc.) reload
normally, but newly-introduced paths are only honoured on the next process start.

Check warning on line 47 in docs/self-hosted/oel/kratos/changelog/v26.2.12.md

View workflow job for this annotation

GitHub Actions / misspell

[misspell] docs/self-hosted/oel/kratos/changelog/v26.2.12.md#L47

"honoured" is a misspelling of "honored"
Raw output
./docs/self-hosted/oel/kratos/changelog/v26.2.12.md:47:46: "honoured" is a misspelling of "honored"
Comment thread
adamwalach marked this conversation as resolved.

#### Symlinks

Symlinks in any configured path — `--config` files, TLS paths, `identity.schemas[].url`, the SQLite DSN,
`security.landlock.allowed_paths`, and so on — are followed by the kernel when the rule is added at startup, so the rule attaches
to the inode the symlink resolves to at that moment. As long as the target does not change, accesses through the symlink keep
working transparently.

Landlock rules are irrevocable for the lifetime of the process. If a symlink target swaps at runtime — for example when
cert-manager or certbot renews a certificate by writing a new file and re-pointing the symlink — the rule still references the
original target inode. Accesses through the symlink then resolve to the new inode, which is not in the allowlist, and the kernel
denies them with `EPERM`. Restart `kratos serve` after any such swap so the rules attach to the new inodes.

#### Troubleshooting

A path that the sandbox does not allow surfaces in the application as `EPERM` ("Operation not permitted") on `open(2)` /
`openat(2)` / `execve(2)` — Kratos typically logs it as `permission denied` while loading a config file, schema, template, or TLS
material. To distinguish a Landlock denial from a regular Unix permission error, use the steps below.

1. **Confirm the sandbox is the cause.** Check the Kratos startup logs for the line:

```
level=info msg="Landlock filesystem sandbox is active."
```

Just before it, two log lines list every path that was added to the allowlist:

```
level=info msg="Landlock: collected roPaths." roPaths=[...]
level=info msg="Landlock: collected rwDirs." rwDirs=[...]
```

If the path that triggered `EPERM` is missing from both lists, Landlock is the cause. As a sanity check, restart with
`security.landlock.disabled: true`: if the error disappears, the denial came from the sandbox.

2. **Read the kernel audit log.** On Linux 6.10 and later, Landlock emits a kernel audit record for every denied access. The
record names the syscall, the resolved path, and the denied access right. Read it with one of:

```bash
sudo journalctl -k --since "5 minutes ago" | grep -i landlock
sudo dmesg -T | grep -i landlock
sudo ausearch -m LANDLOCK_DENY -ts recent # auditd-based distros
```

A typical record looks like:

```
audit: type=1334 audit(...): domain=2 op=fs blockers=fs.read_file path="/etc/kratos/schemas/fragments/address.json" dev="vda1" ino=131072
```

The `path=` field is exactly what to add to `security.landlock.allowed_paths`. Older kernels (5.13 – 6.9) do not emit these
records — fall back to step 3 there.

3. **Trace the syscall directly.** When the audit log is unavailable or the path is templated, attach `strace` to the running
process and watch for `EPERM` on the relevant syscalls:

```bash
sudo strace -f -p "$(pgrep -f 'kratos serve')" -e trace=openat,execve -e status=failed
```

Lines ending in `= -1 EPERM (Operation not permitted)` show the exact path the kernel rejected, even when Kratos's own log
message has been swallowed by a wrapper.

4. **Fix the configuration.** Once the offending path is known, add it to the allowlist (a directory grants every file underneath;
a file grants only itself), then restart `kratos serve` — Landlock rules are immutable for the lifetime of the process, so a
hot reload will not lift the denial.

```yaml
security:
landlock:
allowed_paths:
- /etc/kratos/schemas/fragments/address.json
```

### Normalize phone trait values to E.164 in webhook payloads

When a phone trait is configured as a credential identifier (`code` strategy via `sms`), or as a recovery or verification address
via `sms`, Kratos now rewrites the trait value to its E.164 form during validation.

Previously, only the credential identifier and the recovery and verification address tables stored the normalized value, while the
`traits` JSON kept the raw user input. Webhook payloads templated against `identity.traits.phone` therefore saw a different value
than the one Kratos used internally as the identifier. The trait is now consistent with the canonical form, so webhook consumers
and Kratos see the same phone number.

Existing identities are not rewritten by this change. Use the `kratos cleanup normalize-phones` command to migrate stored
identifiers; the trait values are corrected the next time the identity passes through validation (for example, on a settings or
recovery flow that updates the trait).
1 change: 1 addition & 0 deletions docs/self-hosted/oel/oathkeeper/changelog/v26.2.12.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
No changelog entries found for oathkeeper/oel in versions v26.2.12
1 change: 1 addition & 0 deletions docs/self-hosted/oel/oauth2/changelog/v26.2.12.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
No changelog entries found for hydra/oel in versions v26.2.12
Comment thread
adamwalach marked this conversation as resolved.
1 change: 1 addition & 0 deletions docs/self-hosted/oel/oel-hydra-image-tags.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
| Image Tag | Release Date |
| ---------------------------------------- | ------------ |
| 26.2.12 | 2026-05-20 |
| 26.2.11 | 2026-05-15 |
| 26.2.10 | 2026-05-11 |
| 26.2.9 | 2026-05-04 |
Expand Down
1 change: 1 addition & 0 deletions docs/self-hosted/oel/oel-keto-image-tags.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
| Image Tag | Release Date |
| ---------------------------------------- | ------------ |
| 26.2.12 | 2026-05-20 |
| 26.2.11 | 2026-05-15 |
| 26.2.10 | 2026-05-11 |
| 26.2.9 | 2026-05-04 |
Expand Down
1 change: 1 addition & 0 deletions docs/self-hosted/oel/oel-kratos-image-tags.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
| Image Tag | Release Date |
| ---------------------------------------- | ------------ |
| 26.2.12 | 2026-05-20 |
| 26.2.11 | 2026-05-15 |
| 26.2.10 | 2026-05-11 |
| 26.2.9 | 2026-05-04 |
Expand Down
1 change: 1 addition & 0 deletions docs/self-hosted/oel/oel-oathkeeper-image-tags.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
| Image Tag | Release Date |
| ---------------------------------------- | ------------ |
| 26.2.12 | 2026-05-20 |
| 26.2.11 | 2026-05-15 |
| 26.2.10 | 2026-05-11 |
| 26.2.9 | 2026-05-04 |
Expand Down
1 change: 1 addition & 0 deletions docs/self-hosted/oel/oel-polis-image-tags.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
| Image Tag | Release Date |
| ---------------------------------------- | ------------ |
| 26.2.12 | 2026-05-20 |
| 26.2.11 | 2026-05-15 |
| 26.2.10 | 2026-05-11 |
| 26.2.9 | 2026-05-04 |
Expand Down
1 change: 1 addition & 0 deletions docs/self-hosted/oel/polis/changelog/v26.2.12.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
No changelog entries found for polis/oel in versions v26.2.12
Loading