Cap LUKS Argon2id memory to avoid SIGKILL on live install

[vincenzopalazzo]

Summary

• Encrypted install dies during cryptsetup luksFormat with Signals.SIGKILL: 9 and Could not encrypt volume on memory-constrained live/PXE boots
• Root cause: archinstall/cryptsetup default Argon2id memory cost is ~1 GiB; the live environment OOM-kills cryptsetup mid-format
• Cap --pbkdf-memory from MemAvailable (min 64 MiB, max 1 GiB, target 1/8 available) while keeping --iter-time 2000 as the PBKDF hardness control

Reproduction

What I was doing:

• Installing Omarchy 3.8.40 on a bare-metal desktop over LAN/PXE (USB boot not working on this machine)
• PXE server on the LAN; client NIC MAC d0:50:99:5a:3e:5e (ASRock-like desktop)
• Went through the Omarchy configurator with disk encryption enabled (default path)
• Selected /dev/sda as the install disk and confirmed overwrite
• Installer started archinstall full-disk layout: keyring populate, package DB sync, wipe /dev/sda, create partitions
• Failed immediately while formatting the LUKS root partition /dev/sda2

Live environment notes:

• ISO label OMARCHY_202607, airootfs ~7.1G squashfs over HTTP/PXE
• archinstall under Python 3.14 (/usr/lib/python3.14/site-packages/archinstall/...)
• Config used archinstall disk_encryption with encryption_type: luks and iter_time: 2000 (no --pbkdf-memory)

Failure log

Captured from the installer TTY when it crashed:

==> Appending keys from archlinux.gpg...
==> Updating trust database...
gpg: next trustdb check due at 2026-10-21
==> Appending keys from omarchy.gpg...
==> Updating trust database...
gpg: next trustdb check due at 2026-10-21
:: Synchronizing package databases...
 offline downloading...
Cleaning up existing holders on install disk: /dev/sda
Fetching Arch Linux package database...
Starting device modifications in
5...4...3...2...1
Wiping partitions and metadata: /dev/sda
Creating partitions: /dev/sda
Traceback (most recent call last):
  File "/usr/lib/python3.14/site-packages/archinstall/lib/disk/luks.py", line ..., in encrypt
    result = run(cmd, input_data=passphrase)
  File "/usr/lib/python3.14/site-packages/archinstall/lib/command.py", line ..., in run
    return subprocess.run(
      cmd,
      ...
      check=True,
    )
  File "/usr/lib/python3.14/subprocess.py", line 578, in run
    raise CalledProcessError(retcode, process.args,
                             output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command
  ['cryptsetup', '--batch-mode', '--verbose', '...', 'luksFormat', '/dev/sda2']
  died with <Signals.SIGKILL: 9>.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.14/site-packages/archinstall/main.py", line 197, in main
    rc = run()
  File "/usr/lib/python3.14/site-packages/archinstall/main.py", line 171, in run
    module.main(arch_config_handler)
  File "/usr/lib/python3.14/site-packages/archinstall/scripts/guided.py", line 248, in ...
    fs_handler.perform_filesystem_operations()
  File "/usr/lib/python3.14/site-packages/archinstall/lib/disk/filesystem.py", line 70, in ...
    self._format_partitions(mod.partitions)
  File "/usr/lib/python3.14/site-packages/archinstall/lib/disk/filesystem.py", line 93, in ...
    device_handler.format_encrypted(
      part_mod.safe_dev_path,
      ...
      self._enc_config,
    )
  File "/usr/lib/python3.14/site-packages/archinstall/lib/disk/device_handler.py", line ...
    key_file = luks_handler.encrypt(iter_time=enc_conf.iter_time)
  File "/usr/lib/python3.14/site-packages/archinstall/lib/disk/luks.py", line 107, in encrypt
    raise DiskError(f'Could not encrypt volume "{self.luks_dev_path}": {output}')

cryptsetup is SIGKILL'd mid-luksFormat with no cryptsetup stderr — classic OOM of the Argon2id memory cost on a RAM-tight live session.

What changed

• New omarchy-luks-pbkdf-memory helper picks a safe memory cost
• Protected-mode path: configurator luksFormat now passes --pbkdf argon2id --pbkdf-memory … --iter-time 2000
• Full-disk path: omarchy-iso-install patches archinstall Luks2.encrypt before Python imports it so the same cap applies
• Test: test/luks-pbkdf-memory-test.sh

Test plan

• bash test/luks-pbkdf-memory-test.sh
• Patcher applied cleanly to ISO-shipped archinstall luks.py and py_compile succeeded
• bash -n on modified shell scripts
• Rebuild ISO / boot live image with encryption enabled on a machine with limited free RAM (or PXE)
• Confirm install log shows the chosen Argon2id memory cost and luksFormat succeeds without SIGKILL

[Copilot]

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

[omarchybot]

Reviewed this on a disposable VM; test/all is green (11 cidata cases, 21 Python tests, plus your new test). One fix pushed as af859a7, and a few things that need you.

Pushed: configs/airootfs/usr/local/bin/omarchy-luks-pbkdf-memory was missing from file_permissions in configs/profiledef.sh. bin/omarchy-iso-make runs lint_file_permissions() before every build and exits non-zero for any executable under configs/airootfs/{usr/local/bin,root} that isn't declared there, so the ISO could not be built with this branch at all. Verified by running that lint against the tree before and after.

Worth knowing why that guard matters more than it looks: both call sites gate on command -v omarchy-luks-pbkdf-memory, and in bash command -v succeeds for a non-executable file that is on PATH — I checked on bash 5.3, it prints the path and returns 0. So if the helper had ever shipped 0644, neither caller would have taken the 262144 fallback. omarchy-iso-install would have died at line 44 under set -e before archinstall started, and the configurator would have handed cryptsetup an empty --pbkdf-memory value. Testing the exit status and that the output is numeric would be sturdier than command -v.

The memory cap costs more than --iter-time gives back. --iter-time 2000 isn't a compensating control here: 2000 ms is already cryptsetup's default, and the full-disk path already sends "iter_time": 2000 in user_configuration.json, so neither path raises it. What actually happens is that cryptsetup's benchmark raises the iteration count to refill the 2 s budget once memory is pinned lower — but Argon2's cost to an attacker is area-time, roughly memory squared times passes, so time recovers linearly what memory loses quadratically. Dropping from cryptsetup's 1 GiB to 256 MiB is about 4x cheaper to brute-force (~2 bits); the 64 MiB floor is about 16x (~4 bits). RFC 9106 does list 64 MiB as its memory-constrained recommendation so the floor isn't absurd, but the comments at configurator:348 and omarchy-iso-install:29 describe hardness being preserved, and what is preserved is unlock latency.

Deriving the parameter from MemAvailable makes it non-deterministic. On one 4 GiB VM the helper printed 281698, 282241 and 285078 KiB across three runs, and 102782 KiB with 1.5 GiB of ballast resident — a 2.75x swing in key-derivation strength from nothing but what else happened to be in memory at that moment. On a PXE boot with a 7 GiB squashfs in RAM the spread is wider. Two users on identical hardware get materially different disks, and neither is told which. cryptsetup luksDump does record the real parameters, so it's recoverable after the fact, but it isn't a choice anyone made. A fixed value that every Omarchy install shares, with the dynamic path only as a documented fallback when it genuinely will not fit, would be easier to reason about and to change later.

The floor breaks the 1/8 promise exactly where it matters. With 256 MiB available the helper asks for 64 MiB, which is a quarter, not an eighth; with 64 MiB available it asks for all of it. The clamp is doing the opposite of its stated job on the most constrained machines — the ones this PR is for — so those can still be OOM-killed. If 64 MiB is a security floor rather than a safety fraction, the helper probably needs to say so and fail (or warn) when the floor doesn't fit, rather than return a number that cannot work.

The archinstall patch fails open on every path. find_spec returning nothing, the file not being writable, python erroring — all land on return 0 behind 2>/dev/null || true with no message at all, and a regex miss prints a warning and sys.exit(0). In each case the install proceeds and gets SIGKILLed exactly as before, which is the one outcome the PR exists to prevent. Since a silent no-op is indistinguishable from a successful patch, it's worth failing loudly, or at minimum logging on every path. Two smaller things in the same function: find_spec("archinstall.lib.disk.luks") imports archinstall and its parent packages as a side effect, which is precisely the import-time behaviour main.py and archinstall_adapter.py go out of their way to control, and any exception it raises is swallowed by that 2>/dev/null. And the idempotence check greps the whole file for --pbkdf-memory, so if upstream archinstall ever grows its own flag the patch silently skips. The regex itself is correct against current archinstall — \s* spans the newline and tabs between '--iter-time', and str(iter_time),. The needle variable at line 61 is dead.

The test doesn't test the helper. It runs the helper once against the real /proc/meminfo and only checks the result falls between the floor and the cap, then reimplements the formula inline against the fake meminfo files. Replacing the helper's body with printf '65536\n' leaves every assertion passing, so a divisor or parsing regression would not be caught. Letting the helper read its meminfo path from an env var would give the test something real to drive.

No header-compatibility problem: --pbkdf-memory only changes the stored keyslot KDF parameters, unlocking reads them back out of the LUKS2 header, and a lower cost is strictly easier for the installed system's initramfs to satisfy.

The branch is conflicting and the rebase is yours. The single conflict is the luksFormat block itself: quattro now wraps both luksFormat and cryptsetup open in if ! ...; then disk_abort_hook ...; fi and folds cryptsetup's stderr into stdout so failures reach the install log, with a comment explaining the passphrase must not become an argv. Resolving toward this branch would drop all of that, so the new flags want to go into that guarded pipeline. run_partition_install is also run_partition_execute now. PR #111 touches the same file but different functions, so it shouldn't get in your way.

[vincenzopalazzo]

@omarchybot thanks for the review — every point held. Pushed a rebase onto current quattro plus the contract changes.

Rebase / conflict. The luksFormat conflict was exactly run_partition_install vs run_partition_execute. The --pbkdf argon2id --pbkdf-memory … --iter-time 2000 flags now sit inside quattro's existing if ! printf | cryptsetup … 2>&1; then disk_abort_hook; fi pipeline (passphrase stays off argv, stderr still folded). --tailscale-authkey-file and --defer-provisioning-file are preserved on omarchy-iso-install. file_permissions for the helper is still there.

Helper contract. Dropped the MemAvailable/8 sliding scale. Omarchy default is a fixed 256 MiB (262144 KiB) so every install gets the same KDF params. We only step down to the RFC 9106 64 MiB floor when that default cannot fit (MemAvailable >= 2 × cost), and we refuse to format (non-zero exit) when even the floor cannot. Comments no longer claim --iter-time preserves attacker cost — Argon2id area-time scales with memory² × passes, so lowering memory is a real hardness cut. OMARCHY_LUKS_MEMINFO is overridable for tests.

Patcher. Encrypted full-disk path is fail-closed: missing/unwritable luks.py, helper failure, or a regex miss aborts instead of leaving cryptsetup at the 1 GiB default. Locate the file by walking sys.path for archinstall/lib/disk/luks.py (no find_spec, so no side-effect import). Dead needle is gone. Idempotence is the --iter-time/str(iter_time)/--pbkdf-memory sequence, not a whole-file grep --pbkdf-memory.

Callers. Both configurator and omarchy-iso-install require helper exit 0 and numeric stdout. No command -v (bash 5.3 treats a 0644 PATH hit as found, which would invoke and die 126 / hand cryptsetup an empty --pbkdf-memory).

Tests. test/unit/luks-pbkdf-memory-test.sh now actually invokes the helper against fake meminfo via OMARCHY_LUKS_MEMINFO (default, floor, refuse). Inlining the formula is gone, so a stubbed printf '65536\n' helper would fail.

Happy to drop the 256 MiB default to something else if that's the Omarchy number you want — 256 was the value you sketched.

[omarchybot]

Re-reviewed the rebase on a disposable VM. test/all is green before and after both commits below — 11 cidata cases, your luks-pbkdf-memory-test.sh (all six), the partition-numbering suite and 58 Python tests, exit 0.

The conflict resolution is right, and I checked it rather than took your word for it. Both guards survived: configurator:757 sits inside if ! printf "%s" "$password" | cryptsetup luksFormat …; then disk_abort_hook "Formatting LUKS2 … failed"; fi, cryptsetup open keeps its own at 763, 2>&1 is still folded so a failure lands in the install log, and disk_abort_hook still reaches abort, which exits 1. The new --pbkdf argon2id --pbkdf-memory --iter-time 2000 flags are inside that pipeline, not beside it. cryptsetup luksFormat appears exactly once in the whole tree, so with archinstall's own call patched there is no third call site running at the 1 GiB default. The file_permissions line survived the rebase, and moving the test into test/unit/ was the right call — test/all only globs that directory since #109. Everything from the last round holds up as fixed.

Pushed 9e2cec8 — the new install_is_encrypted guard asked a file that does not decide encryption. It gated the patch on user_encrypt_installation.txt, but whether archinstall runs luksFormat comes from disk_config.disk_encryption in user_configuration.json (archinstall_adapter.py:145, reached from phases_impl.py:243), and README says of the flag file, in as many words, that it drives the encrypted install's autologin and boot validation, "not the encryption". So an autoinstall drive whose config carries a disk_encryption block while the flag file says false — or True, or 1, both of which context.py:109 accepts for that same file and your comparison did not — skipped the patch and left cryptsetup at ~1 GiB. That is the SIGKILL, on exactly the unattended path where nobody is watching the TTY. It now asks disk_config.disk_encryption first, the way phases_impl.py:1237 already does, and falls back to the flag file only when the config cannot be read. I checked the twelve combinations of config and flag; the old guard skipped the patch on an encrypting config with false in the flag file, the new one does not.

Pushed 9b6ee70 — the already-patched branch kept the first run's number. It exited before rewriting, so a second run of omarchy-iso-install in the same live session left whatever the first run wrote. The helper re-reads MemAvailable each time and can cross the 128 MiB floor between attempts, so a retry after a failed install could print capping archinstall LUKS Argon2id memory to 262144 KiB and hand archinstall the 65536 still in the file — or the reverse, which is the OOM again, on the retry. It now substitutes the value in place. Tested against the real upstream archinstall/lib/disk/luks.py: patches once, refreshes on re-run without ever producing a second --pbkdf-memory, the file still compiles, and an unmatched file still exits 1.

One left for you: the locator only knows 4.x. The comment at omarchy-iso-install:107 says it matches "the stable archinstall 3.x/4.x shape", but 3.x kept the class at archinstall/lib/luks.py — the move under lib/disk/ is a 4.x change (checked 3.0.15 against 4.4 upstream). The shipped ISO is 4.x, so nothing is broken today; it is the comment that is wrong, and now that the path is fail-closed it is worth being deliberate about, because a locator miss aborts every encrypted install rather than quietly under-protecting one. Either widen the walk to both paths or say 4.x in the comment.

While verifying, two things I could confirm for you: archinstall already passes --pbkdf argon2id itself, so --pbkdf-memory is honoured without your patch adding the algorithm; and the tabs you insert are fine because they land inside a list literal, where indentation is not significant.

The 256 MiB itself is still the maintainer's number to pick, not mine and not yours — it is a permanent, silent reduction from cryptsetup's ~1 GiB for every Omarchy encrypted install, about 4x cheaper to brute-force. Nothing else is blocking.

[vincenzopalazzo]

@omarchybot thanks — both of the commits you pushed check out, and I took the remaining locator item.

9e2cec8. Gating on user_encrypt_installation.txt was the wrong question. archinstall_adapter.is_encrypted() reads disk_config.disk_encryption out of user_configuration.json; the flag file is the autologin/boot-validation signal, not the encryption one. Asking the config first (and only falling back to the flag, including True/1/yes) is the right guard. An autoinstall whose config encrypts while the flag says false would have skipped the patch and hit the SIGKILL on the unattended path.

9b6ee70. Exiting on the already-patched branch kept the first run's number. The helper re-reads MemAvailable and can cross the floor between attempts, so a retry could log 262144 and hand archinstall 65536 — or the reverse, which is the OOM again. Substituting in place keeps a single --pbkdf-memory and makes the log match what archinstall uses.

Locator. Widened the walk in b4d3eaf: lib/disk/luks.py (4.x, what the ISO ships) then lib/luks.py (3.x). Still fail-closed if neither file is on sys.path. The comment no longer claims the 4.x-only path is a 3.x/4.x shape; the encrypt argv match ('--iter-time', str(iter_time),) is what both versions share.

256 MiB remains the maintainer's number to pick.

[omarchybot]

Re-reviewed the delta only — b4d3eaf, the widened locator. Nothing found in it. test/all is green on the new head on a disposable VM (11 cidata cases, all six luks-pbkdf-memory-test.sh assertions, the partition-numbering suite, 58 Python tests, exit 0). Nothing pushed this round.

Your push is a fast-forward and nothing of ours was lost. b4d3eaf's parent is exactly 9b6ee70, so there was no force-push and no partial revert. One thing worth stating plainly because it looks alarming in a log: af859a7 is not in the history any more — it was rebased away in the earlier round and survives as 517d960, and the two have an identical patch-id, so the file_permissions line is the same change byte for byte, still at configs/profiledef.sh:43. The other two are intact in the source, not just in the history: the guard at omarchy-iso-install:42 still asks .disk_config.disk_encryption first, and the already-patched branch at :129 still refreshes the value instead of exiting.

The 3.x claim checks out, and I checked it against the real upstream file rather than the commit message. archinstall 3.0.15 does have archinstall/lib/luks.py, Luks2.encrypt there builds the same argv — '--pbkdf', 'argon2id', … '--iter-time', str(iter_time), *key_file_arg, '--use-urandom', 'luksFormat' — at the same three-tab depth as 4.x, and it already passes --pbkdf argon2id itself. So the regex and the \t\t\t insertion need no version-specific handling. Mutation-tested on the worker against the genuine 3.0.15 and current-master files: 4.x-only patches, 3.x-only patches and still compiles, a re-run at a different cost refreshes in place and never leaves two --pbkdf-memory entries, a renamed --iter-time argv exits 1, an unwritable file exits 1, neither file present exits 1.

The loop nesting is right, and it is right for a reason the comment doesn't say. The walk is outer over sys.path, inner over the two candidates, so a 3.x tree in an earlier sys.path entry beats a 4.x tree in a later one — the comment reads as if 4.x always wins. That ordering is the correct one: it matches Python's own resolution, so the file that gets patched is the file import archinstall would actually load. Preferring 4.x globally would patch a package that never gets imported. Not asking you to change anything; noting it because the comment and the code say slightly different things and the code is the one that is right.

On the low-memory path the cap cannot be bypassed. The helper's only two successful outputs are 262144 and 65536, both below cryptsetup's ~1 GiB default; there is no branch that returns "use the default". Below 128 MiB MemAvailable it exits non-zero, and that refusal is handled rather than swallowed on both paths — configurator:749 routes it to disk_abort_hook, which unmounts, closes the mapping, rolls back the partitions it created and aborts; omarchy-iso-install:60 reports and exits before archinstall or any disk operation starts. And if cryptsetup is SIGKILLed anyway despite the cap, configurator:756 catches the non-zero status with stderr already folded into the install log, and on the full-disk path the exception becomes a PhaseError at phases.py:61, recorded as a failed phase with a traceback and a non-zero installer result.

One thing the second reviewer contributed that I had not reasoned about: --pbkdf-memory is a limit, not an allocation — cryptsetup 2.8.7's own help calls it "PBKDF memory cost limit" — so its benchmark can settle below 262144, and the number in the log is a ceiling rather than what every disk ends up with. It agreed with the rest of the above, though its independence is not currently guaranteed, so treat the agreement as weaker evidence than the checks themselves.

Still the maintainer's call and still the only thing outstanding: 256 MiB as Omarchy's Argon2id cost for every encrypted install, down from cryptsetup's ~1 GiB. Nothing else is blocking.