Skip to content

chore: Lean specification draft - #59

Open
aakoshh wants to merge 21 commits into
mainfrom
af/lampe-spec
Open

chore: Lean specification draft#59
aakoshh wants to merge 21 commits into
mainfrom
af/lampe-spec

Conversation

@aakoshh

@aakoshh aakoshh commented Apr 27, 2026

Copy link
Copy Markdown

Problem Resolved

We want to have formal specification for our cryptographic libraries. This PR uses lampe to extract the Noir code into Lean, give it a specification, and prove that the extracted code satisfies it. The main correctness theorem, sha256_var_correct, is fully proved: for any input and any message_size within the array bounds, the Lampe translation of sha256_var returns the SHA-256 digest of the first message_size bytes of the message.

Summary of Changes

  1. The Noir code has been extracted into Lean by running lampe in the root directory. The extraction now works on the source code as-is: Stack overflow: Large array literals reilabs/lampe#270 and Application type mismatch trying to "overwrite" a vector during pop reilabs/lampe#263, which previously required local workarounds, have been fixed upstream, so Extracted/ is reproducible from a pristine checkout. One caveat: extraction currently requires the pending fix in fix: extract inclusive for-loop ranges with their final iteration reilabs/lampe#310 — without it, lampe drops the final iteration of the inclusive loop in build_msg_block, and the spec for it becomes unprovable.
  2. The Lean code is built by running lake build in the sha256/lampe directory (Lean v4.29.1, Lampe pinned to the extractor's commit). The extracted test vectors need a deeper elaborator stack, configured in lakefile.toml per the resolution of Stack overflow: Large array literals reilabs/lampe#270.
  3. lampe/sha256-0.0.0/Spec.lean contains a reference implementation of SHA-256 written directly from FIPS 180-4, checked at compile time against test vectors from src/sha256/tests.nr and the standard, plus STHoare theorems relating each extracted function to it. Functions that depend on the builtin sha256CompressionFn take a lambda so the reference compression implementation can be executed in tests while the (noncomputable) builtin is used in the theorems.
  4. All theorems are proved — there are no sorrys. The proof of sha256_var_correct composes specs for process_full_blocks, build_msg_block, add_padding_byte_and_compress_if_needed, attach_len_to_msg_block, hash_final_block and finalize_sha256_blocks, covering both the compression fold and the lookup-table construction of the constrained path.
  5. Three theorem statements gained hypotheses they were unprovable (or false) without, all discharged at the call sites:
    • a field-size bound 2^64 < p — the asserts that pin the unconstrained oracles' outputs (encode_len, build_msg_block_helper) only do so in a large enough field;
    • msg_start % 4 = 0 for build_msg_block — the word-boundary asserts assume aligned starts, which all callers satisfy;
    • message_size <= N for process_full_blocks — without it the constrained lookup table returns the initial state while the reference folds compressions, so the original statement was false. This is supplied by sha256_var's own assert(message_size <= N).
  6. The remaining trusted base is: the sha256Compression builtin behaving as some fixed compression function (the explicit h_comp hypothesis), the lampe extraction, and Lean's kernel.
  7. Supporting proof infrastructure that may be worth upstreaming to lampe: an STHoare associativity lemma for the nested letIn blocks the extractor emits, a steps closer for the uGeq builtin, and a u32 instantiation of the stdlib's generic min spec.

This PR is currently based on #62, since that fixes soundness bugs in the source that would otherwise be extracted; once #62 merges this rebases onto main cleanly.

PR Checklist

  • I have tested the changes locally.
  • I have formatted the changes with Prettier and/or cargo fmt on default settings.

asterite and others added 19 commits July 23, 2026 15:09
…_end

Reproduces noir-lang/noir-library-claude#16: the constrained path of
partial_sha256_var_end drops the trailing real_message_size % 64 preimage
bytes, diverging from sha256_var. Existing fuzz tests only used 192-byte
(block-aligned) inputs, so this case was never exercised.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…224)

Adds constrained-vs-unconstrained agreement tests and reference tests for
partial_sha256_var_end / partial_sha224_var_end across a range of totals,
including non-block-aligned sizes that the existing 192-byte fuzz tests never
exercised. Also adds an interstitial agreement test.

Non-block-aligned cases currently FAIL (noir-lang/noir-library-claude#16);
block-aligned cases and the interstitial step pass. These tests are landed
before the fix so the fix can be validated against them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The constrained path processed only the block-aligned `message_size` bytes,
so the trailing `real_message_size % BLOCK_SIZE` bytes of the final chunk were
dropped and left unconstrained, diverging from the unconstrained oracle and
from sha256_var. Process the full chunk (block-aligned part plus the trailing
partial block) so the tail is bound, and assert real_message_size >=
message_size.

Fixes noir-lang/noir-library-claude#16.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ained path

partial_sha256_var_end's unconstrained path located the trailing partial block
via finalize_last_sha256_block using the global offset (real_message_size / 64)
* 64. For a composed hash (interstitial calls followed by end) of a
non-block-aligned message, the final chunk is re-based to offset 0, so the
global offset points past it — the oracle read out of bounds (or the wrong
bytes), diverging from the constrained path.

Pass the chunk's local block-aligned offset (message_size) separately from the
length (real_message_size) so the tail is read from the correct place. Both
runtimes now agree for composed non-block-aligned inputs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…nputs

When message_size plus the trailing real_message_size % 64 bytes exceeds the
array capacity N, the partial-hash API reads past the end of msg. Today the
constrained end path silently hashes the missing bytes as zeros (the tests
"pass when they should have failed"), while the unconstrained path and the
interstitial path abort with a bare "Index out of bounds".

These should_fail_with tests expect the call to be rejected with a clear
message; they fail until that guard is added. SHA-224 is covered too since it
delegates to partial_sha256_var_end.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
partial_sha256_var_end and partial_sha256_var_interstitial never checked their
size arguments against the array capacity N. When the block-aligned prefix plus
the trailing partial block exceeded N, the constrained end path silently hashed
the missing bytes as zeros while the unconstrained and interstitial paths read
out of bounds, so the two runtimes diverged on a malformed input.

Assert that the accessed region fits within N (message_size + real_message_size
% BLOCK_SIZE for end, message_size for interstitial), matching the message_size
<= N guard sha256_var already has. Both runtimes now reject the input
identically. Note real_message_size itself may exceed N (it counts bytes
absorbed by earlier interstitial calls), so only the locally-accessed region is
bounded.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Re-run extraction with lampe main (919a3871) on the pristine source:
the workarounds for reilabs/lampe#263 and #270 are no longer needed, so
Extracted/ is now reproducible from a clean checkout. Bump the Lean
toolchain to v4.29.1 and lock Lampe/stdlib to the extractor's commit.
The extracted test-vector array literals need a deeper elaborator stack
(--tstack / maxRecDepth), per the reilabs/lampe#270 resolution.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The new Lampe models tuple destructuring with Builtin.indexTpl
projections, which simp_all does not reduce; they are definitionally
equal to the plain projections used by the reference functions, so
close the residual goals with rfl.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The extractor dropped the final iteration of Noir's inclusive
`for k in msg_start..=msg_end` loop in build_msg_block, so the Lean
model never constrained the last word of a message block, making the
block spec unprovable. Re-extracted with the fix proposed in
reilabs/lampe#310; the only change is the loop's upper bound becoming
`msg_end + 1`.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the five sorry'd base lemmas with full proofs:
build_msg_block_spec, process_full_blocks_spec,
attach_len_to_msg_block_spec, hash_final_block_spec and
add_padding_and_compress_spec. sha256_var_correct now rests on no
sorries; the only assumption is the opaque compression function
(h_comp), plus the extraction and Lean's kernel.

Three theorem statements gained hypotheses they were unprovable (or
false) without:

* a field-size bound `2^64 < p` — the length/word asserts pin the
  unconstrained oracles' outputs only in a large enough field;
* `msg_start % 4 = 0` for build_msg_block — the word-boundary asserts
  assume aligned starts, which all callers satisfy;
* `message_size <= N` for process_full_blocks — without it the lookup
  table returns the initial state while the reference folds, so the
  original statement was false. This is discharged by sha256_var's own
  assert, which also makes the previously-unused h_bound load-bearing.

Adds supporting infrastructure: an STHoare letIn associativity lemma to
flatten the nested blocks lampe emits, a steps closer for the uGeq
builtin, constant specs, a u32 instance of the stdlib min spec, and the
reference-side helper lemmas the loop invariants are stated with.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@asterite
asterite changed the base branch from main to test/partial-hash-dual-mode-coverage July 28, 2026 12:20
@asterite
asterite marked this pull request as ready for review July 28, 2026 12:29
Base automatically changed from test/partial-hash-dual-mode-coverage to main July 29, 2026 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants