reacher-z · reacher-z · Jul 26, 2026 · Jul 26, 2026 · Jul 26, 2026
@@ -4,29 +4,40 @@ Graph Engineering separates portable graph meaning from language-specific
 execution.
 
 ```text
-TypeScript builder ─┐
-YAML / JSON ────────┼─> versioned Graph IR ─> compiler ─> durable scheduler
-Python builder ─────┘                             │             │
-                                                  │             ├─ node executors
-                                                  │             ├─ event/checkpoint store
-                                                  │             ├─ artifact store
-                                                  │             └─ telemetry exporters
-                                                  └─ plan / visualize / audit
+Canonical Graph IR JSON ─> compiler ─┬─> in-memory ready-queue scheduler
+                                     ├─> event-sourced start/resume
+                                     └─> plan / Mermaid / DOT / audit
+
+Independent JSON items ─> standalone bounded pipeline ─> terminal item results
 ```
 
 The protocol in `spec/` defines serialization, stable diagnostics, events, and
 conformance fixtures. Native runtimes may use idiomatic APIs internally, but
 their observable behavior must agree on the shared corpus.
 
+General TypeScript/Python builders, YAML input, ArtifactStore/LockManager,
+SQLite/PostgreSQL/S3, distributed workers, and telemetry exporters remain target
+surfaces. Local memory/JSONL events and file checkpoints exist today. Recovery
+currently rebuilds from the authoritative event history; checkpoint
+acceleration is not wired into scheduling.
+
 ## Execution principles
 
 - An edge exists only when data or control policy genuinely flows.
 - Independent ready nodes run concurrently up to explicit limits.
-- Pipelines stream independent items; barriers exist only for cross-item needs.
-- Model output is validated before downstream consumption.
-- Every cycle and dynamic expansion has semantic and hard resource limits.
+- The standalone pipeline streams independent items through bounded buffers;
+  Graph IR `edge.mode: "stream"` remains declarative and is not durable item
+  streaming.
+- Pure router and settled-barrier evaluators are deterministic; scheduler-level
+  conditional routing and deadline/quorum waiting remain planned.
+- Executor output is checked for detached portable JSON before downstream
+  consumption; runtime JSON Schema validation remains planned.
+- Every future cycle and dynamic expansion must have semantic and hard resource
+  limits; the current scheduler rejects implicit cycles and does not execute
+  dynamic GraphPatch revisions.
 - Node results are persisted as they succeed, so a crash does not discard an
-  entire parallel stage.
+  entire parallel stage when callers use event-sourced start/resume. Plain
+  `runGraph` remains in-memory.
 - State recovery never pretends non-idempotent external effects are exactly once.
 
 See `spec/README.md` for protocol details and the master plan under

@@ -19,14 +19,22 @@ migration note.
   side effects may retry within their original budgets, while omitted or
   non-idempotent declarations surface `IN_DOUBT_SIDE_EFFECT`. Resuming a valid
   terminal run returns its recorded result with no new event or executor call.
+- Native standalone `runPipeline` and `run_pipeline` APIs with lazy source
+  intake, bounded end-to-end backpressure, per-stage concurrency and retry,
+  input/completion delivery order, explicit stop/drop/dead-letter outcomes,
+  cooperative cancellation, and shared cross-language behavioral cases.
+- Evidence-gated progress scanning and a full Day 1-21 delivery control surface:
+  107 registered tasks, a dependency graph, ownership map, coverage matrix,
+  three organic-growth plans, and a 178-item stable-v1/RC release checklist.
 
 ### In progress
 
 - Scheduler checkpoint acceleration, replay/fork, and distributed lease/fencing
   providers. Recovery correctness currently comes from the complete event
   stream; checkpoint files are not wired into the scheduler.
-- Streaming pipelines, conditional edge lowering, verifier panels, and bounded
-  runtime loops.
+- Graph IR stream-edge lowering and durable item recovery, conditional edge
+  lowering, verifier panels, and bounded runtime loops. The new standalone
+  pipeline deliberately does not claim these graph/durability semantics.
 
 ## [0.1.0-alpha.1] - 2026-07-26
 

@@ -26,13 +26,15 @@ decisions while scheduler-level conditional routing remains an explicit v1 goal.
 
 The project is an early alpha. The DAG compiler, ready-queue schedulers, safe project
 initializer, machine-readable CLI, structured failure handling, retries,
-timeouts, budgets, settled barrier/router decisions, safe Mermaid/DOT rendering,
-pattern constructors, local event/checkpoint stores, read-only MCP server, and
-development progress scanner are executable today. Both native runtimes also
-provide event-sourced durable start/resume: committed successes are reused after
-process loss and unsafe ambiguous effects fail closed. Checkpoint acceleration,
-replay/fork, distributed leases, and the broader v1 surface remain under active
-development; the repository does not silently mock unfinished capabilities.
+timeouts, bounded concurrency and attempt budgets, settled barrier/router
+decisions, standalone bounded pipelines with backpressure, safe Mermaid/DOT
+rendering, pattern constructors, local event/checkpoint stores, a read-only MCP
+server, and a development progress scanner are executable today. Both native
+runtimes also provide event-sourced durable start/resume: committed successes
+are reused after process loss and unsafe ambiguous effects fail closed.
+Checkpoint acceleration, replay/fork, distributed leases, Graph IR stream
+execution, and the broader v1 surface remain under active development; the
+repository does not silently mock unfinished capabilities.
 
 > **Source-only alpha:** npm and PyPI packages are not published yet. Clone this
 > repository to try the current release candidate; registry publication remains
@@ -90,6 +92,7 @@ uv run --project python pytest python/tests
 | Stable compiler diagnostics | Yes | Yes |
 | Ready-queue DAG scheduler | Yes | Yes |
 | Bounded concurrency | Yes | Yes |
+| Standalone bounded pipeline and backpressure | Yes | Yes |
 | Retry, timeout, attempt budget | Yes | Yes |
 | Failure isolation and named ports | Yes | Yes |
 | Shared compiler/runtime conformance | Yes | Yes |
@@ -100,7 +103,7 @@ uv run --project python pytest python/tests
 | Event-sourced scheduler start/resume | Yes | Yes |
 | Scheduler checkpoint acceleration | Not yet | Not yet |
 | Read-only validation/planning MCP | Yes | Uses the same portable IR |
-| Streaming and scheduler-applied routers/verifier panels/loops | Target v1 | Target v1 |
+| Graph IR streaming and scheduler-applied routers/verifier panels/loops | Target v1 | Target v1 |
 
 ## Design commitments
 
@@ -144,6 +147,7 @@ python3 scripts/check-python-artifacts.py
 - [Runtime semantics](spec/runtime-semantics.md)
 - [Persistence semantics](spec/persistence-semantics.md)
 - [Durable recovery semantics](spec/durable-recovery-semantics.md)
+- [Bounded pipeline semantics](spec/pipeline-semantics.md)
 - [Primitive semantics](spec/primitives-semantics.md)
 - [21-day delivery plan](codex_plans/Graph-Engineering-21-Day-Master-Plan.md)
 

@@ -25,7 +25,10 @@ earned by executable cross-language tests.
 - [ ] Scheduler checkpoint acceleration; correctness already rebuilds from the
   authoritative event history.
 - [ ] Replay and fork with immutable lineage.
-- [ ] Pipeline buffers/backpressure and explicit barrier policies.
+- [x] Standalone native pipeline buffers/backpressure with structured terminal
+  outcomes and shared TypeScript/Python behavioral cases.
+- [ ] Graph IR-integrated/durable item streaming and scheduler-integrated
+  deadline/quorum barrier policies.
 - [ ] Deterministic routers, verifier verdicts, quorum/unknown outcomes, reflection,
   and bounded loop-until-dry primitives.
 - [ ] Deterministic mock, OpenAI, Anthropic, Gemini, OpenAI-compatible, HTTP, shell,

@@ -19,10 +19,17 @@ Until v1, only the newest pre-release is supported. After v1, the latest minor
 release receives security fixes. The policy will be revised before a second
 stable major version.
 
-## Security defaults
-
-- Shell, network, filesystem writes, and secrets require explicit capabilities.
-- MCP mutation is disabled by default.
-- Prompt and response bodies are not recorded by default.
-- External side effects require idempotency declarations or human approval.
-- Worktree cleanup never targets an unresolved broad path.
+## Security boundary and target defaults
+
+The alpha validates graph structure, bounds scheduler work, keeps MCP read-only,
+and gates ambiguous durable retries using the declared side-effect class. Node
+executors still inherit the host process's ambient filesystem, network, shell,
+and environment authority; capability metadata is not yet an enforcement
+boundary. Run untrusted executors only inside isolation you configure outside
+the runtime.
+
+Target-v1 defaults are deny-by-default shell, network, filesystem-write, and
+secret capabilities; explicit approval for non-idempotent external effects;
+prompt/response capture off by default; and path-resolved worktree cleanup.
+These are requirements, not claims about the current alpha. See the detailed
+[security boundary](docs/SECURITY.md#controls-implemented-in-alpha).
@@ -251,3 +251,78 @@ Canonical IR -> language models/builders -> compiler -> deterministic scheduler
   dependency-review, and Python/JavaScript CodeQL check passed. The pull request
   remains intentionally unmerged and review-required; protected-branch policy
   was not bypassed.
+
+## 18:22 UTC bounded-pipeline and full-plan control checkpoint
+
+- Ran parallel full-plan, pipeline, architecture, security, delivery, and growth
+  audits after every assigned agent explicitly read the 355-line master plan.
+  The supporting plan set now exceeds 7,000 lines, including deep Graph IR,
+  runtime, persistence, security, builder/YAML, redaction, release, ownership,
+  dependency, growth, and 178-leaf acceptance documents. These documents record
+  Open work honestly and do not count file presence as implemented capability.
+- Expanded the executable registry from 91 to 107 unique tasks after the gap
+  audit found 16 missing control/implementation lanes. Added independent native
+  adapter and redaction tasks, approval authority, runtime chaos versus durable
+  operations, privacy/usability, support/education, canonical npm distribution,
+  release-leaf mapping, historical candidate revalidation, and a final
+  fail-closed release roll-up. A fresh graph check found no duplicate ID,
+  dangling dependency, or cycle.
+- Completed the standalone bounded pipeline in TypeScript and Python with lazy
+  source admission, bounded queues and global in-flight work, stage concurrency,
+  retry/timeout, stop/drop/dead-letter outcomes, ordered/completion delivery,
+  cooperative cancellation, cleanup, and a hard 2,048-stage construction bound.
+  Eight shared cases agree across languages.
+- Independent adversarial review caught and closed three late defects before the
+  milestone commit: TypeScript accepted explicit-null configuration defaults;
+  Python policy extension keys such as `max_concurrency` could be silently
+  rewritten and change canonical hashes; and Python did not close an overflowing
+  stage iterator. Two new shared policy fixtures now prove exact canonical/hash
+  parity for both null and numeric extension values.
+- Final local evidence is green: workspace build/typecheck/lint/test; Core 41,
+  Persistence 27, Primitives 147, Runtime 115, Patterns 93, MCP 15 and CLI 131
+  tests; Python 587 tests plus 2 subtests, Ruff and strict mypy; 14 Graph IR
+  fixtures plus all runtime/persistence/durable/router/barrier/pipeline joins;
+  seven npm tarball-content and clean-install checks; Python wheel/sdist checks;
+  193 local Markdown links; production audit with no known vulnerability; and
+  diff hygiene.
+- Committed the immutable code/protocol milestone as
+  `3df201db016f2e74d4ee9a96bc28ddabe0d010d8`, authored and committed by
+  `reacher-z <mtrxcop@gmail.com>` without co-author trailers. The registry now
+  binds D7 completion evidence to that revision; push and the stacked draft PR
+  follow after the control/document commit.
+- Verified that the real user-level `graph-progress.timer` remains active on its
+  30-minute schedule. The scanner has explicit evidence supersession and
+  timestamp-integrity rules covered by 17 tests; it reports planned and external
+  dependencies as waiting rather than falsely complete.
+- Started the next critical path at full parallelism: native TypeScript and
+  Python D2 builder/safe-YAML lanes, plus the canonical D9 redaction protocol.
+  Redaction remains a critical open implementation blocker: current durable
+  events expose more raw fields than the initial input/output finding, so stable
+  release claims remain no-go until both language implementations, migration,
+  sink-byte canaries, and independent security review pass.
+
+## 18:50 UTC bounded-pipeline delivery checkpoint
+
+- Committed the full-plan audit, 107-task dependency/evidence registry,
+  architecture and growth controls, and hardened progress scanner as
+  `a39dbdc3f7349965aa2a843dfaebd9475744e040`. Both milestone commits use
+  `reacher-z <mtrxcop@gmail.com>` as author and committer and contain no
+  co-author trailer.
+- Pushed `feat/pipeline-runtime` and opened stacked draft pull request
+  [#15](https://github.com/reacher-z/GraphEngineering/pull/15) against
+  `feat/durable-recovery`. Protected `main` remains untouched and the draft was
+  not merged.
+- All 13 reported GitHub gates passed: Node 20 and 22, Python 3.11 through 3.13,
+  protocol fixtures, cross-language conformance, npm package artifacts,
+  progress scanner, dependency review, JavaScript/TypeScript CodeQL, Python
+  CodeQL, and the aggregate CodeQL result.
+- Reconfirmed the user-level `graph-progress.timer` is active with its next
+  30-minute run scheduled. The latest manual scan reports 107 tasks, 42 healthy,
+  65 dependency-waiting, zero stale/blocked/integration-risk/warning items, and
+  6 of 77 evidence gates satisfied.
+- Froze D2 authoring, safe-YAML, strict typed-port, and initial-revision
+  semantics plus a machine-readable identity schema and ADR. Added initial
+  positive, negative, typed diagnostic, identity, and YAML safety fixtures;
+  these remain work in progress until both native lanes, CLI, three-way shared
+  conformance, package-install smoke, independent review, and immutable
+  candidate evidence all pass.
@@ -0,0 +1,92 @@
+# ADR-0001: Ship the first bounded pipeline as a standalone runtime primitive
+
+- Status: accepted for v1alpha1 implementation
+- Date: 2026-07-26
+- Owners: main, TypeScript runtime, Python runtime
+- Canonical contract: `spec/pipeline-semantics.md`
+
+## Context
+
+Graph IR already reserves `edge.mode: "stream"`, but both native graph
+schedulers currently execute one node once and retain one portable JSON
+`NodeResult` per node. Durable history identifies attempts by run and node, and
+`EdgeEmitted` records one value-edge marker after node success. It has no item
+identity, item offset, acknowledgement, queue state, window/join policy, or
+per-item attempt history.
+
+Directly interpreting a stream edge would therefore change node readiness,
+input/output contracts, attempt and budget accounting, graph outputs, fan-in,
+event folding, crash recovery, and replay at once. Treating an async iterator as
+an ordinary node JSON value would instead be a false implementation: it is not
+portable, hashable, recoverable, or safely replayable.
+
+The Day-5 plan still needs an executable pipeline in which independent items
+occupy different stages concurrently, slow consumers exert source backpressure,
+buffers stay bounded, retries stop, and failures remain structured.
+
+## Decision
+
+The first implementation is a standalone, in-memory `runPipeline` /
+`run_pipeline` primitive shared by the TypeScript and Python runtime packages.
+It has:
+
+- a synchronous factory and single-pass asynchronous result iterator;
+- synchronous or asynchronous sources;
+- immutable ordered stage contracts;
+- bounded stage queues and a bounded global in-flight credit window;
+- a hard item admission limit and statically bounded maximum attempts;
+- input-order or completion-order terminal delivery;
+- bounded deterministic retries and attempt timeouts;
+- explicit stop, drop, and dead-letter policies;
+- structured item and run failures rather than `null` placeholders;
+- cooperative cancellation, explicit close, and task/listener cleanup; and
+- shared cross-language fixtures and a normative protocol document.
+
+The primitive does not activate `edge.mode: "stream"` and does not claim
+item-level durability. When called inside a graph node, the entire pipeline is
+part of that node attempt and can be replayed as a unit under existing recovery
+rules.
+
+## Safety and resource decisions
+
+The source cannot be pulled until an in-flight credit is available. Credit is
+released only when a terminal result reaches the consumer. This makes consumer
+pressure propagate to the source rather than hiding an unbounded result list.
+
+`maxItems` defaults to a finite value and is checked before each source pull.
+The product of `maxItems` and the sum of per-stage attempt bounds must be a safe
+integer. Infinite or adversarial sources consequently have bounded work even
+when callers forget to cancel.
+
+Input ordering applies only to terminal delivery. Internal stage flow remains
+completion-driven so a fast later item can enter a downstream stage while a slow
+earlier item is still upstream. The reorder buffer remains bounded by the
+in-flight window.
+
+Retries are at-least-once and require idempotent external effects. Timeout and
+cancellation can detach non-cooperative user code after observing its eventual
+outcome; they cannot revoke an external side effect already initiated.
+
+## Consequences
+
+This delivers useful pipeline/backpressure semantics without destabilizing the
+ordinary or durable DAG scheduler. It also creates a precise test bed for future
+stream-edge work.
+
+The tradeoff is that a standalone pipeline is not yet a first-class graph edge,
+cannot be checkpointed per item, and does not support stream joins, windows,
+materializing barriers, replay, or fork. Documentation must keep that boundary
+visible.
+
+## Required follow-up before Graph IR stream lowering
+
+1. Add portable item identity and ordinal rules.
+2. Specify per-edge enqueue, consume, acknowledgement, and offset events.
+3. Define crash reconstruction for bounded queues and in-flight handlers.
+4. Define value/stream fan-in: merge, zip, window, and materializing barrier.
+5. Bind per-item attempts, budgets, idempotency keys, and external effects.
+6. Define stream graph outputs, cancellation, replay, fork, and schema evolution.
+7. Add cross-process leases/fencing before distributed workers advance a stream.
+
+Until those contracts exist and pass cross-language crash conformance,
+`edge.mode: "stream"` remains declarative only.