Conductor: define reusable use-case workflows ("Paperclip Process")

[Weegy]

Problem

Conductor workflows today are built ad hoc for each situation. There is no library of predefined, reusable use-case templates that a team could pick up and run without designing the whole flow from scratch each time. This makes it hard to standardize common processes across installs. It also blocks downstream features that assume a catalog of ready-made workflows already exists.

Proposed solution

Add a "Paperclip Process" concept to Conductor: a set of predefined, reusable use-case workflows that ship with omadia or can be authored once and reused across runs. Users would select from this library instead of building a workflow from zero each time. Issue #330 (Facilitator) depends on this, since it needs a pool of ready-made use-case workflows to draw from.

[Weegy]

Triage

Priority: P2 · Category: feature · Effort: M · Status: needs design (v1 path is clear, three decisions open)

Nothing of this exists on main yet: there is no template/library/catalog concept anywhere in middleware/src/conductor/ or the conductor UI — workflows are authored ad hoc via the designer + builder chat and published through POST /v1/operator/conductors. The good news: the substrate (versioned workflow store, graph validation, roles, schedules) is complete enough that a v1 template library is mostly plumbing. The hard part is parameterization: validate() in @omadia/conductor-core rejects graphs that reference agents/roles/actions/events unknown to the install (unknown_agent_ref, unknown_role_ref, unknown_action_ref, unknown_event_ref), so templates cannot be publish-ready JSON as-is — they need declared "slots" the operator maps to install-local entities before publish.

Implementation notes

Affected code:

• middleware/src/conductor/workflowStore.ts — ConductorWorkflowStore.createOrPublish() is the reuse point; instantiating a template is just a publish with a fresh slug. No new persistence needed for a file-based catalog.
• middleware/src/conductor/routes.ts — add GET /templates and POST /templates/:id/instantiate next to the existing GET / / POST / handlers (router is mounted auth-gated at /api/v1/operator/conductors/* from middleware/src/index.ts).
• middleware/packages/conductor-core/src/ — validate(), conductorGraphSchema, and WorkflowGraph (types.ts) are the validation/type surface; a TemplateManifest type (graph + slot declarations) would live here so both middleware and web-ui share it.
• middleware/src/conductor/templates/ — new directory for bundled template JSON (pattern precedent: middleware/assets/boilerplate/ for shipped assets).
• web-ui/app/conductor/page.tsx + web-ui/app/conductor/_components/ConductorCanvas.tsx — the page already has an edit flow that loads a workflow into the designer; "start from template" is the same flow seeded from a template graph. API client additions go in web-ui/app/_lib/api.ts (CONDUCTOR_BASE = '/v1/operator/conductors').
• middleware/src/conductor/roleStore.ts / builderAgent.ts — role slot mapping and optional builder-chat awareness of templates ("customize this template").

Suggested approach:

1. Define a TemplateManifest format in conductor-core: { id, name, description, useCase, graph, slots: { roles: [...], agents: [...], actions: [...], events: [...] } }. Slots enumerate every external ref the graph makes, with human-readable labels.
2. Ship 3–5 curated manifests as JSON in middleware/src/conductor/templates/, each shape-checked against conductorGraphSchema in a unit test so the catalog can never drift from the graph schema.
3. Add GET /templates (list manifests, no DB) and POST /templates/:id/instantiate (body: slug, name, slot→entity mapping; server substitutes refs, runs validate(), then createOrPublish). Missing-mapping errors reuse the existing conductor.invalid_graph error envelope.
4. Web-ui: template gallery section on /conductor above the workflows list; picking one opens a slot-mapping form (roles from GET /roles, agents from GET /agents, actions from GET /actions — all existing routes), then either publishes directly or loads the substituted graph into the designer for editing before publish.
5. Defer plugin/hub-distributed templates to a follow-up; the plugin surface's only Conductor-facing capability today is ctx.events.emit (see middleware/src/platform/pluginContext.ts), and adding a template-registration capability is a separate security review.

Open decisions (why not ready-to-implement yet):

• Which use-cases ship in v1 (approval chain, escalation, recurring report, onboarding checklist are the obvious candidates — needs a maintainer pick).
• Product naming: "Paperclip Process" reads as an internal codename; user-facing copy should probably say "workflow templates".
• Whether omadia Facilitator — agent that moderates group chats to a defined outcome (+ Conductor JIT/ephemeral workflows & Core group primitives) #330 (Facilitator) needs templates to be instantiable ephemerally (JIT, no persisted workflow row) — if yes, that changes step 3 and should be decided before building.

Risks / dependencies:

• Slot substitution must run before validate(); publishing unmapped templates will fail with unknown_*_ref — treat that as the safety net, not the UX.
• File-based catalog avoids a migration entirely. If DB-backed templates are chosen instead: conductor's own migration chain is next free at 0006 (middleware/src/conductor/migrations/), and the shared chain at middleware/migrations/0008 — but the open MCP PR feat(mcp): MCP as a first-class capability across Orchestrator/Sub-Agent/Skill/Plugin + Control Center (epic #459) #464 claims 0008–0017 there, so coordinate to avoid collision.
• Cron triggers in templates are reconciled at publish via scheduleStore.reconcileOnClient (already wired in the POST / handler, atomic with the publish transaction) — instantiated templates with schedules become active as soon as they are published; the mapping form should surface that.
• omadia Facilitator — agent that moderates group chats to a defined outcome (+ Conductor JIT/ephemeral workflows & Core group primitives) #330 depends on this issue (stated in this issue's description); sequencing the manifest format before Facilitator work avoids rework.

Acceptance criteria:

• GET /api/v1/operator/conductors/templates returns the bundled catalog with slot declarations.
• Instantiating a template with a complete slot mapping creates an enabled, versioned workflow visible in GET / and runnable via POST /:slug/runs.
• Instantiating with a missing/invalid mapping returns 400 with the specific unknown_*_ref validation codes.
• Every bundled template passes conductorGraphSchema shape validation in CI.
• Web-ui /conductor offers template selection + slot mapping, and can hand the substituted graph to the designer for editing before publish.

---

Automated triage against main@1e5b24d; posted after adversarial fact-check review.

[Weegy]

Implementation plan: Conductor workflow templates (v1)

Plan of record for this issue, based on the triage above (re-verified against current main@527fff3), cross-platform research (n8n, Zapier, Make, Power Automate), and a read of the current conductor code. User-facing name: workflow templates — "Paperclip Process" stays an internal codename.

Design summary

A TemplateManifest (new shared type in @omadia/conductor-core) is a complete WorkflowGraph whose external references are replaced by slot:<kind>:<key> placeholder strings, plus catalog metadata (id, name, description, useCase, defaultSlug) and a slots declaration enumerating every placeholder with a human-readable label and help text. Placeholders appear only in the five ref fields: step.agentId, step.actionId, step.human.principal.ref (role principals), step.human.channel, trigger.eventId. The syntax is deliberately not {{...}}, which is reserved for the existing run-context prompt interpolation.

Instantiation is server-side and copy-not-reference (the universal model across all four researched platforms, and free with our immutable version store): substitute the operator's slot→entity mapping (pure function in conductor-core), validate with live KnownRefs built from the existing agent/action/role/event catalogs — stricter than POST /, which validates structurally only; deliberate, because a template instance must be runnable, not merely well-formed — then publish through the existing ConductorWorkflowStore.createOrPublish() with the existing atomic cron-schedule reconcile. The result is an ordinary versioned, editable workflow with no link back to the template.

Storage: file-based catalog, zero migrations. Manifests are JSON assets in middleware/src/conductor/templates/, loaded at wire time. If templates ever become DB-backed, the conductor subsystem's own migration chain is next free at 0006 (middleware/src/conductor/migrations/); the shared chain is not touched (its 0022 is claimed by open PR #476).

Triage open decisions, resolved:

1. v1 templates: expense-approval (approval chain with threshold/escalation), notify-and-escalate, weekly-report (cron), onboarding-checklist.
2. Naming: "workflow templates" in all user-facing copy.
3. omadia Facilitator — agent that moderates group chats to a defined outcome (+ Conductor JIT/ephemeral workflows & Core group primitives) #330 ephemeral instantiation: yes — POST /templates/:id/resolve returns the substituted, validated graph without persisting. Facilitator consumes GET /templates (full manifests) and resolve; persistent creation uses instantiate. One shared code path.

API surface (all under the auth-gated /api/v1/operator/conductors, registered before the /:slug catch-all)

• GET /templates → { templates: TemplateManifest[] } — full manifests including graph and slot declarations (machine-readable for omadia Facilitator — agent that moderates group chats to a defined outcome (+ Conductor JIT/ephemeral workflows & Core group primitives) #330).
• POST /templates/:id/resolve — body { mapping } → 200 { graph }. Errors: 404 conductor.template_not_found; 400 conductor.template_slot_mapping_incomplete with missing: [{ kind, key, label }]; 400 conductor.invalid_graph with the existing unknown_*_ref validation errors.
• POST /templates/:id/instantiate — body { slug, name?, description?, mapping, enable? } → 201 { workflow, version } (same shape as POST /). Additionally 409 conductor.slug_exists on slug collision — instantiation means "create new", never silently publish over an existing workflow. enable defaults to false.

Units (5, backend before frontend, one commit each)

B1 — conductor-core template contract. TemplateManifest/TemplateSlots/TemplateSlotMapping types in packages/conductor-core/src/types.ts; pure functions in new src/template.ts: extractSlotRefs (structural walk of the five ref fields, never string-replace over JSON), missingSlotMappings, applyTemplateSlots (deep-clone, field-targeted substitution), checkTemplateManifest (shape + structural validate + bidirectional slot coverage + duplicate keys). Exported via index.ts. Unit tests in test/template.test.ts incl. substitution round-trip passing validate(graph, knownRefs) and non-mutation.

B2 — bundled catalog + loader + build plumbing. Four manifest JSONs in middleware/src/conductor/templates/ (each 3–6 steps, respecting engine validation rules — deadline-bearing human steps carry fallbacks; cron values pass isValidCron). New templateCatalog.ts loader (dirname-relative like the conductor migrator; invalid files skipped with a loud log — the CI test is the hard gate). Build plumbing in the same commit: middleware/scripts/copy-build-assets.mjs ASSETS entry src/conductor/templates → dist/conductor/templates and matching Dockerfile COPY next to the conductor-migrations line. CI gate test middleware/test/conductorTemplateCatalog.test.ts: every bundled template passes checkTemplateManifest and a synthetic-mapping instantiability round-trip; cron validity; unique ids. No migration (would be conductor chain 0006 if ever needed).

B3 — routes + wiring + docs. The three routes in middleware/src/conductor/routes.ts (with the other static routes, before /:slug at line 291); wireConductor in middleware/src/conductor/index.ts constructs the catalog and a templateKnownRefs provider from getRegistry().list(), listActions(), roleStore.listRoles(), eventCatalog.list(). Instantiate publishes exactly like the POST / handler including onPublished schedule reconcile. Route tests in middleware/test/conductorTemplateRoutes.test.ts (node:test, stubbed deps) covering all error codes, the 409, the createOrPublish call shape, and a route-order regression. Docs: new endpoints documented in docs/middleware-agent-handoff.md.

F1 — API client + template gallery. web-ui/app/_lib/api.ts: local wire types + fetchConductorTemplates / resolveConductorTemplate / instantiateConductorTemplate beside the existing conductor block (CONDUCTOR_BASE). New TemplateGallery.tsx in app/conductor/_components/: card grid with name, description, useCase tag, "you will map: N roles · M agents · …" summary, and a cron badge for scheduled templates; "Use template" button. Rendered as a section on /conductor above the workflows list. Vitest tests in the new _components/__tests__/. Lume: state colors text/edge only, four button variants, no spinners (verb + dots), tokens not hex; i18n keys under the conductor namespace in both messages/en.json and messages/de.json (edited directly).

F2 — slot mapping form + flows. New TemplateInstantiateForm.tsx: slug/name fields (prefilled), one picker per declared slot grouped by kind — roles/agents/actions/events fed by the existing catalog fetchers, channels via the exported ChannelSelect from GuidedControls.tsx — each with the slot's label and help text; enable toggle (default off) with a persistent schedule-transparency notice for cron templates. Three actions: Create workflow (primary → instantiate, on success reload the workflow list), Open in designer (secondary → resolve, then the existing chat→canvas mechanism: setChatGraphRequest({ graph, nonce }) hydrates ConductorCanvas via its loadGraphRequest prop — no canvas changes), Cancel (ghost). Server errors map to inline field-level messages (missing slots highlighted; 409 on the slug field). Same Lume/i18n constraints as F1; Vitest tests for gating, error mapping, both flows, and the schedule notice.

Test plan

CI: conductor-core unit tests (b1); catalog gate proving every bundled template is shape-valid and instantiable (b2); route tests for all status codes and route order (b3); Vitest component tests (f1/f2). Gates: middleware typecheck+lint+test; web-ui lint+tsc --noEmit+vitest; build check that dist/conductor/templates/ exists after npm run build and the Dockerfile COPY is present. Manual E2E before PR: list templates via curl; instantiate expense-approval end-to-end (run starts, await reaches the mapped role); missing-slot 400; slug-collision 409; weekly-report schedule notice + no firing while disabled; open-in-designer hydration; both locales.

Risks

Route-order swallow by /:slug (regression-tested); forgotten build asset (both plumbing files in one commit); catalog drift (CI gate re-validates every run); strictness divergence vs manual publish (documented); schedule surprise (enable off by default + notice); slug-collision TOCTOU falls through to the idempotent upsert (accepted, commented); channel values unvalidated by validate() (UI select mitigates); TemplateManifest is a contract for #330 — later changes must be coordinated.

Out of scope for v1

• "Save as template" from an existing workflow — needs slot inference; v2 once the manifest format is proven.
• Hub/plugin-distributed templates — separate security review (plugin surface has no template capability today).
• DB-backed storage — not user data yet; avoids the migration chain entirely.
• Graph preview thumbnails, categories/search — not worth it at 4 templates; useCase field makes facets additive later.
• Builder-chat template awareness — resolve→designer already covers customization; keep v1 to one new concept.
• Instantiation telemetry, community submission/ratings — platform-growth machinery without a current need.
• Slot substitution inside prompt/message text — ref fields only in v1; text-level slots are an additive extension.
• Template versioning / update propagation to instances — copy-not-reference is the contract, consistent with every researched platform.

[Weegy]

Implementation report

The plan of record above is implemented and shipped as PR #479 (branch feat/issue-429-workflow-templates, 10 commits on base 527fff3).

Delivered, unit by unit (one commit each, stacked):

• b1 96ffc09 — conductor-core template contract: TemplateManifest/TemplateSlots/TemplateSlotMapping, pure extractSlotRefs / missingSlotMappings / applyTemplateSlots / checkTemplateManifest (structural walk of the five ref fields; {{ctx.*}} text interpolation untouched by construction).
• b2 2313d80 — four bundled templates (expense approval w/ escalation, notify-and-escalate, weekly report, onboarding checklist), wire-time catalog loader, build plumbing (copy-build-assets), CI gate re-validating every manifest against the engine.
• b3 cd5ede7 — operator routes GET /templates, POST /templates/:id/resolve (ephemeral, the omadia Facilitator — agent that moderates group chats to a defined outcome (+ Conductor JIT/ephemeral workflows & Core group primitives) #330 seam), POST /templates/:id/instantiate (publishes through createOrPublish with the atomic schedule reconcile); fail-clear error envelopes; registered before the /:slug catch-all with a route-order regression test.
• f1 41344fa — template gallery on /conductor (card grid, slot summary, text/edge schedule badge; Lume-conform, i18n en+de).
• f2 0c32131 — guided slot-mapping form: one upfront form per template, pickers per slot kind fed by the live catalogs, Create-workflow and Open-in-designer paths (resolve → canvas hydration), inline server-error mapping, enable default OFF with schedule notice.

Review process (3 rounds, dual-gate): every unit passed its own verify gate; the whole branch then went through an adversarial same-family review plus a cross-vendor (codex) review. Round 1/2 fixups: designer handoff honoring the form's enable=OFF choice (70a3047), Dockerfile COPY removal (476683d), restored #453 changelog heading (cfc9a6e). Round 3 (both cross-vendor findings, re-approved by both reviewers afterwards):

• 8f25c15 — instantiate is now atomically create-new: expectNew flips the store's conflict clause to DO NOTHING, zero rows → rollback → WorkflowSlugExistsError → 409 conductor.slug_exists; the TOCTOU the plan had accepted is actually fixed, and POST / / canvas save keep upsert semantics (regression-tested).
• a0a3df3 — template metadata is LocalizedText in the manifest (en required, de shipped for all four bundled templates, CI-gated parity); web-ui resolves locale with en fallback; GET /templates still returns unresolved manifests, keeping the omadia Facilitator — agent that moderates group chats to a defined outcome (+ Conductor JIT/ephemeral workflows & Core group primitives) #330 contract machine-readable.

Verification: middleware typecheck (27 workspace packages) + 4037 tests green; web-ui tsc --noEmit + 235 tests + i18n parity green; all four manifests pass checkTemplateManifest plus a synthetic full resolve.

Deviations from the plan: (1) the slug-collision TOCTOU listed as accepted risk is fixed properly instead; (2) manifest-borne localization was added (the plan under-specified locale handling for template metadata); (3) no Dockerfile change shipped — copy-build-assets alone covers the deploy artifact.

Out-of-scope v2 themes are now tracked as required scope in #478.