Conductor workflow templates v2: save-as-template, distribution, DB storage, discovery, chat awareness, telemetry, text slots, versioning

[Weegy]

Context

Issue #429 ships workflow templates v1 (file-based catalog, slot parameterization, guided instantiation -- see the plan of record posted there). The v1 plan deliberately deferred eight themes. This issue makes them the v2 scope: they are REQUIRED, not nice-to-haves.

Scope (all items required)

1. "Save as template" from an existing workflow -- slot inference over a concrete graph (agents, actions, roles, events, channels become declared slots); authoring UX in the designer.
2. Hub/plugin-distributed templates -- templates as a distributable artifact (hub.omadia.ai / plugin packages), WITH the security review the v1 plan called out: the plugin surface has no template capability today, so the capability model, validation on import, and trust boundaries must be designed, not assumed.
3. DB-backed template storage -- move from bundled JSON assets to persisted, user-authored templates (conductor migration chain; next free number must be re-verified at build time).
4. Gallery discovery: categories/search + graph preview -- facets over useCase, text search, and a graph preview/thumbnail per template.
5. Builder-chat template awareness -- the Conductor builder agent can propose, fetch, and prefill templates in chat.
6. Instantiation telemetry + community submission/ratings -- usage signals per template; a submission path for user templates (review gate before they become shared).
7. Slot substitution inside prompt/message text -- extend slot:<kind>:<key> (or a text-level variant) into prompt/message strings, without colliding with the existing {{...}} run-context interpolation.
8. Template versioning / update propagation -- versioned manifests and an explicit, opt-in update path for instances (v1 contract is copy-not-reference; any propagation must be deliberate and reviewable).

Constraints

• Builds on the v1 groundwork from Conductor: define reusable use-case workflows ("Paperclip Process") #429 (TemplateManifest contract in @omadia/conductor-core, catalog loader, routes, gallery UI). Until Conductor: define reusable use-case workflows ("Paperclip Process") #429's PR is merged, work stacks on its branch.
• omadia Facilitator — agent that moderates group chats to a defined outcome (+ Conductor JIT/ephemeral workflows & Core group primitives) #330 (Facilitator) consumes GET /templates + resolve; the v2 changes must keep that contract stable or version it explicitly.
• Lume design-system invariants and i18n parity (en/de) apply to all UI work.

[Weegy]

Workflow templates v2 — plan of record (revision 2)

Builds on the #429 groundwork; work stacks on feat/issue-429-workflow-templates (PR #479) until it merges. All eight scope items are covered. Two review findings from the previous revision are fixed (reviewer-reachable review gate; UI path to publish version > 1); the rest carries over unchanged.

Design summary

Adapt, not clone. The researched platforms (n8n, Zapier, Make, Power Automate) converge on: copy-not-reference instantiation, per-field slot annotation instead of a template DSL, a review gate between private and shared distribution, and credential-free manifests. Where their designs conflict with our core, our core wins:

• One artifact, one validation path. TemplateManifest stays the single canonical unit across bundled JSON, DB rows, and plugin-borne templates — no n8n-style parallel serving protocol, no split browse/import payloads. Every source passes checkTemplateManifest + validate(graph, knownRefs).
• Declared slots stay mandatory. No raw-graph export path; distributed manifests are validated in strict mode where any undeclared concrete ref (agent/action/role/event/channel) is rejected — those are confusion/exfiltration vectors pointing at install-local entities.
• Copy-not-reference stays the contract. Versioned immutable manifests; instances stamped with {templateId, version} at creation; updates surface as an opt-in hint plus deliberate re-instantiation. No silent propagation.
• Text slots use the existing slot: prefix family: slot:text:<key> inside designated text fields (step.prompt, human-step message text), disjoint from {{...}} run-context interpolation (renderTemplate). Substitution is a structural walk over designated fields, never string-replace over serialized JSON.
• Review gate is Make's team-template shape (private → pending → shared, one approval action), not a certification pipeline. Telemetry is an anonymous per-template instantiation counter (modeled on 0009_mcp_call_log.sql); no ratings, no per-user tracking.
• Hub/plugin distribution rides the plugin surface: plugins declare manifests under permissions.templates (package-relative JSON paths); install-time validation (strict manifest check, conductorGraphSchema, isValidCron on cron values, symlink-unwrapped path confinement, plugin:<pluginId>:<name> id namespacing) registers them as read-only catalog entries, removed on uninstall. Templates are data, never code — no runtime ctx.templates capability, nothing executed. This is the designed capability model the issue requires; the trust boundary is recorded in docs/security-architecture.md.

Storage. Conductor's own migration chain (middleware/src/conductor/migrations/, _conductor_migrations bookkeeping). Next free number: 0006 (0001–0005 present; #479 shipped no migration) — re-verified at build time against open PRs (the top-level chain's 0022 is claimed by PR #476 and is not used here). One migration 0006_templates.sql: conductor_templates (id, created_by, status private|pending|shared — no CHECK on growable enums, latest_version, reviewed_by, timestamps), conductor_template_versions (template_id, version, manifest JSONB, PK(template_id, version) — immutable, mirroring the workflow version store), conductor_template_instantiations (append-only, denormalized template_name, no payloads), plus template_id/template_version provenance columns on conductor_workflows.

Visibility rule (revised — fixes the unreachable review gate). Bundled and plugin templates are visible to all operators. A user template is visible to a viewer iff status = 'shared', OR createdBy = viewer, OR status = 'pending' — pending templates are visible to every authenticated operator, because the operator API surface is single-tier and every operator is a potential reviewer. GET /templates/:id applies exactly the same rule, and the approve/reject routes resolve templates through that same viewer-scoped get — a non-author reviewer can list, open, and approve a pending template. Write authorization is separate: PUT/DELETE/submit are author-only; approve/reject are open to any operator, with reviewed_by recorded. Self-approval is permitted (single-operator installs must not deadlock); separation of duties is explicitly deferred.

Version publishing in the UI (revised — fixes the missing v>1 path). The save-as-template dialog is owner-aware: if the chosen template id already exists as a user template owned by the viewer, the primary action switches from "Publish template" (POST) to "Publish as v{latestVersion+1}" wired to PUT /templates/:id; a foreign/bundled/plugin id shows an inline "id taken" error; a 409 race on POST re-checks ownership and offers the switch. This makes publisher maintenance (fix reaches future instantiations) possible without raw API calls.

API surface (all additive over v1; GET /templates + resolve stay compatible for #330)

• GET /templates → { templates: TemplateSummary[] } where TemplateSummary = TemplateManifest & { source: 'bundled'|'user'|'plugin', status?, createdBy?, version, latestVersion, instantiationCount, updatedAt? }
• GET /templates/:id (new, visibility-gated) · GET /templates/:id/versions
• POST /templates (create private, 409 conductor.template_id_exists, 400 conductor.template_invalid) · PUT /templates/:id (author-only, appends version n+1) · DELETE /templates/:id (author-only, user-source only)
• POST /templates/:id/submit (author, private→pending) · POST /templates/:id/approve / reject (any operator, pending→shared/private, records reviewed_by)
• POST /workflows/:slug/save-as-template → inferred draft manifest (not persisted; UI edits then POST/PUT)
• POST /templates/:id/resolve / instantiate: optional version (default latest); TemplateSlotMapping gains text?: Record<string,string>; instantiate stamps provenance + appends a telemetry row
• Workflow list/detail gain template?: { id, version, latestVersion, updateAvailable }
• POST /builder/turn response gains templateProposals?: [{ templateId, version, reason, prefill }] (≤3, server-filtered to viewer-visible ids, prefill validated against live KnownRefs)

All new routes register before the /:slug catch-all in middleware/src/conductor/routes.ts; viewer identity is req.session?.sub ?? 'operator'.

Units (backend before frontend; one coherent commit each)

B1 — conductor-core contract v2 (packages/conductor-core): version on TemplateManifest (absent = 1); slots.text: TemplateTextSlot[] + slot:text:<key> tokens in designated text fields; applyTemplateSlots/missingSlotMappings extended (kind:'text' entries additive); checkTemplateManifest validates text-slot declaration/usage and gains { strict } mode rejecting undeclared concrete refs; new inferTemplateManifest(graph, opts) reversing the extractSlotRefs walk (one declared slot per distinct concrete ref, slot: placeholders preserved, round-trip tested). Pure functions; node tests in packages/conductor-core/test/template.test.ts.

B2 — DB template store + composite catalog + CRUD/versioning routes + migration 0006_templates.sql: templateStore.ts (create/addVersion/get/list/delete/setStatus/listVersions/getVersion/recordInstantiation/instantiationCounts); composite catalog (bundled + user + plugin) behind a viewer-scoped { list(viewer), get(id, viewer) }; the revised visibility rule above; routes GET/POST/PUT/DELETE /templates(/:id), GET /templates/:id/versions, versioned resolve/instantiate with text mappings, provenance stamp + telemetry. Tests include the explicit reviewer-reachability cases: pending template of operator A appears in operator B's list and get; private stays hidden; PUT bumps to v2 and non-author PUT is 403; GET /templates wire-shape additivity as a contract test.

B3 — authoring + review gate + plugin templates + update hint: POST /workflows/:slug/save-as-template (inference draft from the published workflow version); submit/approve/reject state machine (approve/reject open to any operator via the viewer-scoped catalog get — tested with a non-author approver); workflow endpoints report template.updateAvailable; plugin capability: permissions.templates parsing in manifestLoader.ts, install-time strict gate in installService.ts, read-only registration/unregistration on the composite catalog, security design recorded in docs/security-architecture.md. Tests: state machine incl. non-author approve, plugin gate rejection/acceptance/uninstall/path-traversal.

B4 — builder-chat template awareness: builderAgent.ts gets a capped, viewer-scoped catalog digest in its system prompt and may emit a templateProposals block; /builder/turn returns it after server-side filtering (unknown ids dropped, prefill entries failing KnownRefs stripped). Chat proposes and prefills; instantiation stays on the existing form/routes. Tests in conductorBuilder.test.ts incl. no-proposal regression.

F1 — save-as-template dialog (SaveAsTemplateDialog.tsx, designer entry point on the conductor page): metadata (en+de LocalizedText), inferred ref-slot editor, manual text-slot editor with token hint; owner-aware primary action — fresh id → POST; owned existing id → "Publish as v{n+1}" via PUT (with 409-race recovery into PUT mode); foreign id → inline error. Vitest cases include the required "owned id switches to Publish as v2 and calls PUT". API client: saveWorkflowAsTemplate, createConductorTemplate, updateConductorTemplate, fetchConductorTemplate.

F2 — gallery v2 (TemplateGallery.tsx): facets All / Bundled / My templates / Shared / Plugins / Pending review (the reviewer queue — populated for every operator thanks to the revised visibility rule; count badge; Approve/Reject rendered directly on pending cards for all operators, not inside an own-templates manage menu); useCase chips + substring search; source/status badges (text + edge color only), v{n} tag, instantiation count; author manage menu (Submit for review, Delete). Vitest includes: pending template by another operator is listed with working Approve/Reject.

F3 — instantiate form v2 + preview + update flow (TemplateInstantiateForm.tsx, new TemplatePreview.tsx): required-fill text-slot inputs mapped to server kind:'text' errors; version display + explicit-version instantiation; collapsed-by-default read-only Cytoscape preview rendered from the manifest graph (no stored thumbnails); workflow update hint "Template updated (v{n} → v{m})" with "Re-instantiate from v{m}" (new workflow; old untouched). Accepts a prefill prop for F4.

F4 — chat proposal cards (ConductorChatPane.tsx): up to 3 proposal cards per turn (name, version, reason, slot-coverage line) with a single "Use template" action handing off to F3's form with the prefill mapping, via the same page-state pattern as the existing chat→canvas hand-off. No auto-instantiation.

All frontend units: Lume invariants (state colors text/edge only, four button variants, no spinners — verb + dots, theme.css tokens not hex) and en/de i18n parity (keys in web-ui/messages/en.json + de.json directly; CI-gated).

Test plan

Per-unit gates: middleware node:test, web-ui Vitest, next build + tsc --noEmit + lint + i18n parity; migration 0006 idempotency (apply twice, and on a v1-era DB).

End-to-end (two operator sessions A and B):

1. A: publish workflow → Save as template → inferred slots → edit labels, add text slot → Publish (v1, private, under "My templates").
2. A: Submit for review → pending. B (non-author): "Pending review" facet lists it, GET succeeds, Approve → shared for everyone, reviewed_by = B.
3. B: instantiate (ref pickers + required text slot + graph preview) → workflow created with {templateId, v1} provenance; usage count increments.
4. A: improve the workflow → Save as template with the same id → dialog offers "Publish as v2" (PUT) → gallery shows v2 → B's step-3 workflow shows the update hint → B re-instantiates from v2 deliberately; the old instance is untouched.
5. Chat: "I need a vacation-request approval flow" → proposal card → prefilled form → Create.
6. Plugin: install with one valid + one strict-mode-violating manifest → valid appears as read-only source "plugin", invalid rejects at install; uninstall removes it.
7. Contract: v1-era GET /templates / resolve responses unchanged modulo additive fields (omadia Facilitator — agent that moderates group chats to a defined outcome (+ Conductor JIT/ephemeral workflows & Core group primitives) #330).

Risks

• Migration number: 0006 on the conductor chain, re-verified at build time against open PRs (top-level 0022 belongs to PR feat(scripts): spec-driven wave workflows + W0 unit manifest (epic #470) #476; a numbering collision has bitten before).
• Stacked base: PR feat(conductor): workflow-template library — bundled catalog, slot mapping, guided instantiation (#429) #479 is open; rebase promptly on merge; B1 first (isolated package) limits conflict surface.
• omadia Facilitator — agent that moderates group chats to a defined outcome (+ Conductor JIT/ephemeral workflows & Core group primitives) #330 drift: additive-only wire changes, locked by a contract test.
• No separation of duties on review: any operator (incl. the author) can approve; reviewed_by audits it; role-gated review deferred. Accepted for single-tier operator installs.
• PUT on a shared template serves the new version without re-review — deliberate (publisher-maintenance path), acceptable inside one install's operator team; revisit before hub submission opens.
• Text-slot grammar is disjoint from {{...}}; a test pins a prompt containing both. Literal "slot:text:x" strings in prompts are a documented limitation.
• Preview cost: one collapsed-by-default Cytoscape instance per opened form; degrade to a summary if profiling shows jank.
• Plugin gate is security-sensitive: strict validation, path confinement, id namespacing — tested, and documented in docs/security-architecture.md per CONTRIBUTING.

Out of scope (deferred)

• Star ratings / community scoring — noise at team-sized n; instantiation counts are the signal.
• hub.omadia.ai standalone template artifacts / remote registry — plugin packaging covers distribution; no parallel serving protocol (the N8N_TEMPLATES_HOST anti-pattern).
• Diff/merge update propagation to instances — hint + deliberate re-instantiation only; copy-not-reference stands. Publisher fixes reach future instantiations via F1's UI version-publish path.
• Separation of duties on review — deadlocks single-operator installs today; ships when operator roles exist.
• Automatic text-slot inference — authors declare text slots manually; inferring install-specific prose is guesswork.
• Full-text search infra — substring match suffices below ~50 templates.
• Funnel/per-user telemetry — anonymous counters only, per the privacy positioning.
• Stored thumbnails — previews render live from the manifest.
• Raw JSON template upload in the UI — would need the same strict gate; defer until requested.
• Auto-instantiation from chat — creation stays a deliberate form action.

[Weegy]

Implementation report

The v2 plan of record above is implemented and shipped as PR #484 (branch feat/issue-478-templates-v2, 11 commits, stacked on feat/issue-429-workflow-templates / PR #479 — merge #479 first).

Delivered, unit by unit (stacked, one commit each):

• b1 e5320e9 — conductor-core manifest v2: integer version (absent = 1), slot:text:<key> text slots in the two designated text fields (structural walk, disjoint from {{...}} interpolation), inferTemplateManifest (slot inference over a concrete graph), strict import gate for distributed manifests, malformed-input hardening.
• b2 c42a868 — DB template store (conductor migration 0006_templates.sql: templates, immutable version rows, append-only anonymous instantiation telemetry, provenance columns), composite catalog (bundled + DB + plugin), CRUD + versioning routes, viewer-scoped visibility with reviewer-reachable pending.
• b3 287a3c1 — save-as-template inference route, private → pending → shared review gate (submit/approve/reject, reviewer recorded, self-approval allowed so single-operator installs cannot deadlock), plugin-borne templates through the existing package validation (strict gate, read-only, removed on uninstall), template-update hint data.
• b4 3e3bce3 — builder-chat template awareness: catalog-aware proposals (templateProposals, additive).
• f1 879fe07 — save-as-template dialog: inference draft editor (metadata + slot labels en/de), text-slot declaration + token placement, owner-aware "Publish as v{n+1}".
• f2 05d490c — gallery v2: source facets incl. pending-review queue, text search, manage/approve/reject actions, version + usage metadata.
• f3 a572b7e — instantiate form v2: text-slot inputs with defaults, read-only graph preview, version pinning, "template updated" hint with deliberate re-instantiation.
• f4 34bafd7 — chat template proposal cards (max 3, hand off to the prefilled instantiate form; chat never auto-instantiates).
• plus c88c181 (step-kind colors → Lume tokens, 500-line splits).

Review process (3 dual-gate rounds): codex round-2 findings (malformed POST /templates 500'd; text-slot authoring undeliverable without token placement UI; whitespace hygiene) fixed in 8538199; codex round-3 finding (owner publish path dead-ended while getAuthMe pending) fixed in c56e837 with a live-derived pending publish mode. Final state: same-family adversarial review AND codex both prReady:true (each verified the fixes by execution/inspection; git diff --check clean).

Verification: middleware typecheck + 4129 tests green; conductor-core 96 tests; web-ui tsc --noEmit + 274 tests (40 files) + i18n parity (en/de, 2841 keys) green.

Adaptation decisions (per plan): plugin-borne distribution through the existing packaging/scan surface instead of a separate template-registry protocol; anonymous instantiation counts instead of ratings; copy-not-reference kept — updates surface as an opt-in hint + re-instantiation, never silent propagation. Deferred scope is listed in the plan comment above.

All eight required themes of this issue are delivered.