Skip to content

[Program] Elsa 3 integration ecosystem: monorepo consolidation, independent releases, and researched connector catalog #8194

Open
Feature
1 of 10 issues completed
#8657
Open
[Program] Elsa 3 integration ecosystem: monorepo consolidation, independent releases, and researched connector catalog#8194
Feature
1 of 10 issues completed
#8657
Labels
elsa 3This issue is specific to Elsa 3epicAn 'issue of issues', representing a large or conceptual enhancementprio highIs on the roadmap for the near-futuretriaged

Description

@sfmskywalker
sfmskywalker
opened on Sep 23, 2026

Owner priority change — 2026-10-08: structural cutover first

The owner has redirected delivery toward completing the repository consolidation and cutover as soon as possible. Production-grade Slack delivery is no longer a prerequisite for structural consolidation or source-repository archival. The earlier broader integration roadmap below is retained as future work, not an obligation to finish every connector capability before cutover.

Current completion boundary: implement the approved root core/, extensions/, studio/ layout (#8667); preserve all accepted source/history and latest release inclusion; verify the relocated solution, product builds/tests, packages and relevant developer workflows; reconcile contributor issues/PRs and contribution ownership; establish working release/publisher ownership and retire competing source workflows; then perform and verify the source-repository read-only/archival cutover with accurate documentation and recovery instructions. Existing concrete operational/publisher decisions remain explicit; a directory move alone is not cutover.

Pilot boundary: reuse existing accepted module/package evidence where sufficient. If an additional structural example is needed, timebox a simple deterministic sample module to roughly one hour of implementation, with no external credentials, live Slack setup, durable message delivery or production connector platform expansion. Required verification remains explicit; the timebox is not a claim that all CI finishes within one hour.

Slack disposition: #8666 is deferred, incomplete, with its branch and worktree changes preserved. Stop new Slack implementation and extra verification runs. Source and Admission evidence already accepted remains retained, but unmerged Socket code and uncompiled new discard scenarios must not enter the structural migration merely to save prior effort. Remaining Slack provider/fault/process/six-cell/outbound/live-operational scope stays in its existing issues as post-cutover work, not silently accepted or deleted.

Current delivery state: structural #8667, Core contribution register #8670 and source notices #8676 are accepted and closed. Studio #1128 and Extensions #288 are merged with reviewed/tested tree equality and verified nonpublishing merge-event readbacks. No implementation leaf is active while the next maintenance/npm ownership and archival sequencing choices remain pending. Preserve deferred Socket work.

After cutover: investigate an alternative React + React Flow Elsa Studio alongside the existing Blazor implementation. First inventory the complete current Blazor feature surface. Separately inspect Elsa Foundation Studio as an architectural reference for modular projects serving React modules and React Flow integration. Reuse suitable infrastructure patterns, not Elsa 4 product functionality or an assumed Elsa 4 migration. This discovery begins after cutover and does not block it.

Delivery milestones — owner clarification, 2026-10-08

The owner clarified that the first useful deliverable is structural repository consolidation. The lead will close that bounded milestone separately from the complete program and will surface any requirement that materially expands its outcome or effort. The full goal remains open until its required later outcomes are verified.

  1. Consolidated code on main ([Task] Relocate Core, Extensions and Studio into separate root product directories #8667): preserve source/history and 3.8.x/3.9.0 inclusion, establish root core/, extensions/, studio/ boundaries, update affected references/tooling, and verify builds/tests and developer paths. This milestone is accepted and [Task] Relocate Core, Extensions and Studio into separate root product directories #8667 is closed: Promote consolidated product directories and package surfaces to main #8669 and documentation correction Fix operator command paths after documentation relocation #8674 are merged, with exact-head review/CI and final main-tree/source verification. No Slack feature, full release certification, contribution backlog implementation or live publisher/archival operation is a prerequisite to closing this structural milestone.
  2. Source-repository handoff and retirement ([Task] Complete contribution routing and source issue/PR handoff for repository cutover #8670, [Feature] F2.4: Migration of contribution/release documentation and source ownership #8216, [Feature] F3.4: Artifact provenance, release notes, and recovery procedures #8220): record contributor dispositions and Core routing, establish the selected maintenance/npm publishing replacement, contain competing source publisher authority, then obtain any required concrete operational approval and verify read-only archival. Existing maintenance/release decisions and publication acceptance remain explicit; milestone 1 does not waive them. Keep one active Task and adopt existing matching work.
  3. React Studio discovery after cutover: inventory Blazor Studio functionality and review Foundation Studio's modular React/React Flow infrastructure, producing an architecture proposal before implementation. This is subsequent work and does not block consolidation or archival.

Each next delivery slice will state its usable outcome, done criteria, exclusions and estimate/checkpoint before implementation. Related improvements remain separate unless needed to meet that slice's concrete outcome.

Current delivery checkpoint — structural and contribution records accepted

Structural milestone #8667 remains accepted at 658a9da06b2b77417f5c07eee00fff68478ba15c. Contribution handoff #8670 is now accepted through PR #8675, merged on 2026-10-08 at main 8802a883d6069d689d110008c904787b3b691042. Exact-head independent review, Greptile 5/5 and CI passed. The final clean main tree equals the reviewed/tested candidate and contains only the four reviewed documentation/data changes relative to the structural baseline. The register preserves all 205 issues and 31 PRs from its dated snapshots; it does not implement those proposals or freeze future source activity.

Source notices #8676 are accepted through Studio #1128 at dab6831ac11aa0d2caef93f39c8e705d9459bbaa and Extensions #288 at 2611ee6d9b6fb5e50f4c0b3fbc9cc15b355b4751. Exact-head independent review and required CI passed. Actual merge parents/tree/subject and source main refs were read back; bounded exact-merge and Packages-workflow queries showed no Packages runs. Dynamic analysis is separate. Extensions Copilot did not deliver an advisory review; it is not claimed as passed. The notices preserve existing 3.8/3.9 maintenance procedures and original contributor identities.

No delivery Task is currently active. The next maintenance-support/home, npm ownership and archival-versus-full-release sequencing decisions remain pending. The concrete maintenance-home option and estimate preserves original release layouts in Core: approximately 1½–2 engineering days plus approval/access delays, subject to historical-build results. This option has not been selected or implemented. Keep the next slice bounded and estimated before activation.

A verified maintenance/npm publisher replacement, retirement of competing source authority and actual read-only archival remain open under #8216/#8220. No source settings, package publication or archival occurred. Retained maintenance continues until an explicitly approved replacement is verified or an explicit support-policy decision changes it. The full program is not complete. Slack remains deferred and React discovery starts after cutover.

Historical structural follow-up — 2026-10-08

PR #8669 is merged and the three-product layout is on main. Task #8667 remains active for the final automatic main Packages result and documentation correction PR #8674: 18 stale copyable command paths across nine Markdown files. The correction has exact-head independent APPROVE + HIGH and Greptile 5/5 at 875b69259609fb8ea36a6284e44ca7367ecc281f; CI and merge remain pending. The task's developer-command criterion is explicitly reopened. Main Docker, publication guards and CodeQL/Code Quality are successful; Docker/guard original logs have been verified by the lead. No publication or archival occurred. #8670 remains the queued contribution-handoff task.

Structural main checkpoint — 2026-10-08 17:03 UTC

  • PR #8669 is merged as 598f8fef28cdca503d20373c2e5cd09e129024d2. The actual merge tree equals the reviewed candidate, with both expected parents and baseline/pure-move history preserved. Exact-head independent review, Greptile 5/5 and all premerge CI passed; lead acceptance and original-artifact scope.
  • The clean actual-main checkout has root sibling core/, extensions/, and studio/ product trees. All 10,910 baseline destinations/blobs/modes are accounted for; solution/filter and ADR checks pass. Fresh all-repository advertisements and release pagination verify all 35 selected 3.8.x/3.9.0 refs and source main tips are included. Latest and newest published release remain 3.9.0 in Core, Studio and Extensions. Final-main source audit SHA-256: 8089d8643515f3a3e655b2ed6c5dbf4a0fa8a789c5a5fe0eac72748070243a4f.
  • [Task] Relocate Core, Extensions and Studio into separate root product directories #8667 remains the sole active Task while actual main-push Packages, Docker, publication guards and CodeQL/Code Quality finish. Candidate receipts remain bound to b117; identical tracked trees do not relabel them as main-SHA artifacts. The shared integration branch was restored after GitHub deleted it; its duplicate Packages run was cancelled/requested cancelled, preserving the actual main run.
  • Next Task [Task] Complete contribution routing and source issue/PR handoff for repository cutover #8670 is Todo / Not Ready / Verification Pending under [Feature] F2.4: Migration of contribution/release documentation and source ownership #8216, natively blocked by [Task] Relocate Core, Extensions and Studio into separate root product directories #8667. It owns durable contribution routing and the source issue/PR handoff. All 31 source PRs have bounded technical dispositions, and the 205-issue register preserves original identities. The 13 issues with Core links reference related work, not confirmed migration destinations. No requirement to implement every unaccepted proposal was added.
  • No package publication, deployment or source archival occurred. Maintenance support, npm ownership and publisher/operational cutover decisions remain explicit. Slack stays deferred with WIP preserved; React Studio discovery remains after cutover.

Summary

Build a sustainable integration ecosystem for Elsa 3: consolidate Core, all Extensions, and Elsa Studio into elsa-workflows/elsa-core, enable independently released connectors, provide shared connector infrastructure, and deliver a researched, prioritized catalog of useful integrations.

Elsa 3 is the implementation target. Preserve a practical path to Elsa 4 by separating provider-facing logic from runtime adapters where useful; do not block Elsa 3 delivery on a universal cross-version abstraction.

This issue is the program charter and execution plan. The initial breakdown has been created: 10 epics, 40 features, 6 stories and 12 tasks. E/F/S/T identifiers are planning IDs mapped to actual issues in the execution index below. Research and implementation are in progress; the latest execution checkpoint is below.

Superseded delivery checkpoint — 2026-10-08, before the owner priority change

Problem

Elsa needs both foundational engine/platform modules and a broad set of business integrations. Developers should be able to work on both in one solution, using solution filters for focused development and debugging. A connector release must not require republishing unrelated connectors or core.

Shared cloud users cannot upload arbitrary NuGet packages. They therefore depend on the supplied capabilities, connection experience, and catalog to build complete business processes. Adding API-call activities alone is insufficient: authentication, trigger lifecycle, tenant isolation, data handling, compatibility, and maintenance must be reusable platform capabilities.

Outcomes

  1. Core, Extensions and Studio are developed in one repository under root siblings core/, extensions/ and studio/, each owning its src/, test/ and docs/, with a shared root solution and focused solution filters.
  2. Connectors have explicit, independent release units and compatibility declarations.
  3. Source-based development and published-package compatibility are both verified.
  4. Shared infrastructure handles connections, authentication, actions, triggers, files, resilience, and diagnostics.
  5. Shared cloud exposes a curated catalog with enforced tenant and capability boundaries.
  6. Evidence-based research prioritizes complete customer workflows rather than connector counts.
  7. A pilot cohort validates the architecture and provides reusable implementation and contribution patterns.
  8. The program is decomposed into Program → Epic → Feature → Story → Task issues, with dependencies and acceptance criteria.

Scope and constraints

  • Canonical repository: elsa-workflows/elsa-core. Consolidate all Elsa Extensions and Elsa Studio here, including persistence, scheduling, runtime, expression and integration modules, paired backend/Blazor modules, tests, samples, build assets and relevant documentation. Preserve source history and account for active issues/PRs through an explicit migration ledger. Create new program work here.
  • Reuse or link existing issues and implementations instead of duplicating them.
  • Preserve existing package IDs, activity identities, serialized contracts, and consumer entry points unless an explicit migration is provided.
  • Initially keep engine packages and tightly coupled platform packages in their existing coordinated release groups. Independent connector releases are the first objective.
  • Package release, host deployment, and workflow activity/schema versioning are separate concerns.
  • Shared hosts initially use a curated, pinned connector set. Arbitrary conflicting connector/dependency versions within one process are out of scope.
  • Repository consolidation does not automatically consolidate unrelated commercial products.
  • Shared cloud, dedicated deployments, and self-hosting need explicit capability profiles.
  • Public issues and artifacts must not contain credentials, private customer correspondence, or confidential demand evidence. Record sanitized findings.
  • No schedule, staffing allocation, or demand ranking is established by this charter. Estimate after the inventory and pilots reduce uncertainty.

Studio inclusion and release boundaries

Include Elsa Studio in the monorepo so backend and Blazor functionality can be developed, debugged, reviewed and tested together. Audit paired modules already in Extensions. Provide focused solutions/filters and sample hosts spanning both sides.

Source co-location does not imply a single release train. Tightly coupled server/Studio module pairs may share a release unit; the Studio shell and unrelated packages retain explicit independent release policies. Add backend API/protocol and package compatibility checks. Preserve existing package identities and source ownership during migration.

On 2026-10-08 the product owner explicitly approved separate root product directories as the required final layout. This supersedes the earlier choice to retain the imported src/extensions/ and src/studio/ locations. Earlier import and history-preservation acceptance remains valid; completing the program now also requires the focused relocation and fresh verification below.

Target architecture

Layer Examples Initial release policy
Engine Workflow model, execution, runtime contracts Coordinated core releases
Platform Persistence, expressions, scheduling, HTTP infrastructure Retain compatibility groupings until audited
Connector infrastructure Connections, authentication, trigger lifecycle, execution support Small, explicitly versioned shared release units
Connectors OneDrive, Moneybird, Slack and subsequent providers Independent connector or tightly coupled family releases
Host composition Curated cloud/self-hosted bundles Pinned and tested deployment manifests

Required final directory structure, approved by the product owner on 2026-10-08:

elsa-core/
  core/
    src/       # apps, clients, common, modules
    test/
    docs/
  extensions/
    src/       # existing extension families
    test/
    docs/
  studio/
    src/       # framework, modules, hosts, bundles, wrappers
    test/
    docs/
  docs/        # cross-product architecture and operations
  build/
  scripts/
  .github/
  Elsa.sln
  Directory.Build.props
  Directory.Packages.props

Each product owns its source, tests, documentation and product-specific build settings. Keep shared build/release orchestration, the root solution, common tooling and central dependency policy at the root. Place product-specific samples and assets with their owning product; explicitly classify cross-product assets. Core/Studio contain deployable applications; Extensions chiefly contains consumable packages. Directory boundaries do not change existing package IDs, activity/API identities, dependency direction or the accepted initial 3.10 lockstep release and later independent-release requirements.

Deliver this as a focused history-preserving migration with an explicit old-to-new path map. Update project/solution/filter paths, MSBuild imports and inheritance, source/package references, CI selection and caches, Docker contexts, scripts, tests, docs, catalog/release/source-provenance tooling and ownership metadata. Verify all three product development/build/test entry points and the integrated solution from the relocated tree. Refresh source-bound package, compatibility and release evidence after relocation; pre-move evidence is historical and cannot accept the new source head. Track implementation in #8667 under reopened architecture feature #8213. Final program closeout requires this layout merged into main and verified, not merely recorded in an ADR.

Folders organize source; release manifests define release boundaries. Catalog categories may overlap.

Each connector owns its projects, release metadata, changelog, documentation, tests, and examples. Shared provider code must have a justified boundary and dependency graph; do not create a mandatory dependency on every service from the same vendor.

Hierarchy and issue conventions

Use native GitHub parent/sub-issue relationships where supported. Record semantic level with existing repository conventions, issue types where available, or labels/title prefixes where needed. Verify organization capabilities before inventing new types or labels.

Level Purpose Required content
Program Outcomes and overall scope Charter, epic index, milestones, risks, decisions
Epic Independently assessable workstream Outcome, features, dependencies, exit criteria
Feature Coherent capability Consumer behavior, scope, compatibility, stories
Story Verifiable user/operator/developer outcome Scenario and acceptance criteria
Task Concrete implementation or research work Inputs, deliverable, verification, blockers

Use all five levels when meaningful; avoid artificial one-child chains. If native hierarchy cannot be configured, use explicit Parent and Children links and state that native relationships remain pending.

Every executable leaf must identify its parent, deliverable, in/out of scope, acceptance criteria, verification method, dependencies, and relevant sources/code. Assign ownership and estimates through normal triage; do not invent assignees. Distinguish implementation dependencies from containment.

Create all epics and their initial feature issues first. Decompose near-term research, audit, and release-proof work fully. Decompose later stories/tasks after their prerequisite findings, avoiding a large speculative task backlog.

Epic plan

E0 — Program initialization and current-state audit

Features

  • F0.1: Establish the issue hierarchy, reporting conventions, and decision log.
  • F0.2: Inventory core/extensions projects, packages, dependencies, and existing integrations.
  • F0.3: Map existing issues, release workflows, host composition, and Studio extension points.
  • F0.4: Audit the existing Elsa Secrets module, its provider contracts, storage/read/write capabilities, tenant scoping, authorization, and existing consumers. Record verified capabilities and gaps for integration credentials.

Initial stories/tasks

  • As a maintainer, I can find each release unit and its owner, consumers, dependencies, and current version source.
    • Inventory projects and actual published NuGet artifacts; distinguish README claims from verified functionality.
    • Classify engine/platform/infrastructure/connector/host projects.
    • Record activity IDs, serialization contracts, TFMs, licenses, provider SDKs, and compatibility constraints.
  • As a program contributor, I can identify the next unblocked work.
    • Create and link epic/feature issues, attach existing work, and record blockers.
    • Publish an inventory and unresolved-questions register in the repository.

Exit: inventory tied to inspected commit(s), existing work mapped, hierarchy created, and actual gaps identified.

E1 — Integration landscape research and prioritization

Features

  • F1.1: Establish target personas and workflow scenarios.
  • F1.2: Build a broad candidate catalog.
  • F1.3: Perform technical/commercial feasibility investigations.
  • F1.4: Rank delivery cohorts and define their operation-level scope.

Research method

  1. Start with the existing extensions inventory and public requests.
  2. Identify target scenarios for Microsoft-centric teams, SaaS/business operations, developers, and self-hosted enterprise users.
  3. Survey approximately 50–100 candidate services across the categories below. This is a bounded discovery target, not a completeness claim.
  4. Deeply investigate the top 15–20 candidates using official provider documentation.
  5. Validate demand from authorized evidence and maintainer/customer input. Competitor availability is discovery evidence, not proof of Elsa demand.
  6. Produce ranked cohorts, exclusions, unresolved questions, effort ranges, and confidence levels.

Required catalog fields
Stable ID, provider, categories, target personas, concrete workflows, existing Elsa status, proposed actions/searches/triggers, authentication types/scopes, OAuth app/verification requirements, API and webhook documentation, polling/renewal requirements, pagination and rate limits, file/size constraints, paid-tier/API-access restrictions, sandbox/test availability, SDK/license considerations, data residency considerations where relevant, maintenance burden, ownership/sponsorship, shared-cloud eligibility, Elsa compatibility, evidence URLs, checked date, confidence, and recommendation.

Use machine-readable records (JSON/YAML or equivalent) plus a generated readable summary. Unknowns remain explicit; do not infer API support from a competitor listing.

Proposed scoring
Score 0–5 on each dimension: evidenced demand (30%), complete-workflow coverage (25%), feasibility (15%), shared-infrastructure reuse (15%), and maintenance sustainability (15%; higher means easier to sustain). Publish rationale and confidence separately. Apply hard eligibility checks for usable API/auth access and cloud restrictions; do not let a weighted score conceal blockers. Weights are an initial hypothesis to validate.

Candidate categories

  • Microsoft: OneDrive, SharePoint, Outlook, Teams, Microsoft calendar, Dynamics 365.
  • Google: Drive, Sheets, Gmail, Calendar.
  • Communication: Slack, email/SMTP providers, Twilio, Discord, Telegram.
  • Finance: Moneybird, Stripe, Exact Online, Xero, QuickBooks, PayPal.
  • CRM: HubSpot, Salesforce, Pipedrive, Zoho CRM.
  • Engineering: GitHub, Azure DevOps, GitLab, Jira.
  • Support: Zendesk, Freshdesk, Intercom.
  • Data/storage/messaging: SQL Server, PostgreSQL, MySQL, S3-compatible storage, Azure Blob, Azure Service Bus, RabbitMQ, Kafka.
  • Knowledge/tasks: Notion, Airtable, Asana, Trello.
  • Commerce/documents: Shopify, WooCommerce, DocuSign and document-processing services.
  • AI/agents: model invocation, structured extraction, agent execution, governed remote tools.

These are research candidates, not verified support commitments.

Exit: sourced catalog, deep assessments, scored shortlist, initial operation matrices, maintenance estimates, and recommended complete workflow cohorts.

E2 — Repository consolidation and developer experience

Features

  • F2.1: Architecture decision for folder, dependency, and release boundaries.
  • F2.2: Incremental import of all Extensions and Elsa Studio into Core, including paired backend/Blazor modules.
  • F2.3: Root solution, solution filters, samples, and debugging workflows spanning backend and Studio.
  • F2.4: Migration of contribution/release documentation and source ownership.

Tasks/requirements

  • Inspect repository instructions and branch/release policy before changes.
  • Choose and document history-preservation strategy and migration order.
  • Preserve package/activity identities and existing release support.
  • Establish a single authoritative source and publisher per migrated package.
  • Update project references, SourceLink/repository metadata, CI paths, tests, and documentation.
  • Map old issues/PRs and avoid losing active work. Freeze/archive the old repository only after migration completion is assessed, not as a first step.
  • Verify one focused connector filter brings in the needed core source dependencies for debugging.

Exit: migrated work builds and debugs in core, active changes are accounted for, old consumer contracts are preserved, and rollback/cutover guidance exists.

E3 — Independent versioning, packaging, and compatibility

Features

  • F3.1: Explicit release-unit manifests and scoped versions.
  • F3.2: Dependency-aware CI and selective publication.
  • F3.3: Published-package compatibility validation.
  • F3.4: Artifact provenance, release notes, and recovery procedures.

Required behavior

  • A connector change can publish only that release unit; unrelated packages are not version-bumped.
  • Shared changes expand the impacted build/test set using the dependency graph.
  • Explicit change records declare version intent; connector-specific tags identify immutable release commits.
  • CI validates source/project-reference development and clean consumers using built/published NuGet artifacts against claimed released Elsa versions.
  • Check package dependency metadata and prohibit accidental reliance on unreleased core changes.
  • Verify the exact artifacts being published match the release commit; smoke-test actual packages, not only source.
  • Document supported compatibility ranges/matrix and SDK dependency conflicts.
  • Preserve existing package version monotonicity; do not reset a migrated package from 3.x to 1.x.
  • Document failed/partial publication recovery without overwriting immutable NuGet versions.

Exit: an existing connector demonstrates an independent prerelease/release process, clean host consumption, and no unrelated publication; a shared-dependency change triggers the right additional tests.

E4 — Connector SDK, connections, and execution infrastructure

Features

  • F4.1: Tenant-owned named connections and runtime authorization.
  • F4.2: OAuth/API-key authentication lifecycle and integration with the existing Elsa Secrets module, subject to the architecture assessment below.
  • F4.3: Typed actions/searches, schemas, pagination, and dynamic options.
  • F4.4: Provider-aware execution, throttling, retries, and diagnostics.
  • F4.5: Managed file references and data transformation primitives.
  • F4.6: Governed authenticated API requests for the long tail.

Key acceptance

  • Workflow definitions contain connection references, not embedded secrets.
  • Authorization is enforced on each execution/resumption and on design-time lookups.
  • Refresh concurrency, reconnect/revocation, credential rotation, scope changes, and environment promotion are defined.
  • Retries distinguish safe reads from writes with unknown outcomes; use provider idempotency where available and never promise universal exactly-once delivery.
  • Large files use bounded streams/references with ownership, expiration, quotas, and cleanup.
  • Diagnostics redact secrets and expose actionable provider errors.
  • Reuse existing Elsa abstractions where suitable. Audit JSON/CSV/XML, date/time, validation, approvals, and subworkflow capabilities before adding replacements.
  • Generic authenticated requests enforce provider/destination and credential-forwarding restrictions.

Exit: pilot connectors share these capabilities without connector-specific copies of generic lifecycle logic.

Secrets module and connection architecture decision

Explicitly investigate whether and how integration connections should leverage Elsa's existing Secrets module. Reuse is the preferred starting hypothesis, not an assertion that its current implementation already satisfies the requirements.

Proposed responsibility split to evaluate

  • A connection owns non-secret metadata: provider, account/administration identifiers, granted scopes, status, ownership, and references to credentials.
  • The Secrets module or an adapter over its provider abstractions resolves/protects sensitive material: API keys, OAuth client secrets, refresh tokens, access tokens when persisted, and webhook signing secrets.
  • The authentication/connection layer coordinates consent, refresh, reconnect, and revocation. Secret storage alone does not implement the OAuth lifecycle.
  • Workflows refer to connections; connection records refer to secrets. Avoid exposing raw tokens as normal activity inputs/outputs or persisting them in workflow state, logs, traces, or exports.

Questions and alternatives for the ADR

  1. Can existing Secrets providers support runtime writes, atomic/versioned updates, deletion, and the access patterns needed by rotating OAuth credentials, or are some providers read-only configuration sources?
  2. Compare direct reuse, extending Secrets, and a dedicated credential lifecycle store backed by Secrets/provider abstractions. Justify any separate store and avoid duplicating secret management without a demonstrated requirement.
  3. Decide whether short-lived access tokens should be cached only or persisted; define expiry, cache isolation/invalidation, and durable refresh-token storage.
  4. Define concurrency and recovery for refresh-token rotation across workers, including a crash between provider refresh and local persistence. Do not assume cross-system atomicity.
  5. Distinguish platform-owned OAuth application credentials from tenant-owned connection credentials; define authorization for creating, using, sharing, inspecting metadata, rotating, and deleting them.
  6. Assess encryption/key management and supported external vault integration from actual module capabilities. Define outage behavior, audit events without secret values, and least-privilege access.
  7. Define secret reference/version semantics, environment-specific binding during promotion, disconnect/revocation, tenant deletion, and cleanup of credentials shared by multiple connections.
  8. Verify background polling, webhook renewal, and resumed workflows resolve credentials through an authorized tenant execution context.

Deliverables and acceptance

  • E0 audit documents actual Secrets APIs/providers and gaps.
  • E4 produces an ADR comparing alternatives and selecting a supported approach before credential persistence is implemented.
  • Create feature/story/task issues for reuse, necessary module extensions/adapters, and any credential migration.
  • A pilot demonstrates connection creation, authorized credential resolution, token refresh/rotation, reconnect/revocation, and redacted diagnostics.
  • E6 verifies cross-tenant access denial, export/state redaction, concurrent refresh behavior, storage failure handling, and disconnect/deletion semantics.
  • Record how the chosen approach applies to self-hosted deployments and preserves existing Secrets consumers.

E5 — Durable triggers and event lifecycle

Features

  • F5.1: Webhook routing, verification, subscription provisioning, and renewal.
  • F5.2: Durable polling with cursors, leases, pagination, and fair scheduling.
  • F5.3: Deduplication, durable ingestion, recovery, and operational health.

Key acceptance

  • Tenant routing uses trusted subscription/connection records rather than trusting payload tenant IDs.
  • Events are acknowledged according to a documented durable-ingestion policy.
  • Duplicate/reordered events, missed renewal, crash/restart, invalid signatures, and provider downtime have defined handling.
  • Workflow publish/unpublish, connection revocation, and tenant deletion reconcile subscriptions and polling jobs.
  • Delivery guarantees and reconciliation behavior are explicit; deduplication is not described as exactly-once external side effects.
  • Health signals identify stalled polling, expired subscriptions, lag, and repeated failures.

Exit: webhook and polling scenarios survive restart, isolation, duplicate-delivery, and renewal tests.

E6 — Shared-cloud policy, isolation, and operational readiness

Features

  • F6.1: Shared/dedicated/self-hosted capability profiles.
  • F6.2: Tenant isolation and outbound network policy.
  • F6.3: Quotas, fairness, observability, and failure containment.
  • F6.4: Curated host composition, upgrades, rollback, and deprecation.

Key acceptance

  • Shared cloud accepts curated packages only.
  • Connections, caches, files, webhook mappings, polling state, logs, and quotas are tenant-scoped and tested.
  • HTTP/database/remote-tool capabilities address SSRF, private-network access, redirects, credential leakage, and resource exhaustion.
  • Expressions and scripts have an assessed execution boundary; language-engine configuration is not assumed to be a security sandbox.
  • Host manifests pin the tested connector set. Do not promise per-tenant arbitrary package versions in one process.
  • Upgrade tests cover saved definitions and suspended workflows; incompatible changes require documented migration or routing to compatible hosts.
  • Connection/provider outages are observable without exposing secrets.

Exit: the pilot cohort meets the shared-cloud policy and has deploy/rollback and incident runbooks.

E7 — Pilot connectors and complete workflow examples

Features

  • F7.1: OneDrive pilot: authentication, file operations, and a validated change-detection path.
  • F7.2: Moneybird pilot: authentication, selected business-record actions, and a validated event path.
  • F7.3: Existing Slack connector audit and upgrade to shared infrastructure.
  • F7.4: End-to-end examples and user validation.

These pilots are the initial hypothesis. E0/E1 may justify a replacement before implementation.

For each pilot, define an operation matrix: activity/trigger ID, inputs/outputs, scopes, behavior, pagination, errors, idempotency, limits, and compatibility. Keep initial API coverage narrow and useful.

Example scenarios to validate

  • New document → review/approval → notification.
  • Business event → selected accounting record action → confirmation.
  • Engineering/business event → routed team notification.

Use realistic provider semantics; do not assume that a document event directly maps to invoice creation without a validated scenario.

Exit: published packages, onboarding documentation, reproducible examples, sandbox/live contract checks where available, negative-path tests, and shared-cloud certification evidence.

E8 — Catalog experience and sustainable expansion

Features

  • F8.1: Discoverable connector/operation catalog and Studio integration.
  • F8.2: Connection setup, resource selection, and actionable diagnostics.
  • F8.3: Connector templates, contributor guidance, and certification checks.
  • F8.4: Cohort rollout, support ownership, and lifecycle governance.

Key acceptance

  • Users see actual supported operations, compatible versions, deployment eligibility, limitations, and maturity.
  • Metadata drives docs and discovery where practical; avoid duplicated inventories.
  • Define Experimental, Preview, and Stable requirements.
  • Every maintained connector has ownership, provider-change monitoring, support expectations, deprecation policy, and test-account arrangements.
  • Expansion is selected by E1 evidence and complete workflow coverage, with explicit maintenance capacity.

Exit: catalog and contribution process are usable, and the first post-pilot cohort is scoped and owned.

E9 — Elsa 4 portability assessment

Features

  • F9.1: Identify reusable provider clients, schemas, authentication definitions, and test fixtures.
  • F9.2: Define Elsa 3 versus Elsa 4 runtime adapter boundaries.
  • F9.3: Validate one representative portability spike after the Elsa 3 pilot.

Exit: a documented, evidence-based migration/reuse strategy. Full Elsa 4 connector implementation is a follow-on program, not a gate for Elsa 3 completion.

Delivery sequence and dependencies

Milestone Work Exit gate
M0: Program ready E0; initialize E1 Inventory, hierarchy, evidence plan, unknowns recorded
M1: Architecture and portfolio E1 + E2/E3 design Ranked shortlist, ADRs, pilot scope, effort ranges
M2: Release proof E2/E3 on one existing connector Source debugging + independent package + clean host proof
M3: Vertical slice E4/E5 + first E7 slice; E6 controls included Connect → trigger → execute → observe works safely
M4: Pilot cohort E7 + E6 + E8 essentials Three pilots or justified replacements meet certification
M5: Expansion readiness E8 + E9 assessment Owned next cohort, runbooks, portability findings

Research and architectural discovery can proceed concurrently. E4/E5 should be shaped by pilot vertical slices, not implemented as a large speculative framework. Shared-cloud controls are designed and tested throughout, not deferred until the end.

Do not require the entire repository migration to finish before proving independent release mechanics. Do not migrate a package while two pipelines can publish it.

Definition of done and success measures

  • Initial hierarchy verified on 2026-09-23: all 68 native parent/sub-issue relationships and 26 separate blocking dependencies are configured and read back through authenticated GitHub REST.
  • Inventory and evidence-based research outputs are committed and linked.
  • All Extensions and Elsa Studio are consolidated into Core under a documented cutover plan, with paired module compatibility and independent release boundaries verified.
  • One connector can be released without republishing unrelated packages.
  • Published artifacts are traceable to immutable source commits and pass clean-consumer tests.
  • The Secrets-module reuse/extension decision is documented and validated in a pilot, including token lifecycle, authorization, and redaction.
  • Connections and durable triggers pass lifecycle and tenant-isolation tests.
  • Pilot workflows run using documented setup and curated packages.
  • Saved/suspended workflow compatibility is demonstrated for supported upgrades.
  • Catalog, contribution guidance, support ownership, and runbooks are available.
  • Ranked follow-on cohorts and Elsa 4 portability findings are recorded.

Measure baseline and subsequent change for connector lead time, unrelated packages published per connector release (target: zero), validated compatibility coverage, setup success, trigger lag/recovery, release failures, and maintenance effort per connector. Establish numerical operational targets after baseline measurement rather than inventing them here.

Risks and decisions to record

  • Monorepo builds may become expensive: dependency-aware CI and focused filters.
  • Shared infrastructure changes may create release coupling: keep contracts narrow and validate consumers.
  • NuGet/API compatibility may differ from source compatibility: test exact artifacts and supported host combinations.
  • Provider OAuth verification/API access may block delivery: investigate early.
  • Long-running workflows may break on schema/type changes: stable identities and explicit migration policy.
  • Broad connector coverage may exceed maintenance capacity: narrow operation scope, ownership, cohorts, and deprecation.
  • Scripts/HTTP/remote tools can bypass intended tenant boundaries: enforce capability and execution policies.
  • Cloud data models may already provide connection/secret functionality: inventory and reuse before introducing competing abstractions.
  • Generated OpenAPI wrappers may accelerate implementation but do not solve auth, lifecycle, UX, or semantics: evaluate selectively.
  • Research must distinguish verified facts, assumptions, and demand hypotheses.

Required ADRs: repository migration; release-unit/versioning policy; package compatibility; connection/secret ownership and existing Secrets-module reuse versus extension; trigger lifecycle/delivery guarantees; shared-cloud capability policy; host upgrade/workflow compatibility; Elsa 4 reuse boundary.

Execution index — initialized 2026-09-23

All 68 child issues are in elsa-workflows/elsa-core. Native hierarchy and dependencies verified on 2026-09-23: authenticated GitHub REST configured and individually read back all 68 intended parent/sub-issue relationships and 26 explicit blocking dependencies. The Markdown links remain readable references. No issues were recreated or silently reparented; both graphs are acyclic. Title prefixes identify semantic levels without inventing organization issue types or labels. Audit evidence is being committed under #8205.

Epics

Initial tasks — accepted audit evidence

Initial story index

Execution protocol

  • Start with the three unblocked tasks above; they can proceed independently.
  • Read repository instructions, check for existing work, and record source commits before investigating.
  • Deliver the specified evidence as a repository document/PR; record verification and limitations on the task.
  • Follow the explicit task blocker lists. Parent membership is not a blocking dependency.
  • Advance from inventory to migration/release decisions, and from Secrets audit to the credential ADR.
  • Expand later stories/tasks only when prerequisite findings establish scope.
  • No assignees, deadlines, public package releases, or production changes are implied by backlog creation.
  • Initial link and task-dependency validation found no missing parents or task dependency cycles.

Next execution instructions

  1. Inspect repository instructions and current state; reconcile this charter with existing work.
  2. Use the created E0–E9 issues in the execution index; do not recreate them.
  3. Use the forty created feature issues linked from the epics; native relationships are configured and verified (2026-09-23).
  4. Execute the initial stories/tasks and progressively elaborate remaining E0/E1 and migration work after findings; do not duplicate the existing breakdown.
  5. Link existing relevant issues; do not silently reparent unrelated work or create duplicates.
  6. Update this program with real issue links, decisions, dependencies, and milestones.
  7. Begin the audit and bounded research. Publish evidence-backed results before committing to later connector scope.
  8. Keep later work progressively elaborated; this charter does not imply implementation or research has already been completed.

Starting references

Provider behavior and API access must be revalidated during research. These links are starting points, not a completed feasibility assessment.

Initial execution evidence (historical review snapshot, 2026-09-23)

The original 69 issues were read and preserved. All 68 native parent/sub-issue links and 26 separate blocking dependencies were configured and read back; both graphs are acyclic. No issues were recreated or unrelated work reparented.

Evidence proposed for review PR Backlog work
Reconciliation, live verifier and architecture/next-batch summary #8264 #8205
Core, Extensions and Studio inventory; release ownership #8266 #8251, #8252
Secrets assessment, credential-lifecycle ADR and validation matrix #8265 #8253, #8254, #8261
Catalog schema, survey, deep assessments and conditional priorities #8267 #8255, #8256, #8257, #8258

Historical delivery status, superseded below: evidence PRs were open for review. Their existence is not accepted implementation or completed issue delivery; acceptance checkboxes remain open. No PR was merged, repository archived, package published, or production changed.

Verified findings:

  • Inventory covers 343 project paths at recorded commits, complete module trees and associated tests/samples/build/docs assets. Five package IDs have competing source locations and need explicit canonical source/publisher ownership. Public Slack 3.8.4 artifact provenance was inspected separately; static project declarations are not package compatibility proof.
  • Source co-location must preserve independent release units. Current solution-wide package workflows can publish unrelated previews. A first audit push triggered Core's existing codex/* Packages job; it was canceled before build/publish steps ran. Further audit branches avoid that path, and the reconciliation branch skips publishing CI deliberately.
  • Existing Secrets storage/protection is reusable, but generic version rotation is not distributed OAuth coordination. The proposed lifecycle service separates metadata/authorization, secret persistence and provider state, including consumed-token ambiguity and staged-generation recovery. 359 targeted existing tests passed with retained run evidence; future two-worker outbound lifecycle tests remain unimplemented.
  • Catalog contains 67 candidates, 18 deep assessments, 124 proposed operations, six workflow hypotheses and 172 dated sources. Official documentation, historical public demand leads, engineering estimates and unknowns are distinguished. Pilots remain conditional. All six current Slack Watch activities throw rather than implement triggers.

Recommended next execution batch: review the evidence, then execute the refined #8259 release-unit design and #8262 synthetic credential validation plan. After #8259 is accepted, run #8260 using local artifacts only: selected-package/version isolation, dependency-aware tests, actual nuspec/provenance and clean consumer checks. Slack is the recommended packaging candidate; a natural backend/Studio pair still needs a separate co-debugging proof. Related OneDrive/Slack historical issues remain linked from #8234/#8236, not reparented.

Unresolved decisions: canonical owners for the duplicate package IDs; sole publishers and release-unit ownership; durable lifecycle persistence/fencing and lifecycle-owned generation enforcement; the natural backend/Studio proof pair; and pilot sponsors, maintainers, test accounts, exact consent and operational responsibility. These are explicit follow-up gates, not reasons to invent demand or begin a wholesale migration.

PR check/reviewer results are recorded on each PR and may continue changing. Local audit validation must not be confused with a full solution CI pass or provider certification. Copilot review requests did not register on the first three PRs; no unavailable review is claimed as passed.

End-to-end execution status — 2026-09-23

Canonical delivery board: https://github.com/orgs/elsa-workflows/projects/52. The existing issues remain the authoritative backlog; native parents and blocking dependencies remain separate.

Accepted and merged audit evidence: #8264 (hierarchy), #8266 (inventory and ownership), #8265 (Secrets assessment and proposed lifecycle architecture). Corresponding bounded audit tasks #8251, #8252, #8253, #8254 and #8261 are closed with acceptance evidence. Catalog #8267 is also merged after all three findings were fixed; #8255–#8258 are accepted with schema, evidence and regression verification. No audit merge constitutes completion of the program or its implementation features.

Current integration lane: #8259 then #8260, local independent-packaging proof. Parallel implementation: #8270, synthetic durable credential lifecycle, followed by #8271 workflow/offboarding conformance. The #8262 validation plan is accepted. Backend/Studio co-debugging remains a separate proof from Slack packaging.

All four audit merges used reviewed heads and normal squash merges; their commit messages suppress publishing-capable push workflows. No Packages or deployment run was present for those merge heads on verification. Required PR checks were not bypassed. Subsequent implementation merges must again inspect current triggers; source co-location must not impose a shared release train.

The execution goal now includes implementation, verification, integration and operational handoff. Focused merges are authorized after checks and review gates pass. Publication (including preview feeds), production/cutover, repository archival and external communications remain reserved for explicit approval after concrete artifacts and rollback evidence.

Execution additions: #8268 rehearses exact source relocation with original history for #8213; #8272 refines #8262. New story #8269 under #8222 contains #8270 (synthetic durable lifecycle) and #8271 (workflow/offboarding conformance). These are evidence-derived implementation subtasks, not replacement backlog. The board now has 74 verified issue memberships: original 69 plus credential story #8269/tasks #8270–#8271 and compatibility story #8275/task #8276.

Project visibility limitation (2026-09-23): GitHub confirms the original 72 unarchived Project item nodes and the two new #8275/#8276 memberships and their issue-side memberships/fields, but the Project items collection and web table currently return zero rows. A membership refresh and explicit position update did not repair enumeration. No duplicate board or issues were created. Until GitHub exposes the rows, this program issue is the visible delivery queue: execute #8259→#8260 and #8270→#8271; collision compatibility evidence for #8213/#8214 proceeds independently.

Implementation progress — 2026-09-23 16:49 UTC

No package publication or production/cutover is authorized by these merges. The goal remains active.

Active delivery queue — 2026-09-24

No package feed publication, production change, cutover or repository archival has been performed or approved. Implementation continues; reserved release/cutover approval requires concrete verified artifacts and rollback instructions.

Latest execution checkpoint — 2026-09-24

No package publication, production cutover, repository archival or external communication is authorized by this checkpoint.

Latest execution checkpoint — 2026-09-25

No package/feed publication, production change, cutover, repository archival or external provider communication has occurred. Continue implementation on unblocked slices; request the reserved release/cutover approval only with exact release evidence and rollback artifacts.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    elsa 3This issue is specific to Elsa 3epicAn 'issue of issues', representing a large or conceptual enhancementprio highIs on the roadmap for the near-futuretriaged

    Type

    Feature

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions