Skip to content

[Vision] CAO 3.0 — run anywhere, shared by your whole organisation as a multi-tenant SaaS #777

Description

@haofeif

[Vision] CAO 3.0 — run anywhere, shared by your whole organisation as a multi-tenant SaaS

The vision

CAO should run anywhere, and be shared by everyone.

The same product should work on a laptop with no setup at all, on a server a team shares, and on a cloud or Kubernetes cluster serving an entire organisation. You should be able to start on your laptop and later have your company run CAO as a proper shared service — without switching tools, relearning anything, or leaving your work behind.

In other words: CAO becomes a multi-tenant service, in the same sense as any business application your organisation already runs.

A quick definition, since everything below depends on it. A tenant is one organisation using CAO. Your company is a tenant. A customer running their own is a tenant. On your laptop, you are a tenant of one. The tenant is the outer wall — colleagues inside it may deliberately share things with each other, but two different tenants must never see a trace of one another.

flowchart LR
    A["Your laptop<br/>one person, zero setup"] --> B["A team server<br/>sign in with your work account"] --> C["A cloud cluster<br/>the whole organisation"]
    A -.->|"same product, same commands, same files"| C
Loading

Tracked as CAO 3.0, on the dev-3.0release branch. It's a big change, so it lands piece by piece.

Where we are today

CAO was built for one person on one machine, and it's honest about that. One comment reads "this is a same-user local control, not a privilege boundary", and the security guide says the server runs on localhost and you should put real authentication in front of it before exposing it anywhere.

Those were the right decisions, and they're much of why CAO is quick to install and pleasant to use. But four things are assumed everywhere in the code:

  • There's only one user, so nothing records who owns what.
  • Much of each server's coordination state is process-local.
  • Metadata is in SQLite, while workflow specs, memory content and logs also live in files on that node.
  • Each node executing agents expects a local server and execution backend. The EKS example therefore repeats a full server in each agent pod.

None of those survive contact with a team.

What actually goes wrong today

Two people, one name. Alice builds a workflow called deploy. Bob builds a different one, also called deploy. CAO stores workflows under their plain name, so there is only ever one thing called deploy. Bob's silently replaces Alice's — no warning, no conflict, no copy kept.

Worth being blunt: that isn't a missing privacy feature that can wait. It's people destroying each other's work by accident, on the first afternoon.

Agents finding each other by accident. CAO lets one agent discover and message another. Which ones it can see is decided by the caller's terminal session and a shared group name, and the session limit can be lifted with a supported option. The code already anticipates where that goes:

"two unrelated CAO sessions that happen to reuse the same group prefix (a naming collision, a copy-pasted template, two features that picked the same tenant/project id) would silently discover each other"

Today that's an accident between one person's own workflows. With several organisations sharing a server, the same accident reaches across companies.

Both have the same root cause. CAO can't keep people apart because it has no idea who anyone is.

The one rule that can't be broken

Nothing here is allowed to make the laptop experience worse.

If you use CAO on your own machine, after 3.0 it behaves exactly as it does now. No sign-in. No database server. No message broker. No cluster. Same commands, same files in the same place, same speed. Upgrading from 2.5 has to be uneventful — your sessions, workflows and memories carry over and nothing needs migrating by hand.

How we get that: a single person on a laptop is treated as a tenant of one. The same ownership and authorization implementation serves local and shared callers. On your laptop you're the only member and own everything without sign-in. An operator explicitly enables shared use; missing or broken authentication after that must fail closed, not restore the anonymous local administrator.

What a shared service needs, and what CAO has

The honest inventory. Every row was checked against the code, not assumed.

What a multi-tenant service needs CAO today In 3.0?
Organisations as a real concept Nothing — the code says "nothing here provides tenant isolation" Yes
Data kept apart between organisations Nothing Yes
Sign in with a work account (SAML, Entra ID) Only API token scopes, off by default Yes
Everything having an owner Nothing Yes
Someone in charge — roles, adding and removing people An admin token scope, attached to no person Yes
Sessions that can be ended when someone leaves Nothing Yes
Sharing on purpose within your organisation Nothing Schema only
Knowing which version the stored data is in Nothing Yes
cao-server as a dedicated service agents connect to A Service address exists, but every agent still runs its own full server Yes
Output that survives crossing between pods In-memory and local named pipes; queue overflow is logged but consumers receive no gap marker Yes
Consumer sign-in (Google, Facebook) Nothing No
Automatic joiner/leaver sync (SCIM) Nothing No
Spending limits and per-team cost Optional orchestration telemetry; no built-in attributable model-usage or spend accounting No
A full audit trail Memory/learning events in a file log, not an application-wide tenant audit No
Service accounts for automation Nothing No
An overall core API version/compatibility contract Core routes are unversioned; AG-UI already has /agui/v1 No
Stronger-than-container isolation Nothing No
Horizontal scaling — several cao-servers as traffic grows Nothing; blocked by more than the database No
Automated backup service and failover No general-purpose service No; safe upgrade backup/recovery is part of #775

The foundation is the shared identity and resource model: organisation, owner, membership and visibility affect keys, files, queries and execution context together. Agree and migrate that shape before building more shared features on it. This avoids repeated redesign of the same boundaries; it is not a claim that every future column requires migrating every table.

What 3.0 is, and what it is not

CAO is an open-source project. It ships software that other people deploy; it does not run a service for anyone. So "multi-tenant" here means the software supports being run that way — not that this project operates one.

3.0 is the step up from 2.5, and it has two goals:

  1. Keep everything CAO does today, but let it be hosted remotely in a natural way. Anything that works on a laptop today must still work when the server and the agents are on different machines. This is preservation, and [Feat] Centralize cao-server in one Kubernetes Deployment and Service #745, [Feat] Decouple data access, fix SQLite's concurrency settings, and make upgrades safe #775 and [Feat] Carry terminal output, input and status across the pod boundary created by #745 #776 are about it.

    The release bar is wider than the first-provider slice. CAO has 13 provider types (12 real plus a mock), and several paid providers cannot run live in every CI job. Require a provider-agnostic bridge, a live end-to-end provider path, provider/backend conformance coverage and [Feat] Centralize cao-server in one Kubernetes Deployment and Service #745's complete client/operation matrix. Native CLI, browser, operator/in-session MCP, workflows, scheduling, memory and retained-data operations must be accounted for. One working provider or one browser terminal is not the whole preservation result.

  2. Lay the foundations for multiple organisations to share one deployment. Organisations, ownership, sign-in and administration. This is genuinely new capability, chosen deliberately — [Feat] Tenant model and isolation boundary: make an organisation a first-class thing in CAO #778, [Feat] Sign in with SAML or Entra ID, and give every resource an owner #774, [Feat] Tenant administration: roles, member management, and revocable sessions #779 and [Feat] Sharing and visibility within a tenant (schema in 3.0, experience after) #780.

Saying that plainly matters, because the two goals have different bars. Goal 1 is judged by "does it still work when hosted remotely." Goal 2 cannot be, because there is nothing there today to preserve.

3.0 is the foundation, not a finished commercial service. The distinction decides several things:

The foundation is meant to be adapted. Different operators will want different identity providers, different scaling, and different hardening. The job of 3.0 is to provide the structure and sensible defaults — organisations, owners, a sign-in seam — so that changing those pieces is a swap rather than a rewrite. Where a decision is a default rather than a law, the issue should say so.

What 3.0 delivers

Six things. The bar is deliberately high: without it, 3.0 either quietly loses people's work or isn't really shared.

1. CAO learns what organisation you belong to (#778). The foundation of the data model, including operator-controlled first-tenant/first-administrator setup. A tenant remains an organisation, not a team or an identity-provider directory. The remote bridge (#745/#776) can start in parallel, but its shared-use paths must carry this same tenant/owner context.

2. CAO learns who you are (#774). You sign in with your normal work account. SAML and Microsoft Entra ID are the first-class targets, because that's how organisations actually onboard staff. Consumer sign-in like Google or Facebook is deliberately left out.

Both sign-in adapters establish the same CAO application session and stable internal identity. This includes authenticated paths for the CLI and operator MCP, not just a web login screen. When an upgraded local installation enables sign-in, the operator binds the intended work identity to its existing owner so the user's work remains reachable.

3. Your things belong to you (#774). Workflows, sessions, terminals, messages, memory and custom profiles/skills keep the same tenant/owner boundary through storage, discovery and execution. Alice's deploy and Bob's deploy become two different things. Built-in catalogue content stays read-only, while installation-wide settings remain operator-controlled (#778).

That boundary includes tenant plus owner, not owner alone. A person who belongs to two organisations can reuse names and retry keys without joining the two organisations' data.

4. Someone can run the thing (#779). Members manage their own work; tenant administrators manage membership and intervene within their tenant, without inheriting control of the installation. Removal reaches live connections, delegated tools and queued/scheduled work, with explicit cancellation outcomes and retained private data.

5. Upgrading is safe (#775). CAO currently can't tell which version of the data layout it's looking at, and a failed upgrade step is written off quietly. Survivable for one person with one copy; not for a team's shared data. This is what makes a safe upgrade from 2.5 real rather than hopeful: #775's exit criterion is that "a populated 2.5 database upgrades with nothing lost, everything assigned to the local tenant and user."

6. cao-server becomes a dedicated service (#745, #776). Ten agents should mean one CAO server, not eleven. Today every agent pod runs its own full copy, which is why the count grows with the fleet.

To be precise about what this is and isn't: it is not about running several cao-servers when traffic grows — that's horizontal scaling, and it stays out of 3.0. It's about cao-server being one properly deployed service that thin agents connect to.

The network side already exists: there's a Kubernetes Service at cao-supervisor…:9889, and workers already call back to it. What changes is what travels over it. Today terminal output goes through a named pipe on local disk into an in-memory bus inside the agent's own server — so once agents no longer run a server, that output has to reach the central one instead. That's #776, and it's why these two are one piece of work, not two.

It also needs messages to be countable. #745 asks for "ordered output/event delivery… and explicit gaps or partial output", and a gap can't be reported unless it can be detected — today a dropped message is indistinguishable from no message.

Today's terminal event bus carries output and status, while completion and cancellation use status polling, process-local events and broker HTTP calls. Those broker calls already cross machines; removing each worker's server means the remaining local mechanisms need an explicit remote equivalent. #745 owns command correlation, acknowledgements, retained completion and cancellation outcomes; #776 must carry those semantics, not exclude them under a blanket "no acknowledgements" rule.

The browser terminal also has a separate local PTY attach/input/resize path; it does not consume the FIFO event bus. #776 includes moving that path beside the remote agent while retaining the existing browser interface.

Python workflows and scheduled flow pre-scripts are another existing execution path: they currently start subprocesses on the server host, outside TerminalBackend. #745 includes moving that user code into execution workloads while retaining central scheduling, state and the existing script APIs. Remote-terminal success alone does not cover these capabilities.

The client side must move with them. Today some CLI commands use local services/files or attach to local tmux, and fleet commands proxy the worker servers being removed. #745/#775 cover those paths; #774 covers authenticated HTTPS addressing. A shared target must not quietly fall back to the laptop's database, profile or working directory.

Sharing within a tenant (#780) lands as schema only. User-owned work remains private on API/import/write paths as well as in the interface; future sharing builds on this foundation without a promise that richer audience/transfer features need no further schema.

Why the foundation has to come first

A later release would otherwise have to revisit resource identity, storage, authorization and execution context, including the features already built on them. Establishing those shared contracts together is the foundation; later additive migrations remain normal.

This is a large refactor of premises the code currently takes for granted — one user, one machine, one owner implied by the filesystem. The main risk is therefore not the new features; it is breaking the single-user laptop case that CAO is today. Every step must leave that case working, and that is a testable bar rather than an aspiration.

What we're deliberately leaving out

Consumer sign-in, SCIM, PostgreSQL, spending limits (#781), a full audit trail (#782), service accounts and overall API versioning (#783), stronger-than-container isolation (#784), the agent-manipulation threat model (#785), and an automated backup service, failover and running several servers at once. The backup and recovery procedure required for the 3.0 migration remains part of #775.

Each keeps its own delivery scope and builds on the foundations below. Deployment suitability still depends on the actual workload risks and required controls, not merely on whether a feature is assigned to a later milestone.

What the follow-on tickets build on

Follow-on Foundation it consumes, rather than postpones
Usage and limits (#781) Trusted tenant/owner/execution attribution from #774/#778/#745; actual token collection and tenant quota accounting still belong to #781
Audit (#782) Stable actors, owners, tenant membership and operator boundaries from #774/#778/#779, including events whose caller or tenant is not yet known
Service accounts and public API versioning (#783) Existing delegated runtime credentials remain in 3.0; server/MCP/runtime compatibility is already required by #745/#776, independently of later public route versioning
Stronger isolation (#784) Builds on approved execution/storage/network boundaries, with suitability judged by workload trust and reachable privileges rather than the number of organisations
Agent-manipulation work (#785) Assesses the authority actually granted by #774/#745/#779 and remaining OS/provider permissions, instead of assuming every agent inherits all human permissions

How we'll build it

Piece by piece on dev-3.0release, not one enormous release. Each step has to stand alone and be safe to ship — but the order isn't negotiable:

  1. Migration machinery first ([Feat] Decouple data access, fix SQLite's concurrency settings, and make upgrades safe #775). A schema version, migrations that fail loudly, and a recovery path. The shared-model changes touch data people already have, so this safety net must exist before they run.
  2. Agree the whole shape on paper ([Feat] Tenant model and isolation boundary: make an organisation a first-class thing in CAO #778 with [Feat] Sign in with SAML or Entra ID, and give every resource an owner #774, [Feat] Tenant administration: roles, member management, and revocable sessions #779 and [Feat] Sharing and visibility within a tenant (schema in 3.0, experience after) #780). Organisation, identity, membership, sessions, ownership and visibility are designed together, including the resource/access inventory and the local-to-work-account transition.
  3. One migration, one owner. [Feat] Decouple data access, fix SQLite's concurrency settings, and make upgrades safe #775 carries the agreed records, corrected keys and storage mappings with one named person responsible for it, preserving [Feat] Sign in with SAML or Entra ID, and give every resource an owner #774's default local paths. Sign-in uses this migrated identity/session store, not a separate unversioned one.
  4. Then sign-in and enforcement ([Feat] Sign in with SAML or Entra ID, and give every resource an owner #774, [Feat] Tenant administration: roles, member management, and revocable sessions #779), across owner operations, tenant administration, existing clients and deferred/background execution.
  5. Integrate the dedicated server and execution paths ([Feat] Centralize cao-server in one Kubernetes Deployment and Service #745 with [Feat] Carry terminal output, input and status across the pod boundary created by #745 #776), including scripts, scoped caller context and live-channel revocation.

A note on how these depend on each other. Agree the organisation/identity design before implementing its shared migration. #774, #778, #779 and #780 supply coordinated contracts, not a chain that requires each whole issue to close before another can start. The remote bridge can be developed in parallel with local identity. Its final shared-use integration is not independent, however: assignments, scripts, MCP calls and channels must use the same ownership and revocation contracts.

The shared-use release gate is end to end. A login screen plus filtered browser reads is not enough. Shared use stays gated until ownership, member administration, delegated/background execution and live-channel revocation work together. Intermediate local-only changes can land without presenting an incomplete login/ownership path as multi-user support.

What "done" looks like

  • An operator can establish the first administrator; users can sign in with their work accounts through the supported clients, and an arbitrary first visitor cannot claim existing data.
  • Alice and Bob can both have a workflow called deploy, and neither can overwrite the other's.
  • One person with memberships in two tenants can reuse names and idempotency keys without a collision or cross-tenant result.
  • Ordinary members cannot see, address or touch one another's private resources, including custom profiles and colliding agent-discovery groups. Each can manage their own work; tenant-administrator intervention is explicit and cannot cross tenants or modify operator-only settings.
  • Ten agents means one CAO server, not eleven, and agents reach it as a proper service.
  • Existing Python workflows and scheduled pre-scripts run in execution workloads, not inside the central server container.
  • Native CLI, browser and MCP clients use the shared service with the intended profile/workspace and no hidden local-state fallback. HTTPS and supported component/protocol combinations work as documented.
  • Removing membership closes its live access, prevents new scheduled/queued work and requests cancellation without losing authorized cleanup or falsely declaring an unreachable worker stopped.
  • Server rollout/replacement preserves one active owner; replicas: 1 alone is not treated as proof.
  • An agent finishing its work is never lost in silence. Where the outcome genuinely cannot be known — a worker pod destroyed before its completion was acknowledged — CAO says unknown rather than guessing at success or failure.
  • Upgrading from 2.5 loses nothing; later enabling shared sign-in preserves the intended owner's access through explicit account binding.
  • Running CAO on your laptop is exactly as simple as it is today.

That last point carries the same weight as the others.

Execution trust model

3.0 provides application-level tenant/owner isolation and separately managed execution workloads with approved storage, network and credential access. A trusted operator or a single organisation does not by itself make all executed code trustworthy: repositories, dependencies and model-generated actions can introduce untrusted behavior within one tenant too.

Choose deployment isolation against those actual risks. #784 covers stronger runtime options when the required containment exceeds the baseline; #785 covers manipulation that can still misuse legitimately granted authority. Neither a pod count nor a tenant count is proof that those risks are contained.

The issues

In 3.0

Issue In one sentence
#778 Organisation identity, controlled first setup and the resource boundary used by ownership, administration and shared execution.
#774 Sign in with SAML or Entra ID, and give everything an owner.
#779 Roles, member management, and access that actually stops.
#780 Room to share within an organisation — schema now, experience later.
#775 Decouple data access, fix SQLite's concurrency settings, know which version the data is in, and upgrade it safely. Step 1 — the safety net every migration below relies on. (PostgreSQL is later and optional.)
#745 + #776 cao-server as one dedicated service that thin agents connect to, with output that survives the trip. One piece of work.

After 3.0

Issue In one sentence
#781 Spending limits and per-team cost, because agents burn real money.
#782 An audit trail that covers more than memory and doesn't switch itself off.
#783 Service accounts for automation, and a versioned API.
#784 Stronger isolation than a container — with the trigger for when it's required.
#785 Apply AI-specific prompt-injection guidance to CAO's file, tool, memory and inter-agent content paths.

Related earlier work: #706, #694, #736.

All repository observations verified on main at 29b235cf62ed0f9d624bc9ad9afce09ab72f8ddf.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions