feat: support agent profiles in cloud mode

[hieptl]

Why

Agent profiles were accepted and advertised only when the service runs against a local Agent Server:

• validate_agent_profile_selection rejected any agent_profile_id in cloud mode with 422 Agent profiles require a configured Agent Server.
• GET /v1/capabilities listed agentProfiles only in local mode.
• POST /v1/validate answered every draft that selects a profile with agent_profile_id: Extra inputs are not permitted, in both modes. Since feat: add automation draft lifecycle, endpoints, and synthetic event payloads #439 it normalizes the body through the saved-draft shape, which has no such field, before the creation model sees it. Agent Canvas runs this preflight before it lets a setup form continue, so a form with a profile selected could not get past validation.

On OpenHands Cloud and Enterprise that makes Agent Canvas refuse every automation template that selects a profile (GitHub code review, GitHub issue to PR, GitHub issue triage) with "Not available: agentProfiles". Nothing else needs a local Agent Server: the dispatcher already exports AUTOMATION_AGENT_PROFILE_ID in both modes, and the OpenHands app server starts a conversation with a profile through POST /api/v1/app-conversations (agent_profile_id).

Summary

• Capability. agentProfiles is advertised on every ready deployment. In cloud mode it relies on the app server serving /api/agent-profiles, which the enterprise server does from 1.59.0.
• Validation. Two checks, used by create, update and preflight:
  • validate_agent_profile_combination: a profile owns its model, so the two cannot be selected together. The same in both modes.
  • ensure_agent_profile_exists: in cloud mode, checks the id against the caller's organization with GET {openhands_api_base_url}/api/agent-profiles, using the caller's own credential and X-Org-Id set to the authenticated user's organization, the one the automation is stored under. The app server falls back to default settings for an id it does not know instead of failing, so an unchecked id would silently run with the wrong agent. Local mode is unchanged: the Agent Server resolves the id.
  • An unknown id returns 422 Agent profile <id> not found. A 401 or 403 from the app server is passed on; any other upstream fault, including a 200 that is not the expected shape, returns 502.
  • Create checks after the template lookup, so a repeat enable of a template stays one query and does not depend on the app server. Update checks only a profile that differs from the current one, so sending the current one back cannot fail on a deleted profile or an unreachable app server.
  • Profile ids an organization was just seen to have are remembered for a minute. Only a hit is trusted; an id that is not remembered is always looked up, so a new profile is found at once.
• Preflight. POST /v1/validate accepts agent_profile_id on the raw /v1 endpoint again. The saved-draft shape is unchanged: the profile skips it and is validated by the model creation uses. The existence check is made before preflight touches the database and reported last, and an upstream failure leaves the profile unjudged instead of discarding the other verdicts; creation still makes the check. The preset endpoints still reject the field, as their creation endpoints do.
• Auth. The upstream auth headers are built in one helper (upstream_auth_headers), shared by authentication and the new check.

Video

demo.mov

How to Test

• uv run ruff check openhands/automation, uv run pyright on the changed files: clean.
• uv run python -m pytest tests/test_capabilities_router.py tests/test_auth.py tests/test_router.py tests/test_git_sync.py tests/test_dispatcher.py tests/test_git_sync_serializer.py: 356 passed, 1 failed. The failure is test_create_automation_shares_template_identity_with_presets (botocore ... EndpointConnectionError), which fails the same way on main on the machine I used because it has no S3 endpoint.
• After the preflight fix: tests/test_capabilities_router.py tests/test_router.py tests/test_draft_router.py tests/test_draft_execution_regressions.py tests/test_auth.py: 236 passed, 2 failed, both with the same EndpointConnectionError (the second is test_incomplete_draft_dispatch_returns_validation_errors).
• The new validator, called with a stubbed OpenHands API (httpx.MockTransport) in cloud configuration:

cloud, profile of the caller's org: accepted
   upstream call: https://app.example.test/api/agent-profiles | auth: Bearer user-key | x-org-id: 22222222-2222-4222-8222-222222222222
cloud, unknown profile id: 422 Agent profile `f1d05024-...` not found
cloud, profile plus a model: 422 An agent profile already specifies the model
cloud, cookie session: accepted   (cookie forwarded upstream)
cloud, upstream error: 502 Unexpected response from OpenHands API for agent profiles
local, any id (passed through): accepted   (no upstream call)

• POST /v1/validate in cloud configuration, through the app with the same stubbed OpenHands API:

/v1, profile of the caller's org:   200 valid
/v1, unknown profile id:            200 invalid  agent_profile_id / invalid_agent_profile / "Agent profile `475165a5-...` not found"
/v1, profile plus a model:          200 invalid  agent_profile_id / invalid_agent_profile / "An agent profile already specifies the model"
/v1, malformed id:                  200 invalid  agent_profile_id / uuid_parsing
/v1, no profile:                    200 valid    (no upstream call)
/v1, upstream error:                502 Unexpected response from OpenHands API for agent profiles
/v1/preset/prompt, with a profile:  200 invalid  agent_profile_id / extra_forbidden

Before the fix the first line answered agent_profile_id / extra_forbidden.

Deployed to the Replicated fleet VM shared-4 (clean install) with the OHE test procedure, together with the Canvas, extensions and chart changes:

• OpenHands-Cloud branch hieple/deploy-2026-10-01 (ac80547) is main plus:
  • the chart change from feat(automation): provision the KV store secret for Replicated installs OpenHands-Cloud#1326 (AUTOMATION_KV_SECRET);
  • automation pinned to sha-03687e7 (feat: support agent profiles in cloud mode #541; image revision label 03687e7);
  • agent-canvas pinned to sha-9f60167 (feat(automations): show templates on cloud backends and offer native git integrations OpenHands#17830; image revision label 9f601672d942), which ships the extensions npm package at feat(automations): run the automation templates on OpenHands Cloud and Enterprise extensions#712's head (bf7e3212);
  • EXTENSIONS_REF: hieptl/ohe-3160 under global.agentServerEnv, so sandboxes load the skills from the extensions branch.
• Published as Replicated release 1731 on channel hieple/deploy-2026-10-01. The fleet run succeeded: https://github.com/OpenHands/infra/actions/runs/36819742198
• Checked on https://app.shared-4.replicated.all-hands-testing.dev without signing in:
  • /server_info: app_version 1.67.0, sdk_version 1.49.6.
  • /api/automation/health and /api/automation/ready: 200. /api/automation/openapi.json reports version 1.16.0; the chart's default image is 1.15.1, so the pinned build is the one running.
  • The deployed /canvas/ bundle (278 assets) contains the script bundles of the extensions branch (CloudConversations, _deliver_cloud, the $AUTH_HEADER skill text) and the Canvas changes (mcp-native-panel, the cloud-providers-configured query).
• With an API key on that instance, running image sha-03687e7 (the first two commits of this PR): GET /api/automation/v1/capabilities listed agentProfiles, kvStore and webhookDelivery.
• Not checked on the instance: a preflight with a profile selected and an end-to-end run of a profile template. The follow-up commits (661b171 onwards) are not deployed there.
• Full suite after the follow-up commits, uv run python -m pytest tests: 1862 passed, 7 skipped, with the two tests that need an S3 endpoint deselected on the machine I used.

Notes

• Tests: TestAgentProfileInCloudMode in tests/test_router.py and five tests in tests/test_capabilities_router.py run creation, update, preflight and capabilities in cloud mode against a stubbed OpenHands API (agent_profiles_api fixture) that answers only the lookup the service should make. They cover the organization the lookup is made in, answers that are not the expected shape, a refused credential, updates that select or keep a profile, and a selection validated repeatedly.
• With a transaction already open (a template create, or an update that changes the profile), a lookup that misses the cache holds that connection for the duration of the upstream call.
• A profile deleted after the automation is created is not caught: the app server still falls back to default settings at run time. Failing on an explicit profile it cannot resolve is an app server change.
• Saved drafts (/v1/drafts) still cannot carry a profile; only preflight and direct creation accept one.
• Git sync imports still refuse a profile in cloud mode: the loop has no caller credential to check an id with, so validate_agent_profile_selection keeps that rule. A profile-backed automation created through the API in cloud mode is therefore exported to git but cannot be edited from it.
• The run side lives in feat(automations): run the automation templates on OpenHands Cloud and Enterprise extensions#712: the template scripts start profile-backed conversations through the OpenHands API when no Agent Server URL is given. Canvas side: feat(automations): show templates on cloud backends and offer native git integrations OpenHands#17830. Tracked in OHE-3160.

[github-actions]

Warning

Your comment is too long (maximum is 65536 characters), so the coverage report was not added. See the job log for how to reduce it.

[hieptl]

Merge order (OHE-3160)

This PR is one of four that together bring the automation templates to OpenHands Cloud and Enterprise. Please merge them in this order:

1. feat: support agent profiles in cloud mode #541 (this PR): agent profiles in cloud mode, and a setup preflight that accepts a selected profile
2. feat(automation): provision the KV store secret for Replicated installs OpenHands-Cloud#1326: the automation key-value store secret for self-hosted installs
3. feat(automations): run the automation templates on OpenHands Cloud and Enterprise extensions#712: template scripts and skills that run on Cloud and Enterprise
4. feat(automations): show templates on cloud backends and offer native git integrations OpenHands#17830: Canvas: templates on cloud backends, native git integrations

Required

• feat(automations): run the automation templates on OpenHands Cloud and Enterprise extensions#712 has to be merged and released to npm before feat(automations): show templates on cloud backends and offer native git integrations OpenHands#17830. Canvas currently pins the extensions npm package to #712's head commit by git URL for testing; that pin must be replaced with the released version before #17830 merges.

Recommended

• feat: support agent profiles in cloud mode #541 and feat(automation): provision the KV store secret for Replicated installs OpenHands-Cloud#1326 go first, so the backend already supports what Canvas then exposes. They do not depend on each other or on #712, and each can be merged on its own.
• feat(automations): show templates on cloud backends and offer native git integrations OpenHands#17830 goes last. Merged earlier, Canvas would show templates on cloud backends that the backend cannot run yet.

After merging

• automation needs a release, and the automation image tag in the OpenHands-Cloud chart (currently 1.15.1) bumped to it, for feat: support agent profiles in cloud mode #541 to reach Enterprise installs.
• Canvas needs a release, and the agent-canvas image tag in the chart (currently 1.24.0) bumped to it.

[all-hands-bot]

This review was posted by an AI agent (OpenHands).

Scope. This change belongs in OpenHands/automation: it extends the automation service's own capability advertisement, request validation, and preflight contract. It does not move agent/tool behavior, conversation handling, or Canvas UI into this repository, and the profile id continues to be passed through to the run rather than resolved here.

What I verified on d53b6b28c4dc1c1fb697f8e79490fa25ed53d188

• Current PR state re-read: head still matches, non-draft, open, label type: feat. No existing reviews or review threads. CI check runs on this exact head are green (unit-tests, backend, Build and Push Automation Image, PR title/description checks).
• uv run ruff check openhands/automation clean; uv run pyright on the four changed modules clean. Pure-logic paths of the new validator exercised directly (combination rejection, None-profile short-circuits). The DB-backed suite could not run in this sandbox (no Docker for the Postgres fixture); CI's unit-tests job covers it on this head.
• Cloud existence check uses the caller's own credential plus X-Org-Id set to user.org_id, i.e. the organization the automation is stored under, not a re-derived session org. The shared _upstream_headers helper keeps authentication and the new lookup consistent.
• Create checks the profile after the template lookup (a repeat enable stays one query and does not depend on the app server); update re-checks only a profile that differs from the current one; preflight strips agent_profile_id from the saved-draft shape and re-adds it for the creation model, then reports the existence verdict last so an upstream fault leaves the profile unjudged instead of discarding other verdicts. All three are consistent with the models creation actually uses.
• Upstream responses are handled explicitly: 401/403 passed through, other non-200 and unexpected-shape 200 answered as 502, unknown id as 422. Local mode is unchanged.

Notes (acknowledged in the PR, not blocking). A profile deleted after creation still falls back at run time on the app server; git-sync import continues to refuse a profile in cloud mode for lack of a caller credential; and a cache-missing lookup inside an open transaction holds that connection for the upstream call. These are documented trade-offs with a clear owner, and none is a correctness, security, or compatibility defect demonstrated on this head.

No material findings. ✅ APPROVED