Skip to content

Add Test Connection to OpenAPI, Azure Maps, Blob Storage, Databricks, Log Analytics, MCP, Snowflake, and Tableau actions - #1269

Merged
Paul Lizer (paullizer) merged 1 commit into
microsoft:Developmentfrom
paullizer:paullizer-action-test-connection-buttons
Aug 18, 2026
Merged

Add Test Connection to OpenAPI, Azure Maps, Blob Storage, Databricks, Log Analytics, MCP, Snowflake, and Tableau actions#1269
Paul Lizer (paullizer) merged 1 commit into
microsoft:Developmentfrom
paullizer:paullizer-action-test-connection-buttons

Conversation

@paullizer

Copy link
Copy Markdown
Contributor

Fixes #1267

Summary

Only SQL, Cosmos DB, and Yamcs actions could be validated before saving. Everything else forced a save-then-fail-at-runtime loop: a wrong warehouse ID, container name, subscription key, PAT, or MCP endpoint only surfaced as a tool failure during a chat.

This adds a Test Connection button to eight more action types — openapi, azure_maps_openlayers, blob_storage, databricks, log_analytics, mcp, snowflake, tableau — following the existing SQL/Cosmos pattern.

Each test authenticates with the credentials entered in the modal and performs one lightweight read against the configured resource:

Type Verification
OpenAPI Parse the stored spec, report operation count, then authenticated GET to the base URL
Azure Maps Fetch tile 0/0/0 of microsoft.base.road with the subscription key
Blob Storage Read container properties, then list one blob under the configured prefix
Databricks GET /api/2.0/sql/warehouses/{id}; report warehouse name and state
Log Analytics LogsQueryClient.query_workspace(..., "print TestConnection = 1")
MCP McpPluginFactory.probe_server_from_config; report transport, auth method, tool count
Snowflake Connect, SELECT CURRENT_VERSION(), close
Tableau Sign in; report server version and site

Changes

Backend

  • New functions_action_connection_tests.py — one tester per type, plus shared error sanitization, result, and timeout helpers. Routes stay thin.
  • Eight POST /api/plugins/test-{type}-connection routes on bpap, matching the existing test-sql-connection naming, delegating through shared _prepare_action_test_manifest and _run_action_connection_test helpers.
  • Masked secrets and reusable workspace identities resolve server-side, so an existing action can be tested without retyping credentials.
  • The MCP route enforces the same stdio scope restriction and outbound destination policy as MCP tool discovery, and does not overwrite discovered tool metadata.

Frontend

  • One shared runner drives all eight buttons, so loading/success/failure/missing-field states are identical everywhere. Server text renders through DOM text nodes, not innerHTML.
  • Log Analytics gains a dedicated Step 3 section (Workspace ID, Cloud, custom authority/endpoint, auth block), replacing the generic endpoint form and the dynamically generated Step 4 fields. Stored additionalFields such as query_history are preserved on edit.
  • Extracted getOpenApiAuthConfiguration / getAzureMapsConfiguration / getBlobStorageConfiguration out of getFormData() so the save path and the test path share one source of truth.

Security

A code review of this PR surfaced a pre-existing cross-scope credential exfiltration path that the new routes would have widened, so it is fixed here.

_resolve_secret_value_for_plugin_test resolved any string that merely looked like a SimpleChat Key Vault reference, with no scope or source check — and that value came straight off the request body. Reference names are deterministic, so an authenticated user could submit another user's, group's, or global action's reference name and have the plaintext secret forwarded to an endpoint the same request controls.

  • The unscoped resolver is removed. resolve_secret_reference_for_context now has exactly one call site in the module, inside _resolve_secret_value_for_action_test, which requires an explicit scope and per-field allowed_sources (no defaults) and fails closed when the scope cannot be determined.
  • Scope is derived from the loaded action via _resolve_plugin_secret_context(existing_plugin, user_id), never from the request body.
  • Sources are per field, matching keyvault_plugin_get_helper: auth.*action, additionalFields.* and MCP custom headers → action-addset.
  • The existing MCP discovery, Cosmos, Yamcs, and SQL test paths now share the same scoped resolver.
  • Loading a global action for a connection test now requires the Admin role at the shared loader, so every test route inherits the check instead of relying on each route to gate it. This closes a path where a non-admin could derive the global secret scope through the Cosmos route.

Only affects deployments with Key Vault secret storage enabled. Normal editing is unchanged — testing an existing action still works without retyping stored credentials.

Additionally, no credential value is ever returned to the browser: sanitize_connection_error strips every literal manifest secret (including base64 forms) plus generic password/AccountKey=/SharedAccessSignature/Authorization patterns from all error text.

Testing

Functional (all passing locally)

  • test_action_test_connection_endpoints.py — all eight routes registered on bpap, POST-only, full decorator stack, delegate to a dedicated tester; MCP retains discovery's stdio + destination policy; secret references resolve only within the owning action's scope and source; the resolver chokepoint is single.
  • test_action_connection_test_secret_redaction.py — manifest secrets, generic credential patterns, and base64-encoded credentials are stripped; result helpers and the timeout clamp behave correctly.
  • test_action_test_connection_modal_wiring.py — markup exists for all eight types, each button maps to a registered route, results avoid innerHTML, Log Analytics is a structured config type and preserves query_history.

The source-mismatch guard is verified non-vacuous: flipping ACTION_AUTH_SECRET_SOURCES to the wrong value makes the test fail with a specific message.

UI (Playwright)

  • New test_workspace_action_test_connection_controls.py, parametrized over all eight types: warning state without required fields, success alert on a mocked 200, danger alert on a mocked 403, plus the submitted payload shape.
  • New test_workspace_log_analytics_action_modal.py covering the new section, conditional custom-cloud and service-principal fields, validation, and the saved manifest.
  • Updated the Azure Maps, Blob Storage, Databricks, MCP, and Tableau modal tests to assert the new control.

Also fixed: test_tableau_action_modal_workflow.py was already failing on Development — the Yamcs section added a second "Server URL" label, breaking that test's ambiguous get_by_label locators under Playwright strict mode. Those assertions are now scoped by id.

Guardrails: check_swagger_routes.py, check_broken_access_control.py, and check_xss_sinks.py (diff-scoped, as CI runs it) all pass. route_tests 12/12 pass.

Notes

  • config.py VERSION0.250.215.
  • Feature doc: docs/explanation/features/ACTION_TEST_CONNECTION.md.
  • No files under deployers/ changed, so deployers/version.txt is untouched.
  • Yamcs keeps its own dedicated test handler from Make workflow alerts conditional with severity-graded rules #1264; it was left as-is rather than folded into the shared runner to keep this PR's blast radius contained.

Only SQL and Cosmos DB actions could be validated before saving. A wrong
warehouse ID, container name, subscription key, PAT, or MCP endpoint only
surfaced as a tool failure during a chat.

Adds a Test Connection button to OpenAPI, Azure Maps, Blob Storage, Databricks,
Log Analytics, MCP, Snowflake, and Tableau actions. Each test authenticates with
the credentials entered in the modal and performs one lightweight read against
the configured resource, then reports an actionable success or failure.

Backend:
- New functions_action_connection_tests.py with one tester per action type plus
  shared error sanitization, result, and timeout helpers.
- Eight POST routes on the admin_plugins Blueprint, matching the existing
  test-sql-connection naming, each delegating through the shared
  _prepare_action_test_manifest and _run_action_connection_test helpers.
- Masked secrets and reusable workspace identities resolve server-side, so an
  existing action can be tested without retyping credentials.
- The MCP route enforces the same stdio scope restriction and outbound
  destination policy as MCP tool discovery, and does not overwrite discovered
  tool metadata.

Frontend:
- One shared test runner drives all eight buttons, so the loading, success,
  failure, and missing-field states are identical everywhere. Server text is
  rendered through DOM text nodes rather than innerHTML.
- Log Analytics gains a dedicated Step 3 configuration section, replacing the
  generic endpoint form and the dynamically generated Step 4 fields. Stored
  additionalFields such as query_history are preserved on edit.

Security hardening:
- Action Key Vault references are now resolved strictly within the owning
  action's scope and source. A reference name arrives in the request body, so
  the previous unscoped resolution would let an authenticated user read another
  user's, group's, or global action's secret and forward it to an endpoint they
  control. The unscoped resolver is removed, and MCP discovery, Cosmos, and SQL
  test paths now share the single scope-checked chokepoint.
- Loading a global action for a connection test requires the Admin role at the
  shared loader, so every test route inherits the check.

Fixes microsoft#1267

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@paullizer
Paul Lizer (paullizer) force-pushed the paullizer-action-test-connection-buttons branch from d8475c3 to 3bc606b Compare August 18, 2026 12:46
@paullizer
Paul Lizer (paullizer) merged commit b88ae40 into microsoft:Development Aug 18, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant