Skip to content

fix: resolve operator, agent networking, auth, alerting, and tracing issues - #26

Merged
NP-compete merged 2 commits into
mainfrom
fix/operator-agents-platform-fixes
Jun 1, 2026
Merged

fix: resolve operator, agent networking, auth, alerting, and tracing issues#26
NP-compete merged 2 commits into
mainfrom
fix/operator-agents-platform-fixes

Conversation

@NP-compete

Copy link
Copy Markdown
Owner

Summary

Fixes multiple issues preventing the platform from operating correctly:

  • Operator couldn't reconcile agents — wrong API group (arcana.ai vs arcana.io) and hardcoded namespace scope
  • Agent pods crash-looping — NetworkPolicy blocked PyPI egress, preventing pip install at startup
  • Studio UI returning 403/404 — missing auth headers on fetch calls, nonexistent /api/v1/mcp route, and ingress hijacking SPA routes
  • Alertmanager notifications silently dropped — webhook endpoint didn't exist
  • Traces lost across 14 services — OTEL endpoint missing http:// scheme prefix
  • Agents not independently addressable — no per-agent ingress; agents now get {name}.arcana.localhost.me

Changes

Operator (cmd/operator/main.go)

  • Fix GVR group arcana.aiarcana.io
  • Remove .Namespace("arcana") from List call for cluster-wide agent discovery

Agent Networking (cmd/mesh/k8s.go, cmd/mesh/handlers.go)

  • Add HTTPS egress rule (port 443) to agent-isolation NetworkPolicy
  • Add ingress rule allowing ingress-nginx namespace traffic on port 5002
  • New CreateIngress() method for per-agent subdomain ingress
  • Provisioning flow now creates Ingress with ARCANA_BASE_DOMAIN config

Studio Frontend (services/studio/src/)

  • Global window.fetch wrapper in main.tsx injects X-Arcana-Role and Authorization headers
  • Dashboard: /api/v1/mcp/api/v1/tools

API Gateway (cmd/api/)

  • Expanded viewer role to read skills, tools, models, connectors, rules, memory, costs, etc.
  • Added /api/v1/webhooks/alerts/ POST handler for Alertmanager

Ingress (deploy/backing/ingress.yaml)

  • Removed /agents/ path that conflicted with Studio SPA routing

Tracing (deploy/helm/*/values.yaml)

  • Added http:// scheme to OTEL_EXPORTER_OTLP_ENDPOINT across all 28 Helm charts

Test plan

  • Operator reconciles 5 agents, patches status to Provisioned
  • All 5 agent pods 1/1 Running after NetworkPolicy fix
  • All 29 platform services healthy (/api/v1/health)
  • Studio dashboard loads with 0 console errors
  • Agent detail page (/agents/{name}) loads without 403
  • Every API endpoint returns 200 (GET) or 405 (POST-only) — no 403s for viewer role
  • Alert webhook accepts POST: {"status":"accepted","severity":"critical"}
  • OTEL tracing: 0 No connection adapters errors across all Python services
  • Agents independently reachable at {name}.arcana.localhost.me:8080/health

…issues

Operator:
- Fix API group mismatch (arcana.ai → arcana.io) preventing CRD reconciliation
- Remove hardcoded namespace scope so operator discovers agents across all namespaces

Agent networking:
- Add HTTPS egress (port 443) to agent-isolation NetworkPolicy for PyPI access
- Add ingress rule allowing traffic from ingress-nginx to agent pods
- Create per-agent Ingress during provisioning for independent agent identity
- Add CreateIngress method to mesh K8s client

Studio frontend:
- Add global fetch wrapper injecting auth headers from localStorage
- Fix dashboard calling nonexistent /api/v1/mcp route (now uses /api/v1/tools)
- Remove /agents/ ingress rule that hijacked SPA client-side routing

API gateway:
- Expand viewer role RBAC to allow read access to platform resources
- Add /api/v1/webhooks/alerts/ endpoint for Alertmanager notifications

Tracing:
- Add http:// scheme prefix to OTEL_EXPORTER_OTLP_ENDPOINT across all 28 Helm charts
@NP-compete
NP-compete merged commit 59f6481 into main Jun 1, 2026
14 checks passed
@NP-compete
NP-compete deleted the fix/operator-agents-platform-fixes branch June 7, 2026 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant