feat: restructure deployment into Kustomize base/overlays

[NP-compete]

Summary

WIP - This PR will be merged after a stable release.

• Restructures deployment/ from a flat layout into a proper Kustomize base/overlays pattern (base/, overlays/openshift/, overlays/full/)
• Adds production hardening: PgBouncer (separate Deployment for independent scaling), StatefulSets for Postgres and Redis, pod anti-affinity, topology spread constraints, PDBs, HPA, startup/liveness/readiness probes, preStop hooks, and restricted security contexts
• Adds OPA as a first-class component in the openshift overlay with its own Deployment, PDB, ConfigMap, and Secret
• Adds DATABASE_URL env var construction for LangGraph compatibility
• Adds ServiceAccount with automountServiceAccountToken: false
• Removes kind overlay and components directory, inlining resources directly into overlays
• Updates CI workflow to reference deployment/base for image tag updates

Overlay structure

Overlay	Use case
openshift	Production OpenShift with OPA, expects external DB/Redis
full	Self-hosted stack with Postgres + PgBouncer + Redis included

Test plan

• Verify kustomize build deployment/base renders correctly
• Verify kustomize build deployment/overlays/openshift renders correctly
• Verify kustomize build deployment/overlays/full renders correctly
• Validate agent-engine compatibility (deployment.zip processing)
• Deploy to a test OpenShift cluster with external DB/Redis
• Deploy full overlay to verify self-hosted Postgres/PgBouncer/Redis

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!