Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
155 commits
Select commit Hold shift + click to select a range
874da55
feat: add agentgateway and update caipe
sriaradhyula Sep 18, 2025
df2c570
fix(agentgateway): updates
sriaradhyula Sep 18, 2025
1ef4951
fix: remove incorrect command from agentgateway deployment
sriaradhyula Sep 18, 2025
12e3482
fix: add security context and resource limits for agentgateway
sriaradhyula Sep 18, 2025
66941ef
fix: simplify agentgateway config to avoid file watcher issues
sriaradhyula Sep 18, 2025
d1fc2a3
fix: run agentgateway without config file to avoid file watcher
sriaradhyula Sep 18, 2025
dab9fba
Fix agentgateway configmap to match quickstart documentation
sriaradhyula Sep 18, 2025
8e63be6
Remove probes and expose port 3000 for MCP proxy
sriaradhyula Sep 18, 2025
d74271e
Fix admin address to bind to all interfaces
sriaradhyula Sep 18, 2025
84b9874
Add missing nginx ingress annotations
sriaradhyula Sep 18, 2025
ff5f01d
Fix admin address binding using ADMIN_ADDR env var
sriaradhyula Sep 18, 2025
29d1dd8
Remove rewrite rule to fix CSS asset paths
sriaradhyula Sep 18, 2025
b70fbce
Fix Helm template error by providing minimal llmSecrets configuration
Sep 18, 2025
72a6ad2
Clean up unnecessary files - fix was in ArgoCD application config
Sep 18, 2025
36272d8
Update ai-platform-engineering to use override values for local Vault…
Sep 18, 2025
c28eb43
Fix ai-platform-engineering ingress to point to correct service name
sriaradhyula Sep 18, 2025
d364923
Add kb-rag ingress to expose web interface
sriaradhyula Sep 18, 2025
dd77059
Fix kb-rag ingress asset path issues with base href
sriaradhyula Sep 18, 2025
3fb1c09
Use subdomain approach for kb-rag ingress like vault
sriaradhyula Sep 18, 2025
89ccc07
feat: update agentgateway ingress to use subdomain
sriaradhyula Sep 18, 2025
54fc0a8
feat: add /mcp path routing to port 3000 for agentgateway
sriaradhyula Sep 18, 2025
5aab019
feat: update agentgateway to use remote MCP target
sriaradhyula Sep 18, 2025
804f0d1
fix: remove invalid adminAddr from agentgateway config
sriaradhyula Sep 18, 2025
84538c2
feat: reduce agentgateway replicas from 2 to 1
sriaradhyula Sep 18, 2025
4432c77
refactor: reorganize files into caipe folder structure with basic, co…
sriaradhyula Sep 19, 2025
9343682
feat: add more profiles
sriaradhyula Sep 21, 2025
b605124
fix: remove ai-platform-engineering use caipe
sriaradhyula Sep 21, 2025
e07842a
fix: update EXTERNAL_URL
sriaradhyula Sep 21, 2025
aa641f6
Add Backstage API token configuration
sriaradhyula Sep 21, 2025
cdb77f6
Add Backstage API token ExternalSecret and ArgoCD token CronJob
sriaradhyula Sep 21, 2025
c11565c
Fix vault logging and ArgoCD CronJob dependencies
sriaradhyula Sep 21, 2025
f8e1b97
Fix ArgoCD CronJob dependencies and apt source
sriaradhyula Sep 22, 2025
376feee
Fix ArgoCD CronJob to suppress vault token logging
sriaradhyula Sep 22, 2025
da412a3
Add ClusterRole for ArgoCD CronJob cross-namespace access
sriaradhyula Sep 22, 2025
02f6434
Add LLM credentials setup script and CAIPE README
sriaradhyula Sep 22, 2025
361a152
Fix README: Download script before running for interactive input
sriaradhyula Sep 22, 2025
2d374d6
Improve credential security in setup script
sriaradhyula Sep 22, 2025
f127d18
Add environment variable support with hints
sriaradhyula Sep 22, 2025
38dbb71
Update credential masking: show 5 chars, only mask API keys
sriaradhyula Sep 22, 2025
9543f99
Show full env var values for visible fields
sriaradhyula Sep 22, 2025
b759911
Strip newlines and whitespace from all input values
sriaradhyula Sep 22, 2025
0219cf0
Strip newlines from environment variables and improve input cleaning
sriaradhyula Sep 22, 2025
dec214c
Remove literal \n and \r strings from credentials
sriaradhyula Sep 22, 2025
64ceac3
Add ArgoCD API technical documentation
sriaradhyula Sep 22, 2025
a1223b1
Add Backstage catalog population script for ArgoCD deployments
sriaradhyula Sep 22, 2025
15c2d42
Fix ArgoCD API connectivity in Backstage catalog script
sriaradhyula Sep 22, 2025
ef09d6a
Add agent secrets setup script
sriaradhyula Sep 22, 2025
2346c74
Update CAIPE README with agent secrets setup instructions
sriaradhyula Sep 22, 2025
d5b8b4e
Update agent secrets script with all available agents from repository
sriaradhyula Sep 22, 2025
beb826c
Store agent secrets in individual Vault paths
sriaradhyula Sep 22, 2025
dbfa82a
Update agent secrets script with correct environment variable names
sriaradhyula Sep 22, 2025
1d8d6ee
Enhance newline stripping for copy/paste secrets
sriaradhyula Sep 22, 2025
af21a91
Add more aggressive newline cleaning for pasted secrets
sriaradhyula Sep 22, 2025
060269f
Fix newline issue by cleaning input immediately after read
sriaradhyula Sep 22, 2025
dfe1f7a
Use IFS= read -rs for better newline handling
sriaradhyula Sep 22, 2025
7a5c203
Fix multi-line secret input with read -d ''
sriaradhyula Sep 22, 2025
f108e0e
Add debug logging for input and vault commands
sriaradhyula Sep 22, 2025
ff38f8c
Ensure debug logs are committed
sriaradhyula Sep 22, 2025
0b296f5
Clean fix for single-line secret input
sriaradhyula Sep 22, 2025
c8e0bc1
Implement clean prompt_with_env function
sriaradhyula Sep 22, 2025
1301baf
fix(caipe): explicitly set image tag version 0.1.15
sriaradhyula Sep 22, 2025
b8d9025
Apply clean prompt_with_env function to LLM credentials script
sriaradhyula Sep 22, 2025
da8c56a
Add clean input handling notes to CAIPE documentation
sriaradhyula Sep 22, 2025
97b0239
Add script to refresh secrets and restart deployments
sriaradhyula Sep 22, 2025
6901a8f
fix(caipe): use agent specific images
sriaradhyula Sep 22, 2025
4710668
Add script to sync all ArgoCD applications
sriaradhyula Sep 22, 2025
46cb2ba
Update CAIPE documentation with utility scripts
sriaradhyula Sep 22, 2025
4db4d7d
Fix auth errors in Backstage catalog population script
sriaradhyula Sep 22, 2025
737fbe4
Add missing warn function to Backstage catalog script
sriaradhyula Sep 22, 2025
83940ba
Simplify Backstage catalog script to avoid ArgoCD API issues
sriaradhyula Sep 22, 2025
331827a
Debug and improve Backstage catalog script authentication
sriaradhyula Sep 22, 2025
0d60bc8
Fix Backstage script to use port 7007 directly
sriaradhyula Sep 22, 2025
ed5f31c
Add catalog-info.yaml for CAIPE components in Backstage
sriaradhyula Sep 22, 2025
2e21964
Update Backstage catalog script with working solution status
sriaradhyula Sep 22, 2025
3640d61
Add externalAccess configuration to Backstage for API authentication
sriaradhyula Sep 22, 2025
bf0fb5e
Update Backstage script to test working API access
sriaradhyula Sep 22, 2025
09fc43c
Add real CAIPE components from ArgoCD to Backstage catalog
sriaradhyula Sep 22, 2025
c682cf6
Fix Backstage catalog population to use location registration
sriaradhyula Sep 22, 2025
cc46676
Update CAIPE configurations to enable all agents
sriaradhyula Sep 22, 2025
7fb7770
Update caipe-complete-p2p to match upstream ai-platform-engineering
sriaradhyula Sep 22, 2025
4fd2de0
fix(caipe): update complete-p2p
sriaradhyula Sep 22, 2025
480d322
fix(caipe): update complete-p2p
sriaradhyula Sep 22, 2025
246bb86
fix(caipe): update complete-p2p
sriaradhyula Sep 22, 2025
1ab8069
feat: add workshop specific caipe stack
suwhang-cisco Sep 23, 2025
089f7f5
Merge pull request #2 from sriaradhyula/add_caipe_workshop
suwhang-cisco Sep 23, 2025
1a5855c
fix: fixes and improvements for caipe-basic-p2p
suwhang-cisco Sep 23, 2025
19fa59e
chore: add setup-ubuntu-prerequisites.sh
sriaradhyula Sep 23, 2025
41aa182
fix(setup-ubuntu-prerequisites.sh): updates
sriaradhyula Sep 23, 2025
d517a88
fix(setup-ubuntu-prerequisites.sh): updates
sriaradhyula Sep 23, 2025
f6e4669
fix(setup-ubuntu-prerequisites.sh): updates
sriaradhyula Sep 23, 2025
b82e139
fix(setup-ubuntu-prerequisites.sh): updates
sriaradhyula Sep 23, 2025
5ed1cf1
fix(setup-ubuntu-prerequisites.sh): updates
sriaradhyula Sep 24, 2025
1186052
fix(setup-ubuntu-prerequisites.sh): updates
sriaradhyula Sep 24, 2025
8af6a45
fix(setup-ubuntu-prerequisites.sh): updates
sriaradhyula Sep 24, 2025
36e391c
fix(setup-ubuntu-prerequisites.sh): updates
sriaradhyula Sep 24, 2025
02e7112
fix(setup-ubuntu-prerequisites.sh): updates
sriaradhyula Sep 24, 2025
f4e9db9
fix(setup-ubuntu-prerequisites.sh): updates
sriaradhyula Sep 24, 2025
e5598de
fix(setup-ubuntu-prerequisites.sh): updates
sriaradhyula Sep 24, 2025
d659191
fix(setup-ubuntu-prerequisites.sh): updates
sriaradhyula Sep 24, 2025
2c9afcc
fix(setup-ubuntu-prerequisites.sh): updates
sriaradhyula Sep 24, 2025
f6a94fc
fix(setup-ubuntu-prerequisites.sh): updates
sriaradhyula Sep 24, 2025
b5f22e1
fix(setup-ubuntu-prerequisites.sh): updates
sriaradhyula Sep 24, 2025
db8ac10
fix(refresh-secrets.sh): updates
sriaradhyula Sep 24, 2025
2736e10
fix(refresh-secrets.sh): updates
sriaradhyula Sep 24, 2025
57700a2
fix: disable graphrag for now as this will be deprecated soon
suwhang-cisco Sep 24, 2025
9a7f091
fix: wrong vault in caipe-complete-p2p...
suwhang-cisco Sep 24, 2025
ba84b02
fix(refresh-secrets.sh): updates
sriaradhyula Sep 24, 2025
31886cb
fix(refresh-secrets.sh): updates
sriaradhyula Sep 24, 2025
0b7e35e
feat: utilise .env file
suwhang-cisco Sep 24, 2025
91a3283
chore: remove caipe/sync-vault-secrets.sh as no longer needed
suwhang-cisco Sep 24, 2025
dd09aa8
fix: NOT AWS_REGION it is AWS_DEFAULT_REGION
suwhang-cisco Sep 24, 2025
08e1f47
fix: correct confluence secret ref
suwhang-cisco Sep 24, 2025
c66a56d
feat: allow llm-cred setup to use .env file. Also make aws remote mcp
suwhang-cisco Sep 24, 2025
4574618
fix: NOT KOMODOR_API_KEY but it is KOMODOR_TOKEN
suwhang-cisco Sep 24, 2025
65b2e83
chore: move all scripts inside caipe/scripts dir
suwhang-cisco Sep 24, 2025
67fb379
fix: correct splunk secrets
suwhang-cisco Sep 24, 2025
1294c93
fix: correct webex
suwhang-cisco Sep 24, 2025
b5dbcdc
fix: add fake MILVUS_SECRET and use official aws mcp image
suwhang-cisco Sep 24, 2025
bf69ca0
fix: remove ATLASSIAN_API_URL from agent-confluence
suwhang-cisco Sep 24, 2025
1e6d6a7
fix: set 0.1.15 for mcp and use stdio for mcp
suwhang-cisco Sep 24, 2025
bba9662
Merge remote-tracking branch 'origin/main' into sunny-fixes
suwhang-cisco Sep 24, 2025
7729c61
feat: update helm chart and simplify stacks
suwhang-cisco Sep 26, 2025
cf84c40
Merge pull request #3 from sriaradhyula/sunny-fixes
sriaradhyula Sep 27, 2025
34a4f0a
Merge pull request #4 from sriaradhyula/sunny-fixes-v2
sriaradhyula Sep 27, 2025
bf6f887
Merge pull request #5 from sriaradhyula/sunny-fixes
sriaradhyula Sep 27, 2025
103ad6d
fix(backstage-app): bump container with caipe
sriaradhyula Sep 27, 2025
183605b
fix: updates
sriaradhyula Sep 28, 2025
a23c9b7
feat(caipe): update ai-platform-engineering chart to 0.2.6
sriaradhyula Sep 28, 2025
b5e3f61
Merge pull request #6 from sriaradhyula/update-ai-platform-engineerin…
sriaradhyula Sep 28, 2025
8435ca8
Update install.yaml
sriaradhyula Sep 29, 2025
e1be3eb
fix: rename AWS_DEFAULT_REGION->AWS_REGION
sriaradhyula Sep 29, 2025
e132c11
fix: add more scripts
sriaradhyula Sep 29, 2025
073cc23
fix: add more scripts
sriaradhyula Sep 29, 2025
f07920a
Merge pull request #7 from sriaradhyula/update-ai-platform-engineerin…
sriaradhyula Sep 29, 2025
fabc4c6
Fix: Skip prompting when .env file exists
sriaradhyula Sep 29, 2025
3ce4db6
fix(caipe): update to 0.3.0 helm chart
sriaradhyula Oct 10, 2025
f716d43
Update values.yaml with all other LLM providers
suwhang-cisco Oct 15, 2025
e41af96
fix: apply suggestion from @sriaradhyula
sriaradhyula Oct 15, 2025
eb44e3b
Apply suggestion from @sriaradhyula
sriaradhyula Oct 15, 2025
983ef54
Apply suggestion from @sriaradhyula
sriaradhyula Oct 15, 2025
3fc8074
Apply suggestion from @sriaradhyula
sriaradhyula Oct 15, 2025
14d7657
Apply suggestion from @sriaradhyula
sriaradhyula Oct 15, 2025
53e666b
Delete caipe/scripts/setup-secrets.sh
sriaradhyula Oct 15, 2025
3c41729
Merge pull request #8 from sriaradhyula/update-caipe-0.3.0
sriaradhyula Oct 15, 2025
9416cd5
fix: update setup-ubuntu-prerequisites.sh
sriaradhyula Oct 16, 2025
8dee317
fix: update rag-webui
sriaradhyula Oct 16, 2025
9e78053
fix: update rag-webui
sriaradhyula Oct 16, 2025
ca8bd8e
fix(caipe): update backstage-app with caipe
sriaradhyula Oct 22, 2025
8626915
Add git CLI and GitHub CLI setup to Ubuntu prerequisites script
sriaradhyula Oct 22, 2025
d9db214
Configure Ubuntu setup script for non-interactive installation
sriaradhyula Oct 22, 2025
2d3fbea
Use working xstartup file for VNC i3 desktop
sriaradhyula Oct 22, 2025
65ddd05
Fix KUBECONFIG permission denied errors
sriaradhyula Oct 22, 2025
2db4805
fix(caipe): updated values
sriaradhyula Oct 22, 2025
309fcb8
Fix Firefox/i3 compatibility issues in Ubuntu setup script
sriaradhyula Oct 22, 2025
e80a9aa
Fix Neo4j storage and RAG ingress for KIND compatibility
sriaradhyula Oct 22, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.env
.DS_Store
237 changes: 237 additions & 0 deletions caipe/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,237 @@
# CAIPE - Cloud AI Platform Engineering

CAIPE provides AI-powered platform engineering capabilities with support for multiple LLM providers and automated secret management.

## Quick Start

### 1. Deploy CAIPE Platform

```bash
# Deploy the basic peer-to-peer configuration
./idpbuilder create --name caipe-basic-p2p \
--use-path-routing \
--package https://github.com/cnoe-io/stacks//ref-implementation \
--package https://github.com/sriaradhyula/stacks//caipe/caipe-basic-p2p
```

### 2. Setup LLM Credentials

Run the interactive setup script to configure your LLM provider:

```bash
# Make the script executable
chmod +x setup-llm-credentials.sh

# Run the setup script
./setup-llm-credentials.sh
```

**Note**: The script uses clean input handling - simply type or paste your credentials and press Enter. No special key combinations needed.

The script supports the following LLM providers:

#### Azure OpenAI
- API Key
- Endpoint URL
- API Version (default: 2024-02-15-preview)
- Deployment Name

#### OpenAI
- API Key
- Endpoint (default: https://api.openai.com/v1)
- Model Name (default: gpt-4)

#### AWS Bedrock
- Access Key ID
- Secret Access Key
- Region (default: us-east-1)
- Model ID (default: anthropic.claude-3-sonnet-20240229-v1:0)
- Provider (default: anthropic)

#### Google Gemini
- API Key
- Model Name (default: gemini-pro)

#### GCP Vertex AI
- Project ID
- Location (default: us-central1)
- Model Name (default: gemini-pro)

### 3. Setup Agent Secrets

Configure API keys and tokens for active agents:

```bash
# Make the script executable
chmod +x setup-agent-secrets.sh

# Run the setup script
./setup-agent-secrets.sh
```

**Note**: The script uses clean input handling - simply type or paste your credentials and press Enter. No special key combinations needed.

The script supports the following agents and their required credentials:

#### GitHub Agent
- **Personal Access Token**: GitHub API access token with repo permissions
- **Webhook Secret**: Optional secret for webhook validation

#### GitLab Agent
- **Personal Access Token**: GitLab API access token
- **Webhook Secret**: Optional secret for webhook validation

#### Jira Agent
- **API Token**: Jira API token for authentication
- **Base URL**: Jira instance URL (e.g., https://company.atlassian.net)
- **Username**: Jira username/email

#### Slack Agent
- **Bot Token**: Slack bot token (xoxb-...)
- **App Token**: Slack app token (xapp-...)
- **Signing Secret**: Slack signing secret for request verification

#### AWS Agent
- **Access Key ID**: AWS access key ID
- **Secret Access Key**: AWS secret access key
- **Region**: AWS region (default: us-east-1)

### 4. Access Services

After deployment, access the platform services:

- **ArgoCD**: https://cnoe.localtest.me:8443/argocd
- **Backstage**: https://cnoe.localtest.me:8443/backstage
- **Vault**: https://vault.cnoe.localtest.me:8443/ui
- **Gitea**: https://gitea.cnoe.localtest.me:8443

### 5. Verify Setup

Check that your credentials are properly stored:

#### LLM Credentials
1. Access Vault UI: https://vault.cnoe.localtest.me:8443/ui
2. Navigate to: `secret/ai-platform-engineering/global`
3. Verify your LLM provider configuration

#### Agent Secrets
1. Access Vault UI: https://vault.cnoe.localtest.me:8443/ui
2. Navigate to: `secret/ai-platform-engineering/agent-secrets`
3. Verify your agent API keys and tokens

## Architecture

CAIPE includes:

- **Vault**: Secret management with automated token rotation
- **ArgoCD**: GitOps deployment with API token automation
- **Backstage**: Developer portal with API authentication
- **External Secrets**: Kubernetes secret synchronization
- **AI Agents**: Platform engineering automation

## Security Features

- Automated ArgoCD API token generation and rotation
- Vault-based secret management
- Secure credential storage with encryption
- RBAC for cross-namespace access
- No secrets exposed in logs

## Utility Scripts

### Refresh Secrets and Restart Deployments

After updating secrets in Vault, use this script to refresh Kubernetes secrets and restart deployments:

```bash
# Make the script executable
chmod +x refresh-secrets.sh

# Run the refresh script
./refresh-secrets.sh
```

This script will:
- Check if Vault secrets exist and have data
- Delete Kubernetes secrets and wait for External Secrets to recreate them
- Restart corresponding deployments with rollout status verification

### Sync ArgoCD Applications

Ensure all ArgoCD applications are synced and healthy:

```bash
# Make the script executable
chmod +x sync-apps.sh

# Run the sync script
./sync-apps.sh
```

This script will sync:
- **backstage** - Developer portal
- **vault** - Secret management
- **argocd** - GitOps controller
- **ai-platform-engineering** - CAIPE stack
- **external-secrets** - Secret synchronization
- **ingress-nginx** - Ingress controller
- **gitea** - Git repository

## Raw Script Access

Download and run the setup scripts directly:

### LLM Credentials Setup
```bash
# Download the script
curl -sSL https://raw.githubusercontent.com/sriaradhyula/stacks/main/caipe/setup-llm-credentials.sh -o setup-llm-credentials.sh

# Make it executable and run
chmod +x setup-llm-credentials.sh
./setup-llm-credentials.sh
```

### Agent Secrets Setup
```bash
# Download the script
curl -sSL https://raw.githubusercontent.com/sriaradhyula/stacks/main/caipe/setup-agent-secrets.sh -o setup-agent-secrets.sh

# Make it executable and run
chmod +x setup-agent-secrets.sh
./setup-agent-secrets.sh
```

**Note**: Both scripts use clean input handling - simply type or paste your credentials and press Enter. No special key combinations needed.

### Utility Scripts

#### Refresh Secrets
```bash
curl -sSL https://raw.githubusercontent.com/sriaradhyula/stacks/main/caipe/refresh-secrets.sh -o refresh-secrets.sh
chmod +x refresh-secrets.sh
./refresh-secrets.sh
```

#### Sync Applications
```bash
curl -sSL https://raw.githubusercontent.com/sriaradhyula/stacks/main/caipe/sync-apps.sh -o sync-apps.sh
chmod +x sync-apps.sh
./sync-apps.sh
```

## Troubleshooting

### Prerequisites
- `kubectl` CLI installed and configured
- `vault` CLI installed
- Access to the CAIPE cluster

### Common Issues

1. **Vault connection failed**: Ensure port-forward is working and Vault is running
2. **Permission denied**: Check that you have access to the vault namespace
3. **Invalid provider**: Select a number from 1-5 for supported providers

### Support

For issues and questions, please refer to the [CNOE documentation](https://cnoe.io/docs/) or open an issue in the repository.
Original file line number Diff line number Diff line change
Expand Up @@ -11,19 +11,17 @@ spec:
# Main chart from GHCR
- chart: ai-platform-engineering
repoURL: ghcr.io/cnoe-io/helm-charts
targetRevision: 0.1.10
targetRevision: 0.3.3
helm:
parameters:
- name: tags.basic
value: "true"
valueFiles:
- $values/helm/values.yaml
- $values/helm/values-external-secrets.yaml
- $values/values.yaml
# Values files from Git repository (your custom branch)
- repoURL: https://github.com/cnoe-io/ai-platform-engineering.git
targetRevision: idpbuilder-values
- repoURL: cnoe://ai-platform-engineering
path: manifests
ref: values
# Raw manifests from Git
- repoURL: https://github.com/cnoe-io/stacks.git
targetRevision: main
path: ai-platform-engineering/ai-platform-engineering/manifests
destination:
server: https://kubernetes.default.svc
namespace: ai-platform-engineering
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ spec:
pathType: ImplementationSpecific
backend:
service:
name: ai-platform-engineering
name: ai-platform-engineering-supervisor-agent
port:
number: 8000
- host: cnoe.localtest.me
Expand All @@ -26,6 +26,6 @@ spec:
pathType: ImplementationSpecific
backend:
service:
name: ai-platform-engineering
name: ai-platform-engineering-supervisor-agent
port:
number: 8000
Loading
Loading