🚀 BlackRoad OS Deployment Notes

Version: 2.5 Date: 2025-11-18 Purpose: Production deployment checklist and reference

Pre-Deployment Checklist

✅ Prerequisites

GitHub repository: blackboxprogramming/BlackRoad-Operating-System
Railway account with project created
Cloudflare account with domains added
Domain registrar: GoDaddy (or other)
Railway CLI installed: curl -fsSL https://railway.app/install.sh | sh
Git configured and authenticated

Environment Variables

Required Variables

Backend Service (Railway)

# Core Settings
ENVIRONMENT=production
DEBUG=False
SECRET_KEY=<generate-with-openssl-rand-hex-32>
APP_NAME="BlackRoad Operating System"
APP_VERSION="1.0.0"

# Database (auto-injected by Railway)
DATABASE_URL=${{Postgres.DATABASE_URL}}
DATABASE_ASYNC_URL=${{Postgres.DATABASE_ASYNC_URL}}

# Redis (auto-injected by Railway)
REDIS_URL=${{Redis.REDIS_URL}}

# CORS (update with your domains)
ALLOWED_ORIGINS=https://blackroad.systems,https://os.blackroad.systems,https://blackroad.ai,https://www.blackroad.systems

# Public URLs
API_BASE_URL=https://blackroad.systems
FRONTEND_URL=https://blackroad.systems

# JWT Settings
ACCESS_TOKEN_EXPIRE_MINUTES=30
REFRESH_TOKEN_EXPIRE_DAYS=7
ALGORITHM=HS256

# Blockchain
WALLET_MASTER_KEY=<generate-with-openssl-rand-hex-32>

Optional Variables (Add as Needed)

# AI Integration
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...

# External Services
GITHUB_TOKEN=ghp_...
STRIPE_SECRET_KEY=sk_live_...
STRIPE_PUBLISHABLE_KEY=pk_live_...
TWILIO_ACCOUNT_SID=...
TWILIO_AUTH_TOKEN=...
SLACK_BOT_TOKEN=xoxb-...
DISCORD_BOT_TOKEN=...

# Monitoring
SENTRY_DSN=https://...@sentry.io/...

# Cloud Storage
AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...
AWS_S3_BUCKET=...
AWS_REGION=us-east-1

# Email
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_USER=...
SMTP_PASSWORD=...

Generate Secrets

# Generate SECRET_KEY
openssl rand -hex 32

# Generate WALLET_MASTER_KEY
openssl rand -hex 32

# Generate JWT secret (if separate)
openssl rand -hex 32

Railway Deployment

1. Create Railway Project

Via CLI:

# Login
railway login

# Create new project
railway init

# Link to GitHub repo
railway link

Via Web UI:

Go to https://railway.app/new
Click "Deploy from GitHub repo"
Select blackboxprogramming/BlackRoad-Operating-System
Choose "Deploy Now"

2. Add Database Services

PostgreSQL:

# Via CLI
railway add --plugin postgresql

# Or via Web UI:
# Project → New → Database → Add PostgreSQL

Redis:

# Via CLI
railway add --plugin redis

# Or via Web UI:
# Project → New → Database → Add Redis

3. Set Environment Variables

Via CLI:

# Set individual variables
railway variables set ENVIRONMENT=production
railway variables set DEBUG=False
railway variables set SECRET_KEY=<your-secret-key>
railway variables set ALLOWED_ORIGINS=https://blackroad.systems

# Or upload from file
railway variables set -f backend/.env.example
# (Edit values before uploading!)

Via Web UI:

Project → backend service → Variables
Click "New Variable"
Add each variable from list above
Save

Important: For DATABASE_URL and REDIS_URL, use Railway's variable references:

${{Postgres.DATABASE_URL}}
${{Redis.REDIS_URL}}

These auto-populate with connection strings.

4. Configure Build & Deploy

railway.toml (already in repo):

[build]
builder = "DOCKERFILE"
dockerfilePath = "backend/Dockerfile"

[deploy]
startCommand = "cd backend && uvicorn app.main:app --host 0.0.0.0 --port $PORT"
healthcheckPath = "/health"
healthcheckTimeout = 100
restartPolicyType = "ON_FAILURE"
restartPolicyMaxRetries = 10

Verify Build Settings:

Root directory: /
Dockerfile path: backend/Dockerfile
Start command: Auto-detected from railway.toml

5. Deploy Backend

Via CLI:

# Deploy from current branch
railway up

# Or deploy specific service
railway up --service backend

Via Web UI:

Push to main branch → automatic deployment
Or click "Deploy" button in Railway dashboard

6. Run Database Migrations

After first deployment:

# Connect to Railway service
railway run bash

# Inside Railway container:
cd backend
alembic upgrade head

# Or run directly:
railway run alembic upgrade head

7. Add Custom Domains

Via Railway Web UI:

Project → backend service → Settings → Networking
Under "Custom Domains", click "Add Domain"
Enter: blackroad.systems
Railway provides CNAME target (e.g., blackroad-os-production.up.railway.app)
Add this CNAME to Cloudflare (see next section)
Wait for SSL provisioning (automatic, 2-5 minutes)
Repeat for other domains/subdomains

Domains to add:

blackroad.systems (primary)
os.blackroad.systems (optional alias)
api.blackroad.systems (optional explicit API subdomain)

8. Verify Deployment

Health Check:

# Check Railway URL first
curl https://blackroad-os-production.up.railway.app/health

# Expected response:
{
  "status": "healthy",
  "timestamp": 1700000000,
  "environment": "production"
}

Check Logs:

# Via CLI
railway logs --service backend --tail 100

# Or in Web UI:
# Project → backend → Logs

Look for:

✅ "Starting BlackRoad Operating System Backend..."
✅ "Database tables created successfully"
✅ "Server running on production mode"
❌ No error stack traces

Cloudflare Configuration

1. Add Domain to Cloudflare

If not already added:

Cloudflare dashboard → "Add a site"
Enter: blackroad.systems
Choose: Free plan
Cloudflare scans existing DNS records

2. Update Nameservers at Domain Registrar

At GoDaddy (or your registrar):

Log in to domain management
Find blackroad.systems → DNS settings
Change nameservers from GoDaddy to Cloudflare
Cloudflare provides 2 nameservers (e.g., ns1.cloudflare.com, ns2.cloudflare.com)
Save changes
Wait 5-60 minutes for propagation

Verify:

dig NS blackroad.systems
# Should show Cloudflare nameservers

3. Configure DNS Records

For blackroad.systems zone:

Type	Name	Target	Proxy	TTL	Notes
CNAME	@	`blackroad-os-production.up.railway.app`	✅ Proxied	Auto	Root domain → Railway
CNAME	www	`blackroad.systems`	✅ Proxied	Auto	www redirect
CNAME	os	`blackroad.systems`	✅ Proxied	Auto	Optional alias
CNAME	api	`blackroad.systems`	✅ Proxied	Auto	Optional explicit API
CNAME	prism	`blackroad.systems`	✅ Proxied	Auto	Prism Console alias (future)
CNAME	docs	`blackboxprogramming.github.io`	✅ Proxied	Auto	GitHub Pages docs

Important:

All CNAMEs should be Proxied (orange cloud icon)
This enables Cloudflare CDN, SSL, DDoS protection
Use CNAME flattening for root domain (@) - Cloudflare does this automatically

Add DNS Records:

Cloudflare dashboard → DNS → Records
Click "Add record"
Fill in values from table above
Click "Save"
Repeat for each record

4. Configure SSL/TLS

Settings:

Cloudflare → SSL/TLS → Overview
Set encryption mode: Full (strict)
Enable:
- ✅ Always Use HTTPS
- ✅ Automatic HTTPS Rewrites
- ✅ Minimum TLS Version: 1.2

Edge Certificates:

Cloudflare → SSL/TLS → Edge Certificates
Verify:
- ✅ Universal SSL: Active
- ✅ Certificate status: Active
- ✅ Edge certificates: Covers all subdomains

5. Configure Page Rules (Optional)

Rule 1: Redirect www to apex

URL: www.blackroad.systems/*
Setting: Forwarding URL
Status: 301 - Permanent Redirect
Destination: https://blackroad.systems/$1

Rule 2: Cache static assets

URL: blackroad.systems/static/*
Settings:
- Cache Level: Cache Everything
- Edge Cache TTL: 1 month

Rule 3: Cache Prism assets

URL: blackroad.systems/prism/assets/*
Settings:
- Cache Level: Cache Everything
- Edge Cache TTL: 1 week

6. Verify DNS & SSL

Check DNS propagation:

# Check A/CNAME records
dig blackroad.systems
dig www.blackroad.systems
dig docs.blackroad.systems

# Check from different locations
# Use: https://dnschecker.org

Check SSL:

# Check certificate
curl -vI https://blackroad.systems 2>&1 | grep -i 'SSL\|TLS'

# Or visit in browser and click padlock icon

Test all routes:

# Main OS
curl -I https://blackroad.systems
# → 200 OK

# Prism
curl -I https://blackroad.systems/prism
# → 200 OK (after Prism UI is deployed)

# API Health
curl https://blackroad.systems/health
# → {"status": "healthy", ...}

# Docs
curl -I https://docs.blackroad.systems
# → 200 OK (after GitHub Pages is set up)

GitHub Pages (Documentation)

1. Build Documentation Locally (Test)

cd codex-docs

# Install MkDocs
pip install mkdocs mkdocs-material

# Test build
mkdocs build --strict

# Test locally
mkdocs serve
# Visit: http://localhost:8001

2. Configure GitHub Pages

Via GitHub Web UI:

Repository → Settings → Pages
Source:
- Branch: gh-pages (created by workflow)
- Folder: / (root)
Custom domain: docs.blackroad.systems
✅ Enforce HTTPS
Save

GitHub creates a CNAME file in gh-pages branch automatically.

3. Deploy Documentation

Automatic (via GitHub Actions):

Workflow: .github/workflows/docs-deploy.yml
Trigger: Push to main branch, or manual dispatch
Actions:
1. Checkout code
2. Install MkDocs + Material theme
3. Build docs: mkdocs build --strict
4. Deploy to gh-pages branch

Manual Deploy (if needed):

cd codex-docs

# Build and deploy
mkdocs gh-deploy

# This creates/updates gh-pages branch

4. Verify Documentation

# Check GitHub Pages URL first
curl -I https://blackboxprogramming.github.io/BlackRoad-Operating-System/

# Then check custom domain
curl -I https://docs.blackroad.systems/

GitHub Secrets (CI/CD)

Required Secrets

For Railway deployment:

# Get Railway token
railway login
railway whoami

# Add to GitHub:
# Settings → Secrets and variables → Actions → New repository secret
Name: RAILWAY_TOKEN
Value: <your-railway-token>

For Cloudflare automation (optional):

# Get Cloudflare API token
# Cloudflare dashboard → Profile → API Tokens → Create Token
# Template: Edit zone DNS

Name: CF_API_TOKEN
Value: <your-cloudflare-token>

Name: CF_ZONE_ID
Value: <zone-id-from-cloudflare-dashboard>

Set Secrets

Via GitHub Web UI:

Repository → Settings → Secrets and variables → Actions
Click "New repository secret"
Add each secret from above
Click "Add secret"

Via GitHub CLI:

gh secret set RAILWAY_TOKEN --body "<your-token>"
gh secret set CF_API_TOKEN --body "<your-token>"
gh secret set CF_ZONE_ID --body "<zone-id>"

Monitoring & Maintenance

Health Checks

Automated (Railway):

Railway checks /health endpoint every 30 seconds
Auto-restarts on failure
Sends alerts (configure in Railway dashboard)

Manual:

# Quick health check
curl https://blackroad.systems/health

# Detailed check
curl https://blackroad.systems/api/docs
# Should show OpenAPI docs

Logs

Railway Logs:

# Tail logs
railway logs --service backend --tail 100

# Or in Web UI:
# Project → backend → Logs

Cloudflare Analytics:

Dashboard → Analytics
Check: Requests, bandwidth, threats blocked

Database Backups

Railway automatic backups:

Daily snapshots (free tier: 7 days retention)
To restore: Railway dashboard → Postgres → Backups

Manual backup:

# Export database
railway run pg_dump $DATABASE_URL > backup-$(date +%Y%m%d).sql

# Import database (if needed)
railway run psql $DATABASE_URL < backup-20251118.sql

Cost Monitoring

Railway usage:

# Check current usage
railway usage

# Or in Web UI:
# Project → Usage

Cloudflare analytics:

Free tier: Unlimited requests
Monitor: SSL/TLS usage, cache hit ratio

Rollback Procedures

Railway Rollback

Via Web UI:

Project → backend → Deployments
Find previous successful deployment
Click "..." → Rollback
Confirm

Via CLI:

railway rollback --service backend

Database Rollback

Via Alembic:

# Rollback last migration
railway run alembic downgrade -1

# Rollback to specific version
railway run alembic downgrade <revision>

DNS Rollback

If needed:

Cloudflare dashboard → DNS → Records
Edit CNAME record
Point to previous target
Save (propagates in ~2 minutes due to proxy)

Troubleshooting

Issue: Railway Deployment Fails

Check:

# View build logs
railway logs --service backend

# Common issues:
# - Missing environment variables
# - Dockerfile syntax error
# - Dependencies not installing

Fix:

Verify all required env vars are set
Test Dockerfile locally: docker build -t test backend/
Check requirements.txt for typos
Re-deploy: railway up

Issue: Database Connection Errors

Check:

# Verify DATABASE_URL is set
railway variables | grep DATABASE

# Test connection
railway run psql $DATABASE_URL

Fix:

Ensure PostgreSQL plugin is added
Use ${{Postgres.DATABASE_URL}} reference
Check database is running: Railway dashboard → Postgres

Issue: CORS Errors in Browser

Check:

# Verify ALLOWED_ORIGINS
railway variables | grep ALLOWED_ORIGINS

Fix:

Add your domain to ALLOWED_ORIGINS
Include both https://blackroad.systems and https://www.blackroad.systems
No trailing slashes!
Redeploy: railway up

Issue: Cloudflare 522 Error

Cause: Origin (Railway) is down

Check:

# Test Railway directly
curl https://blackroad-os-production.up.railway.app/health

Fix:

Check Railway logs for errors
Verify health endpoint works
Restart service if needed
Wait 1-2 minutes for Cloudflare to detect

Issue: GitHub Pages Not Updating

Check:

# Check workflow status
gh run list --workflow=docs-deploy.yml

# View workflow logs
gh run view <run-id> --log

Fix:

Verify workflow ran successfully
Check gh-pages branch was updated
Check GitHub Pages settings: correct branch/folder
Wait 5-10 minutes for deployment
Hard refresh browser (Ctrl+F5)

Security Checklist

Pre-Production

All secrets use environment variables (not hardcoded)
SECRET_KEY is unique and strong (32+ hex chars)
DEBUG=False in production
HTTPS enforced (Cloudflare "Always Use HTTPS")
CORS restricted to specific domains (not *)
Database uses strong password (Railway auto-generates)
API rate limiting enabled (TODO: add middleware)
Sentry error monitoring configured (optional)

Post-Production

Monitor Railway logs for errors
Monitor Cloudflare for attack attempts
Enable GitHub Dependabot for vulnerability alerts
Regular database backups (Railway automatic)
Test disaster recovery (restore from backup)

Quick Reference

Production URLs

Service	URL	Source
Main OS	https://blackroad.systems	Railway (backend/static/)
Prism Console	https://blackroad.systems/prism	Railway (backend/static/prism/)
API Docs	https://blackroad.systems/api/docs	Railway (OpenAPI)
Health Check	https://blackroad.systems/health	Railway (endpoint)
Documentation	https://docs.blackroad.systems	GitHub Pages (codex-docs/)

Development URLs

Service	URL	Source
Main OS	http://localhost:8000	Local backend
Prism Console	http://localhost:8000/prism	Local backend
API Docs	http://localhost:8000/api/docs	Local backend
Health Check	http://localhost:8000/health	Local backend
Documentation	http://localhost:8001	`mkdocs serve`

Important Commands

# Railway
railway login
railway link
railway up
railway logs --tail 100
railway variables
railway status

# Cloudflare (via API - optional)
# Use Web UI for most tasks

# GitHub Pages
cd codex-docs
mkdocs build --strict
mkdocs serve
mkdocs gh-deploy

# Database
railway run alembic upgrade head
railway run alembic downgrade -1
railway run pg_dump $DATABASE_URL > backup.sql

Post-Deployment Checklist

After completing deployment:

Where AI meets the open road. 🛣️

Production deployment notes for BlackRoad OS Phase 2.5

Uh oh!

FilesExpand file tree

DEPLOYMENT_NOTES.md

Latest commit

History

DEPLOYMENT_NOTES.md

File metadata and controls

🚀 BlackRoad OS Deployment Notes

Pre-Deployment Checklist

✅ Prerequisites

Environment Variables

Required Variables

Optional Variables (Add as Needed)

Generate Secrets

Railway Deployment

1. Create Railway Project

2. Add Database Services

3. Set Environment Variables

4. Configure Build & Deploy

5. Deploy Backend

6. Run Database Migrations

7. Add Custom Domains

8. Verify Deployment

Cloudflare Configuration

1. Add Domain to Cloudflare

2. Update Nameservers at Domain Registrar

3. Configure DNS Records

4. Configure SSL/TLS

5. Configure Page Rules (Optional)

6. Verify DNS & SSL

GitHub Pages (Documentation)

1. Build Documentation Locally (Test)

2. Configure GitHub Pages

3. Deploy Documentation

4. Verify Documentation

GitHub Secrets (CI/CD)

Required Secrets

Set Secrets

Monitoring & Maintenance

Health Checks

Logs

Database Backups

Cost Monitoring

Rollback Procedures

Railway Rollback

Database Rollback

DNS Rollback

Troubleshooting

Issue: Railway Deployment Fails

Issue: Database Connection Errors

Issue: CORS Errors in Browser

Issue: Cloudflare 522 Error

Issue: GitHub Pages Not Updating

Security Checklist

Pre-Production

Post-Production

Quick Reference

Production URLs

Development URLs

Important Commands

Post-Deployment Checklist