SentryML

Real-time prompt injection detection for LLM applications. Scan every user message before it reaches your model.

The security layer every LLM-powered app is missing.

---

Table of contents

• Prerequisites
• Project structure
• Running locally
• Deploying to GCP
• Environment variables
• SDK usage
• Security
• Architecture
• Docs

---

Docs

• docs/jailbreak-before-after.md — Example “before and after”: a DAN-style jailbreak prompt (before) and the model’s correct refusal (after). Useful for demos and judge briefings.

---

Prerequisites

• Python 3.11+ (API)
• Node.js 18+ and npm (dashboard)
• Google Cloud SDK (gcloud) — for deployment
• Firebase CLI (firebase) — for dashboard hosting
• Docker — not required for local run; Cloud Build uses it in the cloud

For local development with Firestore:

• A GCP project with Firestore and (optionally) Firebase enabled
• Either Application Default Credentials (gcloud auth application-default login) or a service account key file

---

Project structure

sentryml/
├── api/                 # FastAPI backend (scan, explainability, Firestore)
│   ├── app/
│   ├── .env.example
│   ├── Dockerfile
│   └── requirements.txt
├── dashboard/           # React + Vite UI (Firebase Hosting) — see dashboard/README.md
│   ├── src/
│   ├── public/
│   ├── .env.example
│   ├── firebase.json
│   └── package.json
├── infra/
│   ├── deploy.sh             # Full deploy (bash: API + dashboard)
│   ├── deploy-api.ps1        # API-only deploy (PowerShell)
│   ├── deploy-dashboard.ps1   # Dashboard deploy using .env.production (PowerShell)
│   └── gcloud-commands.md     # All gcloud/Firebase commands (project: nastwest-u26wck-618)
├── sdk/                  # Python SDK
└── js-sdk/               # JavaScript SDK

---

Running locally

1. API

cd api
pip install -r requirements.txt
cp .env.example .env
# Edit .env: set GEMINI_API_KEY, GOOGLE_CLOUD_PROJECT / FIREBASE_PROJECT_ID (and GOOGLE_APPLICATION_CREDENTIALS if not using gcloud ADC)
uvicorn app.main:app --reload --port 8000

• Health: http://localhost:8000/health
• OpenAPI docs: http://localhost:8000/docs

2. Dashboard

cd dashboard
npm install
cp .env.example .env.local
# Edit .env.local: set VITE_API_URL=http://localhost:8000 and your Firebase config (see dashboard/.env.example)
npm run dev

• App: http://localhost:5173

Without Firebase config, the dashboard still runs but uses mock/streaming data where Firestore would be used.

3. Optional: Firebase emulators

To develop against Firestore locally without touching production, use the Firebase Emulator Suite and point the API to the emulator (see Firestore emulator docs).

---

Deploying to GCP

We use one GCP project for both the API (Cloud Run) and the dashboard (Firebase Hosting): nastwest-u26wck-618.

Copy-paste reference: All gcloud and Firebase commands with this project ID are in infra/gcloud-commands.md.
Dashboard-only: See dashboard/README.md for local run and deploy using dashboard/.env.production.

0. Authenticate gcloud (do this first)

# Interactive browser login for gcloud CLI
gcloud auth login

# Application Default Credentials (for local API + Cloud Build: Firestore, Secret Manager)
gcloud auth application-default login

One-time setup

1. Set project and enable APIs

gcloud config set project nastwest-u26wck-618
gcloud services enable run.googleapis.com cloudbuild.googleapis.com firestore.googleapis.com secretmanager.googleapis.com artifactregistry.googleapis.com firebase.googleapis.com --project=nastwest-u26wck-618

2. Create Artifact Registry repo (if it doesn’t exist)

gcloud artifacts repositories create sentryml-api --repository-format=docker --location=europe-west2 --project=nastwest-u26wck-618

3. Create secrets (populate with real values)

# Create (first time)
echo -n "your-gemini-api-key" | gcloud secrets create GEMINI_API_KEY --data-file=- --project=nastwest-u26wck-618
echo -n "your-admin-key"      | gcloud secrets create ADMIN_API_KEY   --data-file=- --project=nastwest-u26wck-618
# If they already exist, add a new version:
# echo -n "new-value" | gcloud secrets versions add GEMINI_API_KEY --data-file=- --project=nastwest-u26wck-618

4. Firebase
   In Firebase Console, open project nastwest-u26wck-618 and get the web app config (API key, auth domain, project ID, storage bucket, messaging sender ID, app ID). You need these for dashboard/.env.production when deploying the dashboard.

---

Deploy API (Cloud Run)

Option A — Bash (Linux / macOS / WSL)

# From repo root
export PROJECT_ID=YOUR_PROJECT_ID
bash infra/deploy.sh

This script builds the API image, deploys to Cloud Run, and then builds/deploys the dashboard (see below). If you only want the API, run the build + deploy steps from the script manually and skip the dashboard part.

Option B — PowerShell (Windows)

# From repo root
.\infra\deploy-api.ps1

You may need to allow script execution once:

Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned

deploy-api.ps1 is hardcoded with PROJECT_ID=nastwest-u26wck-618 and REGION=europe-west2. To use another project/region, edit the variables at the top of the script.

After deploy, the script prints the Cloud Run URL (e.g. https://sentryml-api-xxxxx-ew.a.run.app). Use this as VITE_API_URL when building the dashboard.

---

Deploy dashboard (Firebase Hosting)

1. Set production env
   Create dashboard/.env.production with VITE_API_URL (your Cloud Run URL) and Firebase config. This file is gitignored. See dashboard/README.md for the full list of variables and how to get the API URL.

2. Build and deploy (from repo root; ensure you’ve run gcloud auth login and gcloud auth application-default login first)

.\infra\deploy-dashboard.ps1

Or manually: cd dashboard → npm install → npm run build → firebase use nastwest-u26wck-618 → firebase deploy --only hosting.

Dashboard URL: https://nastwest-u26wck-618.web.app

---

Environment variables

API (api/.env)

Variable	Required	Description
GOOGLE_CLOUD_PROJECT	Yes*	GCP project ID
FIREBASE_PROJECT_ID	Yes*	Same as project ID for Firestore
FIRESTORE_DATABASE_ID	No	Firestore DB name (default: sentryml)
GOOGLE_APPLICATION_CREDENTIALS	For local	Path to service account JSON
GEMINI_API_KEY	Yes	Google AI (Gemini) API key for narratives
ADMIN_API_KEY	No	Optional admin key for dashboard/admin endpoints
MODEL_PATH	No	Path to classifier model (default: ./models/sentryml-distilbert)
PORT	No	Server port (default: 8080)
ENVIRONMENT	No	development or production

Copy from api/.env.example and fill in values.

Dashboard (dashboard/.env.local or .env.production)

Variable	Required	Description
VITE_API_URL	Yes	SentryML API base URL (no trailing slash)
VITE_FIREBASE_*	For real data	Firebase web app config (see dashboard/.env.example)

Copy from dashboard/.env.example and fill in values.

---

SDK usage

Python

pip install -e ./sdk

import sentryml
sentryml.configure(api_key="sm_live_xxxx", api_url="http://localhost:8000")
result = sentryml.scan(prompt=user_message, app_id="my-chatbot")
if result.is_threat:
    return "Request blocked."

JavaScript
See js-sdk/README.md for installation and usage.

---

Security

• Secrets: All keys come from environment variables or GCP Secret Manager; never hardcoded.
• Logging: The API logs only scan_id, severity, and latency_ms; prompts and tokens are not logged.
• API keys: Stored as SHA-256 hashes only; the plain key is returned once and never persisted.
• SDK: Use X-API-Key over HTTPS; do not commit API keys to version control.

---

Architecture

• API (FastAPI): Semantic classifier (sentence-transformers) and optional fine-tuned DistilBERT; occlusion-based explainability; Gemini threat narratives; Firestore persistence; carbon estimate per scan.

Is the model tuned?

In the repo today: no. The API uses the semantic (zero-shot) classifier: pre-trained all-MiniLM-L6-v2 plus our attack archetypes (no training on our data). If you place a fine-tuned DistilBERT at api/models/sentryml-distilbert (or set MODEL_PATH), the API will use that instead and classification will be “tuned” for your labels.

Will tuning improve results? It can: a model fine-tuned on labeled prompt-injection data often gets better precision/recall and handles subtle phrasings. It helps most when you have a solid labeled dataset (e.g. from production logs) and retrain periodically as new attack styles appear. For demos, the current zero-shot + broad archetypes is usually enough; tuning is most valuable once you have real traffic and want to optimize thresholds and reduce false positives/negatives.

• Dashboard (React + Vite): Live threat feed, carbon metrics, team filter by app_id, Quick Start with SDK snippets and API key generation, 3D prompt similarity map.
• SDK (Python / JS): configure() and scan() / scan_async() / guard().

Policies (severity and attack type)

Per-app policies can restrict by severity and/or attack type:

• Block by severity: block_severity_at_or_above (e.g. HIGH → block HIGH and CRITICAL).
• Block by attack type: block_attack_types — always block specific types (e.g. data_exfiltration, jailbreak) regardless of score.
• Flag by severity: flag_severity_at_or_above (e.g. MEDIUM → flag MEDIUM and above).
• Flag by attack type: flag_attack_types — always flag (but don’t block) specific types even if severity is low.

Use the policies API (e.g. PUT /v1/policies with admin key) to set these per app_id.

---

License

MIT