Automatically discover scene compositions where Vision-Language Models (VLMs) make incorrect decisions.
REVELIO combines concepts (scene elements like "pedestrian", "rain", "chain barrier") using search algorithms to find adversarial scenes that cause VLM failures. The system supports:
- Driving domain: CARLA simulator with Scenic DSL
- Indoor safety domain: Image generation (CogView4, Flux, etc.)
Create a conda environment and activate it using:
conda env create -f environment.yml
conda activate revelio
Optional: For AV/driving experiments, if Scenic is not properly installed and is an unrecognized keyword by itself, then install Scenic from the official repository https://github.com/berkeleylearnverify/scenic from the commit e17e6a4d1a42173ebb0b4e2499722d835f4d3a50, which was used in our experiments. The run the following:
cd Scenic/
python -m pip install -e .
After running an experiment (whether completely or aborted in between), run ./cleanup.sh <SERVER_PORT> where <SERVER_PORT> can be anything like 12345.
# 1. Start server (requires CARLA-compatible GPU)
# VLM and image-gen pools must use different GPUs (defaults both use 0 and will fail validation).
GOOGLE_API_KEYS="key1,key2" VLM_DEVICES=0 IMAGE_GEN_DEVICES=1 uvicorn scenic_eval.server:app --port <SERVER_PORT>
# 2. Run experiment
python examples/run_driving_experiment.py# 1. Start server (disjoint GPU lists — adjust indices to your machine)
GOOGLE_API_KEYS="key1,key2" VLM_DEVICES=0 IMAGE_GEN_DEVICES=1 uvicorn scenic_eval.server:app --port <SERVER_PORT>
# 2. Run experiment
python examples/run_indoor_safety_experiment.py┌─────────────────┐ ┌──────────────┐ ┌─────────────────┐
│ Concepts │────▶│ Rules │────▶│ Scene Proposal │
│ (scene elements)│ │ (questions) │ │ (IR + question)│
└─────────────────┘ └──────────────┘ └────────┬────────┘
│
▼
┌─────────────────┐ ┌──────────────┐ ┌─────────────────┐
│ VLM Response │◀────│ Evaluator │◀────│ Emitter │
│ (pass/fail) │ │ (server) │ │ (Scenic/prompt)│
└─────────────────┘ └──────────────┘ └─────────────────┘
The /evaluate_indoor_concepts endpoint lets you evaluate any combination of indoor safety concepts against a VLM in a single call — no experiment setup needed.
GOOGLE_API_KEYS="key1,key2" \
AWS_BEARER_TOKEN_BEDROCK="<your-bedrock-bearer-token>" \
VLM_DEVICES=0 \
IMAGE_GEN_DEVICES=1 \
VLM_PYTHON_PATH=/path/to/env/bin/python \
python -m uvicorn scenic_eval.server:app --host 0.0.0.0 --port <SERVER_PORT>GOOGLE_API_KEYS— required for Gemini image generation (comma-separated if multiple)VLM_DEVICES— GPU for local HuggingFace VLMs (not needed for Bedrock/Gemini VLMs)IMAGE_GEN_DEVICES— GPU for image generationVLM_PYTHON_PATH— python path for the VLM subprocess (if separate env)
import requests
response = requests.post("http://<host>:<SERVER_PORT>/evaluate_indoor_concepts", json={
"concept_ids": ["pan_counter", "toddler_walking"],
"llm_name": "bedrock:us.anthropic.claude-haiku-4-5-20251001-v1:0",
"n_samples": 5,
})
print(response.json())| Field | Type | Default | Description |
|---|---|---|---|
concept_ids |
list[str] |
required | Concept IDs to compose into a scene (see table below) |
llm_name |
str |
required | VLM to evaluate with (see VLM options below) |
n_samples |
int |
5 |
Number of images to generate and evaluate |
image_generator |
str |
"gemini" |
Image generator: "gemini" or "cogview4" |
image_generator_model |
str |
"gemini-2.5-flash-image" |
Image generator model ID (leave unset to use default) |
thinking_level |
str |
"minimal" |
Gemini 3 thinking level: "minimal", "low", "medium", "high" |
quantization |
str |
"fp16" |
For local HF VLMs: "fp16", "int8", "int4" |
output_root |
str |
"runs_indoor_eval" |
Directory to save generated images and logs |
| Field | Description |
|---|---|
mean_failure |
Fraction of samples where VLM answered incorrectly (0.0 = all correct) |
correct |
Number of samples with correct VLM answer |
total |
Total samples evaluated |
rule_matched |
Safety rule that was selected for this concept combination |
question_used |
Exact question posed to the VLM |
scene_description |
Text prompt used for image generation |
kept_frames |
Paths to generated images |
model_used |
VLM model identifier |
llm_name |
Description |
|---|---|
bedrock:us.anthropic.claude-haiku-4-5-20251001-v1:0 |
Claude Haiku 4.5 via AWS Bedrock |
bedrock:us.anthropic.claude-sonnet-4-6 |
Claude Sonnet 4.6 via AWS Bedrock |
gemini:gemini-3-flash |
Gemini 3 Flash with thinking support |
For Bedrock: requires AWS_BEARER_TOKEN_BEDROCK env var set on the server.
For Gemini VLM: requires GOOGLE_API_KEYS or GOOGLE_API_KEY env var.
| Concept ID | What it adds to the scene |
|---|---|
kitchen |
Base kitchen scene (usually not needed — added automatically) |
mug_upright |
Upright mug — no spill hazard |
mug_tipped |
Tipped/knocked-over mug — potential liquid spill |
glass_upright |
Upright glass — no spill hazard |
glass_tipped |
Tipped glass — liquid spill on floor |
coffee_spill |
Coffee spilled on surface/floor |
water_spill |
Water spilled on surface/floor |
wet_floor |
Wet floor — slip hazard |
knife_edge |
Knife at counter edge — fall/cut risk |
knife_center |
Knife safely centered on counter |
knife_block |
Knife stored in knife block — safe |
cabinet_open |
Open cabinet with chemicals accessible |
chemicals |
Chemical bottles present |
child_reaching |
Child actively reaching for chemicals |
stove_on |
Stove on with active flame |
towel_near |
Towel hanging near stove — fire risk |
towel_burning |
Towel already on fire |
smoke |
Smoke present |
toddler_standing |
Toddler standing (low immediate risk) |
toddler_walking |
Toddler walking through scene |
child_running |
Child running — higher injury risk |
adult_reacting |
Adult reacting to hazard |
outlet_exposed |
Electrical outlet with missing cover |
outlet_wires |
Electrical outlet with exposed wires |
water_near_outlet |
Water near outlet — shock hazard |
oven_open_hot |
Hot oven door open and glowing |
pan_stove |
Hot pan on stove with handle sticking out |
pan_counter |
Hot pan on counter with handle sticking out |
child_near_oven |
Toddler reaching toward hot oven |
child_near_pan |
Toddler touching hot pan |
glass_floor |
Broken glass shards on floor |
glass_counter |
Broken glass shards on counter |
barefoot_near_glass |
Barefoot toddler near broken glass |
The system automatically selects the most relevant safety rule for the concept combination:
- Each concept adds tags to the scene (e.g.
pan_counteradds"hot_surface","handle_out") - Rules declare which tags they require (e.g.
hot_R0_panrequires"hot_surface"+"handle_out") - The rule with the most matched required terms wins
- The matched rule provides both the question asked to the VLM and the correct expected answer
curl -X POST http://localhost:<SERVER_PORT>/evaluate_indoor_concepts \
-H "Content-Type: application/json" \
-d '{
"concept_ids": ["glass_floor", "barefoot_near_glass"],
"llm_name": "bedrock:us.anthropic.claude-sonnet-4-6",
"n_samples": 5
}'curl http://localhost:<SERVER_PORT>/healthfrom scenic_builder import ExperimentRunner, ServerConfig, SearchConfig
from scenic_builder.unified_scene_system import Concept, Rule, Emitter
# 1. Define concepts
class MyConcept(Concept):
def apply(self, scene_ir):
scene_ir.entities.append({"type": "my_object"})
# 2. Define rules (generate questions)
class MyRule(Rule):
def check(self, scene_ir):
if has_dangerous_object(scene_ir):
return "Is it safe?", "No", "A" # question, answer, option
# 3. Define emitter (convert IR to artifact)
class MyEmitter(Emitter):
def lower_and_render(self, scene_ir):
return f"A scene with {scene_ir.entities}"
# 4. Run experiment
runner = ExperimentRunner(
concepts=[MyConcept()],
rules=[MyRule()],
emitter=MyEmitter(),
server_config=ServerConfig(endpoint="http://localhost:<SERVER_PORT>/run_scenic_eval"),
search_config=SearchConfig(max_scenes=100),
)
result = runner.run()- Random Search: Baseline, samples random concept combinations
- Beam Search: Keeps top-k scenes, expands with new concepts
- Thompson Sampling: Bayesian linear surrogate on concept features; exact discrete Thompson sampling over valid compositions
- hybrid_beam_gp_ts (
create_search_algorithm("hybrid_beam_gp_ts", ...)): Beam search forSearchConfig.beam_phase_scenesevaluations, then GP Thompson sampling on the same discrete pool (scenic_builder/hybrid_beam_gp_ts.py). Requires scikit-learn (pip install scikit-learn).
scenic_builder/ # Core framework
├── experiment_runner.py # Main orchestrator
├── search.py # BeamSearch, RandomSearch, ThompsonSampling
├── hybrid_beam_gp_ts.py # Optional: beam then GP Thompson (needs scikit-learn)
├── evaluator.py # Server communication
├── data_types.py # Data structures
├── driving_concepts.py # Driving domain concepts
├── driving_rules.py # Driving domain rules
├── scenic_emitter.py # CARLA/Scenic emitter
├── indoor_safety_*.py # Indoor safety domain
└── unified_scene_system.py # Base classes
scenic_eval/ # Evaluation server
├── server.py # FastAPI endpoint
├── evaluation_backend.py # CARLA and image-gen backends
├── vlm.py # VLM inference (Gemini, HuggingFace)
├── vlm_pool.py # Parallel VLM workers
└── scenic_worker.py # CARLA scene rendering
examples/ # Runnable experiments
├── run_driving_experiment.py
└── run_indoor_safety_experiment.py
| Variable | Description | Default |
|---|---|---|
GOOGLE_API_KEYS |
Comma-separated Gemini API keys | Required |
IMAGE_GEN_DEVICES |
GPU for CARLA/image-gen (must not overlap with VLM_DEVICES) |
0 |
VLM_DEVICES |
GPU for local HuggingFace VLM workers (must not overlap with IMAGE_GEN_DEVICES) |
0 |
SERVER_PORT |
Server port | 12345 |
CARLA_PORT |
CARLA RPC port | 2000 |
THINKING_LEVEL |
Gemini thinking level | minimal |
Experiments output:
runs_*/scenic_*/- Individual scene renders and logsresults/*/- Aggregated results with statistical tests
See docs/EVALUATION_SYSTEM_SUMMARY.md for detailed system documentation.