Skip to content

Worlds

Jump to bottom
bdambrosio edited this page Jan 9, 2026 · 1 revision

World Integrations

World integrations enable agents to interact with specific environments (e.g., Minecraft, OSWorld, ScienceWorld). Each world provides its own tools and state management.

Configuration

Worlds are enabled per character via world_config in scenario YAML:

world_config:
  world_name: "minecraft"  # Loads tools from src/world-tools/minecraft/
  state: '{"nav": [{"pose": {...}}]}'  # Initial world_state JSON string

World Tools

World-specific tools are loaded from src/world-tools/<world_name>/:

  • Each tool is a subdirectory with Skill.md and tool.py
  • Tools are loaded after core tools (so world tools can override core tool names if needed)
  • Tool catalog groups world tools under #<WORLD_NAME> header

World State

InfospaceExecutor maintains a persistent world_state dictionary for the active world:

Initialization:

  • Parsed from world_config.state JSON string on executor startup
  • If missing or invalid, initializes to empty dict {}

Access:

  • executor.get_world_state(field_name) - Get a top-level field
  • executor.set_world_state(field_name, value) - Set a top-level field
  • set_world_state automatically publishes updates to cognitive/{character}/world_state Zenoh topic

UI Display:

  • The FastAPI UI has a State tab that displays world_state formatted as JSON
  • Updates automatically when set_world_state is called

Init Tool

If a tool named init (or <world_name>-init) exists in src/world-tools/<world_name>/, it is automatically executed after executor initialization:

  • Runs via execute_action_with_log() (so it's logged and published)
  • Typically normalizes initial pose/state and seeds world_state
  • Example: minecraft/init normalizes agent pose to block center, cardinal yaw, pitch zero

Resources

World-specific resources are stored in scenarios/<world_name>/resources/:

  • Notes and Collections: resources.json
  • Vector indexes: *.faiss and *.meta files
  • World-specific data: e.g., Jill_spatial_map.json for Minecraft

Note: scenarios/*/resources/ is in .gitignore (runtime-generated).

Current Worlds

Minecraft

  • Tools: mc-status, mc-observe-blocks, nav-step, nav-turn, mc-map-update, etc.
  • State: world_state.nav is a list of navigation history (most recent first, bounded to 100)
  • Mapping: Uses SpatialMap for cell-based spatial memory
  • See Minecraft for detailed documentation

OSWorld

  • Tools: OS interaction primitives
  • State: TBD

ScienceWorld

  • Tools: Science experiment primitives
  • State: TBD

Adding a New World

  1. Create src/world-tools/<world_name>/ directory
  2. Add tools as subdirectories (each with Skill.md and tool.py)
  3. Optionally create init tool for state normalization
  4. Update scenario YAML with world_config.world_name
  5. Optionally define initial world_config.state JSON

World State Best Practices

  1. Initialize in init tool: Normalize initial state on startup
  2. Update on state changes: Tools that change world state should call set_world_state
  3. Use structured data: world_state fields should be JSON-serializable
  4. Document state schema: Document world_state structure in world-specific wiki page
  5. Keep state minimal: Only store persistent state, not ephemeral plan-local data

Clone this wiki locally