Re-architecture draft: multiple parallel workers, demo with web ui and JSON mode

[dmay]

Summary

This branch introduces infrastructure to maintain multiple parallel workers. This infrastructure is also agnostic of UI implementation, and allows to work with various UIs.

The ultimate goals of this effort are:

• give Q CLI an ultimate version of 'Delagate' tool, that would spawn background workers, executing delegated tasks in parallel
• allow easy swap between UI modes - structured (output everything in JSON), normal (similar to current CX), advanced (full blown TUI with panes, widgets, etc)
• allow to easily plug in web API provider, that could be used by external local web app, providing user a fancy UI torun one or more Q CLI instances in different folders

Demo

TBD

Structure

1: Core Execution Components

Session

Purpose: Central orchestrator managing all workers and jobs in the system.

Key Responsibilities:

• Creates and tracks workers with configured model providers
• Launches jobs by combining workers with tasks
• Manages job lifecycle (active/inactive tracking, cleanup)
• Provides shared resources (model providers, thread pools)

Key Methods:

• build_worker(name) - Creates new worker with model provider
• run_agent_loop(worker, input, ui_interface) - Launches agent loop task
• cancel_all_jobs() - Cancels all active jobs
• cleanup_inactive_jobs() - Removes old completed jobs (keeps last 3)
• wait_for_all_jobs() - Blocks until all jobs complete

Location: crates/chat-cli/src/agent_env/session.rs

---

Worker

Purpose: Complete AI agent configuration containing state, context, and model provider.

Key Components:

• id: Uuid - Unique identifier
• name: String - Human-readable name
• context_container: ContextContainer - Conversation history and context
• model_provider: Arc<dyn ModelProvider> - LLM communication interface
• state: WorkerStates - Current execution state
• last_failure: Option<String> - Error tracking

State Machine:

Inactive → Working → Requesting → Receiving → [Waiting/UsingTool]* → Inactive
                                                                   ↓
                                                             InactiveFailed

States:

• Inactive - Ready for new task
• Working - Preparing request
• Requesting - Sending to LLM
• Receiving - Streaming response
• Waiting - Awaiting user input
• UsingTool - Executing tool
• InactiveFailed - Task failed

Location: crates/chat-cli/src/agent_env/worker.rs

---

WorkerTask (Trait)

Purpose: Interface for executable work units that run on workers.

Contract:

#[async_trait]
pub trait WorkerTask: Send + Sync {
   fn get_worker(&self) -> &Worker;
   async fn run(&self) -> Result<(), eyre::Error>;
}

Implementations:

• AgentLoop - Main conversational agent loop (worker_tasks/agent_loop.rs)
• ProtoLoop - Demo/prototype task (demo/proto_loop.rs)

Location: crates/chat-cli/src/agent_env/worker_task.rs

---

WorkerJob

Purpose: Running instance combining Worker + Task + execution infrastructure.

Key Components:

• worker: Arc<Worker> - Worker executing the task
• worker_task: Arc<dyn WorkerTask> - Task being executed
• cancellation_token: CancellationToken - Cancellation mechanism
• task_handle: JoinHandle - Tokio task handle
• worker_job_continuations: Continuations - Completion callbacks

Lifecycle:

1. new() - Create job (not started)
2. launch() - Spawn async task
3. cancel() - Request cancellation
4. wait() - Block until completion
5. Continuations execute on completion

Job States:

• Active - Currently running
• Completed - Finished successfully
• Cancelled - User/system cancelled
• Failed - Error occurred

Continuations: Callbacks registered via worker_job_continuations.add_or_run_now() that execute when job completes. Used to chain actions (e.g., re-queue prompt after agent response).

Locations:

• crates/chat-cli/src/agent_env/worker_job.rs
• crates/chat-cli/src/agent_env/worker_job_continuations.rs

---

2: Model and Context Infrastructure

ModelProvider (Trait)

Purpose: Abstraction for LLM communication supporting streaming responses.

Contract:

#[async_trait]
pub trait ModelProvider: Send + Sync {
   async fn request(
       &self,
       request: ModelRequest,
       when_receiving_begin: Box<dyn Fn() + Send>,
       when_received: Box<dyn Fn(ModelResponseChunk) + Send>,
       cancellation_token: CancellationToken,
   ) -> Result<ModelResponse, eyre::Error>;
}

Key Types:

• ModelRequest - Contains prompt string
• ModelResponseChunk - Streaming chunks (text or tool requests)
• ModelResponse - Complete response with tool requests

Implementations:

• BedrockConverseStream - AWS Bedrock integration (model_providers/bedrock_converse_stream.rs)

Location: crates/chat-cli/src/agent_env/model_providers/model_provider.rs

---

ContextContainer

Purpose: Manages all contextual information available to a worker.

Structure:

pub struct ContextContainer {
   pub conversation_history: Arc<Mutex<ConversationHistory>>,
}

ConversationHistory:

• Stores alternating user/assistant messages
• push_input_message(content) - Add user message
• push_assistant_message(assistant) - Add assistant response
• get_entries() - Retrieve all messages

ConversationEntry:

• Represents single message (user OR assistant)
• Contains UserMessage or AssistantMessage

Location: crates/chat-cli/src/agent_env/context_container/

---

4: User Interface and web support

TBD