A web application that combines a chat interface with browser automation capabilities using E2B sandbox environment.
This repository hosts the code for an autonomous coding assistant that can browse the web, gather documentation and ultimately execute code inside a container. The system is composed of a Next.js frontend and a FastAPI backend. The backend exposes an orchestrator agent that coordinates specialised agents such as browser automation and coding agents. Once documentation has been collected the orchestrator can start a containerised coding agent to run tasks autonomously.
The orchestrator will use a memory module (implemented with Azure AI Search and vector search) to recall past interactions and enrich new requests. Browser agents are launched via Anchor/E2B services to collect information from the web, while the coding agent runs inside a local or Azure container instance.
graph TD
ORCH[Orchestrator]
ORCH --> ANALYZE[Task Analyzer]
ORCH --> MEMORY[Memory Module]
ORCH --> CTRL[Agent Controller]
CTRL --> BROWSERS[Browser Agents]
CTRL --> CODEX[Codex Agent]
ORCH --> CONTAINERS[Container Manager]
The orchestrator analyses tasks, consults the memory module powered by Azure AI Search and dispatches work to browser or codex agents. The container manager launches coding agents in isolated environments.
graph TD
BROWSER[Browser Agent]
BROWSER --> NAV[Navigator]
BROWSER --> EXTRACT[Data Extractor]
BROWSER --> SYNC[State Sync]
SYNC --> ORCH[Orchestrator]
Browser agents navigate sites, gather content and synchronise their state back to the orchestrator.
graph TD
CODEX[Codex Agent]
CODEX --> PLAN[Task Planner]
CODEX --> GEN[Code Generator]
CODEX --> EXEC[Executor]
EXEC --> REPO[Repository Manager]
CODEX --> PR[PR Creator]
CODEX --> ORCH
The codex agent plans tasks, generates code, executes commands and can create pull requests for resulting changes.
.
├── frontend/ # Next.js frontend application
│ ├── components/ # React components
│ ├── pages/ # Next.js pages
│ └── styles/ # CSS styles
├── backend/ # FastAPI backend server
│ ├── app/ # Application code
│ └── tests/ # Test files
└── README.md # This file
- Next.js 14
- Chakra UI
- TypeScript
- Axios for API calls
- FastAPI
- Python 3.9+
- E2B Desktop
- Browser automation tools
- Node.js 18+
- Python 3.9+
- npm or yarn
- pip
- Clone the repository:
git clone <repository-url>
cd <project-directory>- Install backend dependencies:
cd backend
pip install -r requirements.txt- Install frontend dependencies:
cd frontend
npm install- Start the backend server:
cd backend
uvicorn app.main:app --reload- Start the frontend development server:
cd frontend
npm run devThe application will be available at http://localhost:3000
- A user submits a task through the chat interface in the frontend.
- The frontend sends the request to
/api/orchestratoron the FastAPI backend. - The orchestrator consults the memory module (Azure AI Search & vectors) for relevant past context.
- Depending on the task the orchestrator spins up browser agents to collect documentation and may launch the autonomous coding agent inside a container.
- Agents stream their findings back to the orchestrator which aggregates the documentation.
- Once documentation is complete the orchestrator can trigger a container running the coding agent to execute the desired changes and optionally open a pull request.
- The backend returns session information and live view URLs which the frontend displays to the user in real-time.
- Real-time chat interface
- Browser automation visualization
- Task history tracking
- Responsive design
- Fork the repository
- Create your feature branch
- Commit your changes
- Push to the branch
- Create a new Pull Request
This project is licensed under the MIT License - see the LICENSE file for details.