Skip to content

Commit 0ea7a77

Browse files
jmg-duartejmg-duarte
authored
docs: add onboarding diagrams and entrypoints (#4311)
# Description Addresses review feedback on #4296. New contributors need a single place to see how orderbook, autopilot, driver, solvers, Postgres, and the chain fit together, plus where to open the codebase for each area. This PR adds `docs/ONBOARDING.md` with Mermaid diagrams and practical code entrypoints, and links it from `README.md`. # Changes * Add `docs/ONBOARDING.md`: system context diagram, order→auction→settle sequence diagram, service responsibilities diagram, DB mental model diagram * Add "Where to start reading code" with entrypoints pointing to `run.rs` for each service (orderbook, autopilot, driver, solvers, contracts, database) * Add solver types section explaining colocated vs non-colocated distinction * Add local development quick-start (playground with `ETH_RPC_URL` note, cargo check/nextest, formatting) * Add debugging section linking to `COW_ORDER_DEBUG_SKILL.md` * Update `README.md` with a link to `docs/ONBOARDING.md` ## How to test 1. Open `docs/ONBOARDING.md` on GitHub and confirm all Mermaid blocks render (context flowchart, sequence diagram, responsibilities flowchart, DB flowchart) 2. Click links to `playground/README.md`, `database/README.md`, and `COW_ORDER_DEBUG_SKILL.md` and confirm they resolve 3. Open `README.md` and confirm the new onboarding link points to `./docs/ONBOARDING.md`
1 parent cfbec98 commit 0ea7a77

File tree

2 files changed

+207
-0
lines changed

2 files changed

+207
-0
lines changed

‎README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@
44

55
This repository contains backend code for [Cow Protocol Services](https://docs.cow.fi/) written in Rust.
66

7+
New to the repo? Start with the architecture + flow diagrams in [`docs/ONBOARDING.md`](./docs/ONBOARDING.md), then run the stack locally via the [`playground`](./playground/README.md).
8+
79
## Order Book
810

911
The `orderbook` crate provides the http api through which users (usually through a frontend web application) interact with the order book.

‎docs/ONBOARDING.md‎

Lines changed: 205 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,205 @@
1+
# Onboarding: CoW Protocol Services (this repo)
2+
3+
This repository is a Rust workspace containing the core backend services that run the CoW Protocol off-chain system:
4+
5+
- Users submit **signed orders** to the **orderbook**
6+
- **autopilot** periodically creates **auctions** and sends them to **drivers**
7+
- Each **driver** calls its **solver(s)** for solutions, then simulates + submits the winner to the on-chain **Settlement** contract
8+
9+
For local end-to-end development, use the playground stack. See `playground/README.md`.
10+
11+
## System context (big picture)
12+
13+
```mermaid
14+
flowchart LR
15+
U["User / Wallet"] -->|signed order| OB["orderbook"]
16+
UI["CoW Swap UI"] -->|quotes / orders| OB
17+
OB <--> DB[("PostgreSQL")]
18+
AP["autopilot"] <--> DB
19+
AP -->|auction| DR["driver"]
20+
AP -->|auction| CO["colocated solvers\n(own driver + solver)"]
21+
22+
DR -->|auction request| IS["internal solvers"]
23+
DR -->|auction request| ES["external solver APIs"]
24+
25+
DR -->|simulate + submit tx| CH[("EVM chain\n+ Settlement contract")]
26+
CO -->|submit tx| CH
27+
AP -->|index events| CH
28+
```
29+
30+
## Core "happy path" flow (order → auction → settlement)
31+
32+
```mermaid
33+
sequenceDiagram
34+
autonumber
35+
participant User
36+
participant Orderbook as orderbook
37+
participant DB as Postgres
38+
participant Autopilot as autopilot
39+
participant Driver as driver
40+
participant Solver as solver(s)
41+
participant Chain as EVM chain + Settlement contract
42+
43+
User->>Orderbook: GET /quote (fee + price estimate)
44+
Orderbook-->>User: quote response
45+
User->>Orderbook: POST /orders (signed order)
46+
Orderbook->>Orderbook: validate (signature, app-data, funding/approval, etc.)
47+
Orderbook->>DB: persist order + order events
48+
Orderbook-->>User: 201 Created (order uid)
49+
50+
loop every new block (~12s on mainnet)
51+
Autopilot->>DB: fetch eligible orders + state
52+
Autopilot->>Autopilot: apply policies (fees, filtering, scoring inputs)
53+
Autopilot->>DB: store current auction + competition metadata
54+
Autopilot->>Driver: send auction (or make it available for fetch)
55+
56+
Driver->>Solver: request solution(s) for auction
57+
Solver-->>Driver: proposed solution(s)
58+
Driver->>Driver: encode calldata + simulate
59+
Driver-->>Autopilot: submit bid(s)
60+
Autopilot->>Autopilot: rank bids, pick winner
61+
Autopilot->>Driver: tell winner to execute
62+
Driver->>Chain: submit transaction (time-bounded)
63+
Chain-->>Chain: execute settle(...) + emit events
64+
Autopilot->>Chain: fetch + index relevant events
65+
Autopilot->>DB: update competition/settlement tables + order events
66+
end
67+
```
68+
69+
## Solver types
70+
71+
- **Colocated**: External partners run their own driver + solver. Autopilot sends them the auction and they submit solutions independently. Full control, full responsibility.
72+
- **Non-colocated**: We run the driver, configured with their solver API endpoint. We handle simulation and submission on their behalf.
73+
74+
## Responsibilities by service (what to touch for what change)
75+
76+
```mermaid
77+
flowchart TB
78+
subgraph OB["orderbook"]
79+
OB1["HTTP API: orders, quotes, status"]
80+
OB2["Validations: signatures, app-data, funding/approval checks"]
81+
OB3["Persists orders + lifecycle events"]
82+
end
83+
84+
subgraph AP["autopilot"]
85+
AP1["Auction cutting: boundaries + inclusion"]
86+
AP2["Filtering + fee policies + scoring inputs"]
87+
AP3["Event indexing + competition metadata"]
88+
end
89+
90+
subgraph DR["driver"]
91+
DR1["Receive auction + fetch liquidity context"]
92+
DR2["Call internal/external solvers"]
93+
DR3["Encode solution to calldata, simulate"]
94+
DR4["Submit tx, manage retries/time window"]
95+
end
96+
97+
subgraph S["solvers"]
98+
S1["Pure routing/matching math"]
99+
S2["Return solution objects/calldata recipe"]
100+
end
101+
102+
subgraph DBL["database layer"]
103+
DBL1["migrations + schema"]
104+
DBL2["typed query helpers used by services"]
105+
end
106+
107+
AP --> DR
108+
OB --> DBL
109+
AP --> DBL
110+
DR --> S
111+
```
112+
113+
## Key crates (where shared logic lives)
114+
115+
- `crates/shared`: common utilities (order quoting/validation, fee logic, external prices, argument parsing)
116+
- `crates/price-estimation`: price estimation strategies (onchain, trade-based, native)
117+
- `crates/gas-price-estimation`: gas price estimation
118+
- `crates/database`: schema + DB helpers used by `orderbook` and `autopilot`
119+
- `crates/model`: API/data model types used across services
120+
- `crates/contracts`: Alloy-based contract bindings for on-chain interaction
121+
- `crates/ethrpc` + `crates/chain`: Ethereum RPC / chain interaction helpers
122+
- `crates/observe`: logging/metrics initialization helpers
123+
- `crates/app-data`: order app-data validation
124+
125+
## Where to start reading code (practical entrypoints)
126+
127+
Each service follows the same pattern: `main.rs` just sets up the allocator and calls `start()`, which lives in `run.rs`. The `run.rs` file parses CLI arguments, initializes logging/metrics, connects to the database and/or chain, and wires up the service components.
128+
129+
- **Orderbook (HTTP API + order validation)**
130+
- Start here: `crates/orderbook/src/run.rs` (initialization + service wiring)
131+
- Typical changes: API endpoints, order validation, quoting, DB writes
132+
133+
- **Autopilot (auction creation + policies + event indexing)**
134+
- Start here: `crates/autopilot/src/run.rs`
135+
- Typical changes: auction filtering/inclusion, fee policies, competition persistence, chain event indexing
136+
137+
- **Driver (simulation + settlement submission + solver integration)**
138+
- Start here: `crates/driver/README.md` for context, then `crates/driver/src/run.rs`
139+
- Typical changes: solver API integration, encoding/calldata, simulation logic, submission strategy
140+
141+
- **Solvers (internal solver engines)**
142+
- Start here: `crates/solvers/src/run.rs`
143+
- Typical changes: routing/matching math, solution generation
144+
145+
- **On-chain bindings**
146+
- Start here: `crates/contracts/README.md`
147+
- Typical changes: adding new contract artifacts/bindings, updating ABIs, exposing bindings in `lib.rs`
148+
149+
- **Database schema + migrations**
150+
- Start here: `database/README.md` (schema overview) and `crates/database` (query code)
151+
152+
- **End-to-end tests**
153+
- Start here: `crates/e2e/tests/e2e/` (individual test scenarios) and `crates/e2e/src/setup/` (test harness)
154+
- The e2e crate spins up a local Anvil node (optionally forking mainnet/Gnosis), deploys contracts, starts services (orderbook, autopilot, driver, solver), and runs full order→settlement flows
155+
- Tests are split into `local_node` (clean chain) and `forked_node` (forking a real network via `FORK_URL_MAINNET` / `FORK_URL_GNOSIS`)
156+
- Run local e2e tests: `cargo nextest run -p e2e local_node --test-threads 1 --failure-output final --run-ignored ignored-only`
157+
- Run forked e2e tests: `cargo nextest run -p e2e forked_node --test-threads 1 --run-ignored ignored-only --failure-output final`
158+
159+
## Local development (recommended path)
160+
161+
### Run the full stack (best for onboarding)
162+
163+
See `playground/README.md`. The short version is:
164+
165+
```bash
166+
docker compose -f playground/docker-compose.fork.yml up --build
167+
```
168+
169+
You'll need to set `ETH_RPC_URL` in `playground/.env` first (an Ethereum RPC endpoint for anvil to fork from). This starts a forked chain + Postgres + services + UI/Explorer components with live-reload behavior.
170+
171+
### Fast local compile / test loop (without running the stack)
172+
173+
- **Check**:
174+
175+
```bash
176+
cargo check --workspace --all-targets
177+
```
178+
179+
- **Unit tests (CI-compatible runner)**:
180+
181+
```bash
182+
cargo nextest run
183+
```
184+
185+
### Formatting and linting
186+
187+
This repo formats Rust code with nightly rustfmt (and TOML with Tombi). See `README.md` for the `just` commands.
188+
189+
## Database mental model (what’s in Postgres)
190+
191+
The DB stores orders, auctions, competitions, and indexed on-chain events.
192+
193+
- For a guided schema overview, see `database/README.md`.
194+
195+
```mermaid
196+
flowchart LR
197+
OB["orderbook"] -->|orders + order_events + quotes| DB[("Postgres")]
198+
AP["autopilot"] -->|auctions + competitions + event-derived tables| DB
199+
AP -->|fetch + index chain events| CH[("chain")]
200+
```
201+
202+
## Debugging "why didn’t my order trade"?
203+
204+
When you need to investigate an order lifecycle end-to-end (API → DB → logs → auction inclusion → solver bids → settlement),
205+
see [`COW_ORDER_DEBUG_SKILL.md`](./COW_ORDER_DEBUG_SKILL.md).

0 commit comments

Comments
 (0)