Skip to content

Commit a135e29

Browse files
authored
docs(demo): nest evidence scorer in demo profiles (#686)
## Summary ### Why? Demo profiles still showed a sibling `predictor:` block, which the wired YAML no longer accepts. ### What? Use `defaults.scorer` with `type: evidence`, the RFC example factors (`landing` for the land-stage batch), and a nested heuristic `base` in fake/git/github profiles and the quickstart. ## Test Plan Docs and YAML only; config parse is covered on the wiring PR. ## Stack - #684 - #686 ⬅️
1 parent 4717e89 commit a135e29

4 files changed

Lines changed: 44 additions & 1 deletion

File tree

doc/howto/QUICKSTART.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,11 +74,20 @@ Every change writes all of its files into one folder under `demo/`, and `FOLDERS
7474

7575
Set it deliberately when you want a run to show one thing. `FOLDERS=1` puts every change in the same place, so the queue serializes the lot and each change speculates on the one before it. A number well above `COUNT` keeps them all apart, so they go out together.
7676

77-
How much speculation that turns into is capped by the queue's **build budget** — how many builds it may have occupying CI at once, counted across every in-flight batch rather than per batch. It defaults to 4 and is set per queue in the provider's `profiles.yaml`:
77+
How much speculation that turns into is capped by the queue's **build budget** — how many builds it may have occupying CI at once, counted across every in-flight batch rather than per batch. It defaults to 4 and is set per queue in the provider's `profiles.yaml`. The demo also sets **evidence scorer factors** there so speculation ranking revises the base price when a path passes or fails or a batch is merging or cancelling; omitting `factors` leaves every factor at `1`, which is a no-op and ranks on the nested base alone.
7878

7979
```yaml
8080
defaults:
8181
speculator: {buildBudget: 4}
82+
scorer:
83+
type: evidence
84+
factors:
85+
pathPassed: 10
86+
pathFailed: 0.3
87+
landing: 12
88+
cancelling: 0.1
89+
base:
90+
type: heuristic
8291

8392
queues:
8493
- name: demo-queue

service/submitqueue/demo/provider/fake/profiles.yaml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,18 @@ defaults:
2222
# 1 the queue explores one path at a time, and raising it lets it hedge more
2323
# of the outcomes it is waiting on. Four is the built-in default.
2424
speculator: {buildBudget: 4}
25+
# Example evidence factors from the outcome-scorer RFC. Omitted keys would
26+
# stay at 1 and ranking would stay on the base price; writing them out makes
27+
# the demo show how path results and batch state revise that price.
28+
scorer:
29+
type: evidence
30+
factors:
31+
pathPassed: 10
32+
pathFailed: 0.3
33+
landing: 12
34+
cancelling: 0.1
35+
base:
36+
type: heuristic
2537

2638
queues:
2739
# The queue `make demo-requests` and `make land` use by default.

service/submitqueue/demo/provider/git/profiles.yaml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,17 @@ defaults:
1212
buildRunner: {type: fake}
1313
# Serialize conservatively unless a queue says otherwise.
1414
analyzer: {type: all}
15+
# Same example factors as ../fake/profiles.yaml — see that file for why they
16+
# are written out rather than left at the neutral default of 1.
17+
scorer:
18+
type: evidence
19+
factors:
20+
pathPassed: 10
21+
pathFailed: 0.3
22+
landing: 12
23+
cancelling: 0.1
24+
base:
25+
type: heuristic
1526

1627
queues:
1728
# The queue `make demo-requests` and `make land` use by default. Serializes

service/submitqueue/demo/provider/github/profiles.yaml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,17 @@ defaults:
1010
changeProvider: {type: fake}
1111
buildRunner: {type: fake}
1212
analyzer: {type: all}
13+
# Same example factors as ../fake/profiles.yaml — see that file for why they
14+
# are written out rather than left at the neutral default of 1.
15+
scorer:
16+
type: evidence
17+
factors:
18+
pathPassed: 10
19+
pathFailed: 0.3
20+
landing: 12
21+
cancelling: 0.1
22+
base:
23+
type: heuristic
1324

1425
queues:
1526
- name: demo-queue

0 commit comments

Comments
 (0)