feat(client): show request log events, and keep a watch inside the window

## Summary

### Why?

Two things were wrong with what a watch showed, and one of them got worse the more there was to show.

**Events were dropped.** A request's history holds two kinds of entry: a **status** is a position it reached — `batched`, `speculating`, `landed` — and an **event** is something that happened while it sat at one, like a build starting or a passed path waiting on a dependency. `digest` skipped every entry whose status was empty, which is exactly what an event is. Two requests that both read `speculating → speculated → landed` could have done wildly different amounts of work, one build or eight, and the table said the same thing about both.

**`list` showed less still.** It reads the queue's receipts rather than a history per request, so its trail was empty and its `STAGE` column rendered `…` for every row — including rows whose current status it already had in hand.

**And a watch of more requests than the window holds repainted the whole screen on every redraw.** The renderer never knew how tall the terminal was: `terminalSize` asked for the size and threw the height away, so a frame could be any number of lines. Each redraw moves the cursor up by the number of lines it emitted, on the assumption they are still on screen — and once a frame is taller than the window, they are not. Drawing it scrolls the window, the cursor cannot move above its first row, and the terminal clamps the jump; every frame after that starts from the wrong origin, overwriting the top of the screen and stranding the rest of the previous frame below. The threshold is the window height, which is why it looked like a size problem rather than a bug.

### What?

**Events are shown against the status they happened under**, rather than as steps of their own, because they are not positions and treating them as such would imply the request moved:

```
batched → speculating [building ×8, built ×8, waiting] → speculated → landing → landed
```

Repeats are counted rather than listed. A batch runs one build per speculation path, so a request that speculated widely records `building` many times over, and spelling each one out would say less than the count does while pushing the rest of the row off the line.

**`list` shows the position each request holds** instead of `…`. It still fetches no histories — that is what keeps a listing one round trip — so it reports where a request is without claiming to know how it got there.

**The renderer reads the height along with the width**, re-reading both before every draw so a window resized mid-watch is picked up, and trims a frame that will not fit. Settled requests are dropped first: they have stopped changing, so leaving them out costs a reader nothing `land-list` will not tell them, while dropping the ones still moving would hide the only part of the table that is doing anything. Order is kept within each group so rows do not jump between redraws, and a line reports how many are not shown.

Two cases deliberately keep every row, because nothing is drawn over them and the terminal's own scrollback is the right answer: redirected output, which is a log rather than a window, and a one-shot `list`, which is drawn once and scrolled back through.

## Test Plan

- ✅ `make demo-requests COUNT=4 FOLDERS=1` against a live stack, which forces every change to conflict and so produces real speculation. The deepest request in the chain rendered `speculating [building ×8, built ×8, waiting]` while the first rendered `speculating [building, built]` — the difference this exists to show
- ✅ `make land-list` mid-flight reports `speculating` where it used to report `…`, and `landed` once the run settled
- ✅ 30 requests in a simulated 20-row window: the frame is 19 lines and the cursor moves back 19. The same run emitted over 60 lines into that window before
- ✅ `make land-list LIMIT=30` in that same 20-row window still prints all 30 rows
- ✅ new `digest` cases: events attach to the status they occurred under, repeats are counted, a status repeated around its own events stays one step, each status collects only its own events, an event does not move the request off its status, an event before any status is dropped, and an error carried by an event is still surfaced
- ✅ new frame cases: a frame never exceeds the window and `lastLines` matches what was emitted; an unsettled row survives the trim; piped output and one-shot listings keep every row
- ✅ mutation-checked the height test by making the trim a no-op, confirming it fails without the fix rather than passing vacuously
- ✅ `make test` (105 targets), `make lint`, `make gazelle`

Author: behinddwalls

doc/howto/QUICKSTART.md

@@ -105,7 +105,16 @@ make land-list SINCE=24h LIMIT=200   # a wider window
 make land-watch                      # follow them until they settle
 ```
 
-Both draw the same table `make demo-requests` does — the demo tool and the CLI share it — but against whatever the queue already holds, so watching a queue no longer means adding to it. They do not carry the same information, though: `list` is a one-shot read of the queue's receipts and does not fetch histories, so its `STAGE` column is always `…`, while `watch` follows the history API and fills the trail in as each request moves.
+Both draw the same table `make demo-requests` does — the demo tool and the CLI share it — but against whatever the queue already holds, so watching a queue no longer means adding to it. They do not carry the same information, though: `list` is a one-shot read of the queue's receipts and does not fetch a history per request, so its `STAGE` column shows where each request is and not how it got there, while `watch` follows the history API and fills the whole trail in as each one moves.
+
+That trail carries more than positions. A request records events while it sits at one — a build starting or finishing, a passed path waiting on a dependency — and those are shown against the status they happened under, with repeats counted:
+
+```
+accepted → started → validating → validated → batching → batched →
+speculating [building ×8, built ×8, waiting] → speculated → landing → landed
+```
+
+Eight builds means the batch was speculating down eight paths at once, and `waiting` means one of them passed and then sat on a dependency that had not resolved. A request that sailed through reads `speculating [building, built]` instead — the same position, a very different amount of work behind it.
 
 `land-watch` fixes its set when it starts and exits non-zero if any request in that set finishes anywhere other than `landed`, which makes it usable from a script. A request accepted after the watch begins is not picked up: a watch that grew as the queue did would never finish.
 

submitqueue/client/view.go

@@ -116,13 +116,19 @@ func (rw *Row) elapsed() string {
 	return fmt.Sprintf("%ds", int(end.Sub(rw.Submitted).Seconds()))
 }
 
-// stage is the path the request has taken, as the gateway recorded it. The
-// waiting marker covers the gap between acceptance and the first recorded
-// event, so an accepted request is never shown as though nothing happened.
+// stage is the path the request has taken, as the gateway recorded it.
+//
+// A one-shot listing reads the queue's receipts and does not fetch a history
+// per request, so it has the position each one holds but not how it got there.
+// That is worth showing on its own: a column of "…" says nothing about a queue
+// whose rows are mostly `speculating`.
 func (rw *Row) stage() string {
 	if len(rw.Trail) > 0 {
 		return strings.Join(rw.Trail, " → ")
 	}
+	if rw.Status != "" {
+		return rw.Status
+	}
 	if rw.SQID != "" {
 		return "…"
 	}
@@ -133,34 +139,110 @@ func (rw *Row) stage() string {
 // a caller following requests as they move uses a Tracker instead, which owns
 // the rows and redraws them.
 func Draw(rows []*Row, status string) {
-	newRenderer().draw(rows, status)
+	r := newRenderer()
+	// Nothing will be drawn over this, so a table taller than the window simply
+	// scrolls — which is what a reader of a long listing wants, and why the
+	// height limit a redrawing watch lives under does not apply here.
+	r.oneShot = true
+	r.draw(rows, status)
 }
 
 // digest reduces a request's recorded history to the trail worth showing, the
 // status it currently holds, and the error the latest event carried. A status
 // recorded more than once in a row is one step in the trail, not several.
+//
+// The history holds two kinds of entry. A status is a position the request
+// reached, and those are the trail's spine. An event is something that happened
+// while it sat at one — a build starting, a passed path waiting on a dependency
+// — and never changes the position, so each is shown against the status it
+// occurred under rather than as a step of its own:
+//
+//	batched → speculating [building ×2, built] → speculated
+//
+// Repeats are counted rather than listed. A batch runs one build per
+// speculation path, so a request that speculated widely records `building` many
+// times, and a trail that spelled each one out would say less than the count
+// does while pushing the rest of the row off the line.
 func digest(events []*pb.HistoryEvent) (trail []string, status, note string) {
 	if len(events) == 0 {
 		return nil, "", ""
 	}
+
+	// Events seen since the last status, in first-seen order with their counts,
+	// so they can be attached once the step they belong to is complete.
+	var pending []string
+	var last string
+	counts := make(map[string]int)
+
+	flush := func() {
+		if len(trail) == 0 || len(pending) == 0 {
+			pending, counts = nil, make(map[string]int)
+			return
+		}
+		trail[len(trail)-1] += " [" + strings.Join(annotate(pending, counts), ", ") + "]"
+		pending, counts = nil, make(map[string]int)
+	}
+
 	for _, e := range events {
-		if e == nil || e.Status == "" {
+		if e == nil {
 			continue
 		}
-		if len(trail) > 0 && trail[len(trail)-1] == e.Status {
+		if e.Status == "" {
+			if e.Event == "" {
+				continue
+			}
+			if counts[e.Event] == 0 {
+				pending = append(pending, e.Event)
+			}
+			counts[e.Event]++
 			continue
 		}
+		// A status repeated back-to-back is one step, but anything recorded
+		// against it in between still belongs to that step.
+		if e.Status == last {
+			continue
+		}
+		flush()
 		trail = append(trail, e.Status)
+		last = e.Status
 	}
+	flush()
+
 	if last := events[len(events)-1]; last != nil {
 		status, note = last.Status, last.LastError
 	}
-	if status == "" && len(trail) > 0 {
-		status = trail[len(trail)-1]
+	// The last entry may be an event, which leaves the request where it was.
+	if status == "" {
+		status = currentStatus(events)
 	}
 	return trail, status, note
 }
 
+// annotate renders each event with its count, dropping the count when it
+// happened once.
+func annotate(order []string, counts map[string]int) []string {
+	out := make([]string, 0, len(order))
+	for _, event := range order {
+		if counts[event] > 1 {
+			out = append(out, fmt.Sprintf("%s ×%d", event, counts[event]))
+			continue
+		}
+		out = append(out, event)
+	}
+	return out
+}
+
+// currentStatus is the last position the request reached, ignoring anything
+// recorded while it sat there.
+func currentStatus(events []*pb.HistoryEvent) string {
+	for i := len(events) - 1; i >= 0; i-- {
+		if e := events[i]; e != nil && e.Status != "" {
+			return e.Status
+		}
+	}
+	return ""
+}
+
 // outcome is the one-line verdict shown under the finished table.
 func outcome(rows []*Row) string {
 	landed := 0
@@ -203,17 +285,29 @@ func summarize(rows []*Row) error {
 // the number of lines it *emitted* — so one wrapped line desyncs every redraw
 // after it. Everything wide is therefore wrapped deliberately, into lines the
 // renderer counts itself.
+//
+// A frame may not be taller than the window either, for the same reason in the
+// other axis. Drawing more lines than the window holds scrolls it, and the
+// cursor cannot then move back above the first row — every subsequent redraw
+// starts from the wrong place and repaints the whole screen instead of the
+// table. A watch of more requests than fit therefore shows as many as do and
+// says how many it is not showing.
 type renderer struct {
 	inPlace bool
 
-	// width is the terminal's width, re-read before every draw. A watch runs for
-	// minutes and a window can be resized inside them, so this is not a property
-	// the process can sample once — see resize.
-	width int
+	// oneShot marks a renderer that draws once and returns, so its frame is
+	// free to be taller than the window: no later frame has to line up with it.
+	oneShot bool
+
+	// width and height are the terminal's, re-read before every draw. A watch
+	// runs for minutes and a window can be resized inside them, so neither is a
+	// property the process can sample once — see resize.
+	width  int
+	height int
 
-	// size reports the terminal's width and whether it could be read. Held as a
-	// field so a test can drive a resize without a terminal.
-	size func() (int, bool)
+	// size reports the terminal's width and height and whether they could be
+	// read. Held as a field so a test can drive a resize without a terminal.
+	size func() (int, int, bool)
 
 	wRequest int
 	wChanges int
@@ -231,7 +325,7 @@ type renderer struct {
 }
 
 func newRenderer() *renderer {
-	width, sized := terminalSize()
+	width, height, sized := terminalSize()
 	return &renderer{
 		// Redrawing in place requires knowing the width to wrap to. A terminal
 		// that will not report its size is therefore treated as a log: emitting
@@ -240,6 +334,7 @@ func newRenderer() *renderer {
 		// the lines that appeared, so one of those desyncs every frame after it.
 		inPlace:  sized,
 		width:    width,
+		height:   height,
 		size:     terminalSize,
 		wRequest: len("REQUEST"),
 		wChanges: len("CHANGES"),
@@ -256,12 +351,12 @@ func newRenderer() *renderer {
 // Anything that is not a sized terminal — a pipe, a file, a CI log — falls back
 // to a fixed width, since there is no width to discover and a log wants a
 // stable one anyway.
-func terminalSize() (int, bool) {
-	w, _, err := term.GetSize(int(os.Stdout.Fd()))
+func terminalSize() (int, int, bool) {
+	w, h, err := term.GetSize(int(os.Stdout.Fd()))
 	if err != nil || w <= 0 {
-		return defaultLineWidth, false
+		return defaultLineWidth, 0, false
 	}
-	return w, true
+	return w, h, true
 }
 
 // lineWidth is the width to render to. It tolerates a renderer built without
@@ -289,14 +384,22 @@ func (r *renderer) resize() {
 	if !r.inPlace || r.size == nil {
 		return
 	}
-	if w, sized := r.size(); sized {
-		r.width = w
+	if w, h, sized := r.size(); sized {
+		r.width, r.height = w, h
 	}
 }
 
 func (r *renderer) draw(rows []*Row, status string) {
 	r.resize()
-	body := r.body(rows)
+	// Widths come from every row, not just the drawn ones, so a column does not
+	// resize as rows come and go from view.
+	r.fit(rows)
+
+	visible, hidden := r.visibleRows(rows)
+	body := r.body(visible)
+	if hidden > 0 {
+		body = append(body, fmt.Sprintf("  … %d settled request(s) not shown; the window is too short", hidden))
+	}
 
 	if !r.inPlace {
 		sig := signature(rows)
@@ -325,6 +428,74 @@ func (r *renderer) draw(rows []*Row, status string) {
 	r.drawn = true
 }
 
+// frameOverhead is what a frame spends on lines other than the table: the blank
+// line and the status line below it, plus the row the cursor rests on, which
+// has to stay inside the window or the next redraw starts a line too low.
+const frameOverhead = 3
+
+// visibleRows is the rows that fit in the window, and how many were left out.
+//
+// Settled requests are dropped first. They have stopped changing, so leaving
+// them out costs a reader nothing `land-list` will not tell them, whereas
+// dropping the ones still moving would hide the only part of the table that is
+// doing anything. Within each group the original order is kept, so rows do not
+// jump around between redraws.
+func (r *renderer) visibleRows(rows []*Row) ([]*Row, int) {
+	// headerLines is the column header and its rule.
+	const headerLines = 2
+
+	// Redirected output is a log, not a window: it scrolls, nothing is
+	// overwritten, and a reader wants every row. So does a one-shot listing,
+	// which is drawn once and scrolled back through rather than redrawn.
+	if !r.inPlace || r.oneShot || r.height <= 0 {
+		return rows, 0
+	}
+	budget := r.height - frameOverhead - headerLines
+	if budget < 1 {
+		return nil, len(rows)
+	}
+
+	heights := make([]int, len(rows))
+	total := 0
+	for i, rw := range rows {
+		heights[i] = len(r.rowLines(rw)) + len(r.noteLines(rw))
+		total += heights[i]
+	}
+	if total <= budget {
+		return rows, 0
+	}
+	// One line goes to the note saying what is not shown.
+	budget--
+
+	// Keep the unsettled first, then settled, then restore the original order,
+	// so what survives is the moving part of the table without being reordered.
+	keep := make([]bool, len(rows))
+	used := 0
+	for _, settled := range []bool{false, true} {
+		for i, rw := range rows {
+			if rw.Done != settled || keep[i] {
+				continue
+			}
+			if used+heights[i] > budget {
+				continue
+			}
+			keep[i] = true
+			used += heights[i]
+		}
+	}
+
+	visible := make([]*Row, 0, len(rows))
+	hidden := 0
+	for i, rw := range rows {
+		if keep[i] {
+			visible = append(visible, rw)
+			continue
+		}
+		hidden++
+	}
+	return visible, hidden
+}
+
 // body renders the header and one line per row.
 func (r *renderer) body(rows []*Row) []string {
 	r.fit(rows)