[mefyl]

Description

When building over RPCs, display progress via the existing dune monitor polling.

Example status line output:

Connected to RPC server | Done: 94% (89/94, 5 left)

Related Issue and Motivation

When using passive watch mode, building will properly report the end result, but shows no progress. On longer builds taking minutes, this leaves dune build staring at you silently for what is, IMO, an uncomfortable time. I often end up going to the dune watch output terminal/buffer to check progress.

There is a slight misfeature were, in a pathological case, we could display the very end of the previous build right before it is canceled. We do send the build request first to mitigate that as much as possible, but it is possible that a progress flashes before we display "restarting current build" and the new progress. I did not find a fix simple enough to justify the change, given how non critical this issue seems to be.

Checklist

• Tests added, if applicable.
• Change log entry added for any user-facing changes.
• Documentation added for any user-facing changes.

[Alizter]

The main disadvantage of subscribing to the global count is that it will show unrelated progress on a busy server. For an exaggerated example, consider the main build server currently busy with running some long jobs like some tests. If a client requests a target to be built, it will join the current build, merging their progress. The client may however get a reply that their build is finished before the entire progress finishes.

See the rpc-build-concurrent-finish.t test for such an example.

I've experimented in the past with having per-requests progress reporting that would give the client a more accurate progress. You can see a recently rebased branch here https://github.com/Alizter/dune/tree/prototype/rpc-request-progress

This was just an experiment on my part and I didn't have any near-term plans to finish off this idea. But it may give you some ideas on how you can solve some of the problems that you have encountered.

[rgrinberg]

Disregarding the caveats already mentioned, I think we should try to keep the display code as centralized as possible to avoid UI drift in the various build reporting code.

[rgrinberg]

Going to approve this on agreement on the following:

1. We'd like to consolidate the display code so that we don't have to chase various ends of the code base to update things. This is a thing we'd like to be easy to iterate and improve

2. We accept the limitation that progress is shared.

3. Stick to dense, informative data in the status line. Telling that goals are shared isn't all that helpful or actionable. The number of active goals is sufficient.

[mefyl]

The main disadvantage of subscribing to the global count is that it will show unrelated progress on a busy server.

Good point. Solving this the 100% correct way by attributing each rule to the target(s) it contributes to seems indeed like a big endeavor, if doable at all without impacting performances too much.

May I suggest this compromise, until we assess whether your work can give us the perfect number: if multiple goals are being built, we still display the total progress but indicate that it's shared:

Connected to RPC server | Done: 94% (89/94, 5 left) | shared with 1 other build

I think it achieves most of the feature with almost none of the complexity:

• In a basic setup without concurrent builds it just works (ngl it's 95% of my own usage so I'm mainly dogfooding here).
• With concurrent builds, it might not even be too far off if both goals depend on the main library / executable. Iow the error may be marginal.
• Even if it is way off, I can at least see something moving and check it's making progress instead of being faced with complete silence that leaves me wondering whether it's stuck.

All in all I feel like reporting the progress with the limitation that we may aggregate it with another build is better than not reporting anything.

[mefyl]

Following the discussions, this PR evolved into:

• Add the number of goals in the status line (if > 1)
• When building over RPCs, display the same status line as the server. This does not necessarily correspond to the goal we're targeting over this RPC, but this limitation is accepted for now.

The work by @Alizter could make this output better in the future by showing only the progress for the current goal.

[rgrinberg]

To clarify one more thing: the fact that RPC and regular build display should match is not a hard rule. It's just that differences should be clearly motivated as to why is something is only appropriate for one and not the other. Even with this mind, I would still prefer if the entire UX component was consolidated into one place and we handled the differences via conditionals in this component.