fix: start polling the translation task sooner

[bakiburakogun]

Summary

The first poll after scheduling a translation task happens two seconds in. With a synchronous translation provider that answers in well under a second, nearly all of that is spent waiting for the poll rather than for the translation.

For live subtitles this is more noticeable than it looks, because the translation is only scheduled once the transcript is final. The two seconds land on top of the pause at the end of the sentence, so a viewer reading the translated line is always a beat behind the conversation.

This starts the backoff at 200ms and doubles from there, capped at the same 5 seconds as before:

	polls at (seconds after scheduling)
before	2, 6, 11, 16, 21
after	0.2, 0.6, 1.4, 3.0, 6.2

A provider that takes several seconds still settles into the same 5 second rate, so nothing changes for those. Only the fast case gets picked up earlier.

The backoff was also not doing what the comment says

wait_time is reassigned on every iteration, so wait_time ** i compounds:

i=1: min(2**1, 5) = 2   ->  wait_time = 2
i=2: min(2**2, 5) = 4   ->  wait_time = 4
i=3: min(4**3, 5) = 5

That is 2, 4, 5, 5, not the 1,2,4,5,5,... in the comment. Multiplying instead of raising to the power of the iteration count gives a plain doubling, which is what the comment was describing.

Is polling this often safe

Only the user facing /taskprocessing/schedule endpoint carries a rate limit (UserRateLimit(limit: 20, period: 120)). The tasks_consumer endpoints used here are gated by ExAppRequired and are not rate limited, so the extra polls at the start do not risk tripping the 429 path further down.

The number of extra requests is small in any case: a fast provider is done within the first two or three polls, and a slow one reaches the 5 second rate after five.

Testing

Running on a four node deployment with a synchronous translation provider that answers in about 90ms. Subtitles that previously appeared about two seconds after the speaker finished a sentence now appear in well under a second.

[kyteinsky]

hmm, not sure about this one.
the polling requests can be a heavy in some instances and we had seen issues with it in the past.
on the other hand, it is holding back the fast providers for no reason.

maybe this one and #106 can be combined into one option "FAST_TRANSLATION_PROVIDER" as a boolean? In case of fast providers, we switch to sending the partials and shorter poll times.
we can continue the discussion in the issue as well.

cc @marcelklehr

[bakiburakogun]

We are one of the setups you describe as being held back for no reason, so here are the numbers rather than an opinion. I put the detail in #106; the short version is that our provider answers a subtitle-sized line in about half a second, and even the slowest pair on a full paragraph is a little over two seconds.

Against the current schedule that means the translation is ready and then waits. The task goes to a background job on a one second tick and the first poll is two seconds after scheduling, so a 500 ms translation sits done for roughly 1.5 s before anything looks at it. Starting at 0.2 s and doubling, as in this PR, reads it on the third or fourth attempt instead.

On folding this together with #106 behind one FAST_TRANSLATION_PROVIDER boolean: that works for us, and I would take it over a bare interval change, because the two are the same question — whether the provider can keep up with a sentence as it is spoken or only with the full stop.

One thing worth deciding while you shape it. A single flag describes the provider, but speed is really a property of the provider and the pair: the same service here is 36 ms on one pair and 240 ms on another for the same sentence, and a 7B model we benchmarked was 1.3 to 1.8 seconds on the same texts. So the flag reads best as "assume it keeps up, and back off when a task is not ready" rather than as a promise about every pair — which is more or less what the backoff in this PR already does, with the flag deciding only where the ramp starts.

Happy to rework this into that shape once you and @marcelklehr have settled it, and happy to run whatever measurement would help the decision — the instance this came from has the provider and a four node HPB cluster behind it.

[github-actions]

Hello there,
Thank you so much for taking the time and effort to create a pull request to our Nextcloud project.

We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process.

Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6

Thank you for contributing to Nextcloud and we hope to hear from you soon!

(If you believe you should not receive this message, you can add yourself to the blocklist.)