fix: rework claude batch API implementation

[tasoo-oos]

PR Checklist

• Required Checks
  • Have you added type definitions?
  • Have you tested your changes?
  • Have you checked that it won't break any existing features?
• If your PR uses models¹, if true, check the following:
  • Have you checked if it works normally in all models?
    • It works on all Anthropic variants, both thinking and non-thinking mode.
    • This has also been tested to Adaptive thinking implementation feat: support claude adaptive thinking #1275
  • Have you checked if it works normally in all web, local, and node hosted versions? If it doesn't, have you blocked it in those versions?
• If your PR is highly ai generated², check the following:
  • Have you understanded what the code does?
  • Have you cleaned up any unnecessary or redundant code?
  • Is is not a huge change?
    • We currently do not accept highly ai generated PRs that are large changes.

Summary

• Fixed bug when AI message is not parsed correctly when thinking mode is on.
• Now thinking token is also parsed to the message.
• Made cancel UI work with Batch API.

Related Issues

None

Changes

Before this change:

• Batch API only returns the first entry of the content or empty string (batchData.result.message.content?.[0]?.text ?? '')
  • This implementation does not handle the case when thinking is turned on, because thinking section is now 0th entry of the message.content
    • This behavior is new, and before than it worked just fine. It didn't parse thinking tokens though. More about API format is in the docs
• There was no cancel logic integrated with batch API, so when user press request cancel button (the round rolling thing when it is green), it stops immediately but batch job does not does not cancel under the hood, leading to unexpected billing.

After this change:

• When result arrives, code iteratively look for all blocks, and parse both thinking tokens and text tokens.
• When user press cancel button, batch cancellation API call (v1/messages/batches/$MESSAGE_BATCH_ID/cancel)
  • API invoke to cancel API does not cancel the batch immediately, so polling continues.
  • When batch is cancelled, polling stops and request is cancelled.
  • When batch is successfully ended (generated response), it parses the response and return that.

Impact

• Now thinking token is included in the response when using batch API.
• Now cancel button takes time to be cancelled before actually stops, instead of being cancelled immediately.
• Now, internally, Anthropic batch API uses streaming(StreamResponseChunk) instead of returning plain JSON in order to bypass AbortSignal Check on other parts of the codebase.
  • In order to do otherwise, request.ts and index.svelte.ts should be edited to bypass abort signal check. That decision seemed risky for this small change since that could cause unexpected failure to abort API request with other model requests.
• Now there is a interceptor flag called 'anthropic_batching_cancel' passed down to fetchNative.

Additional Notes

I have written very few PR before, so I hope this comment is helpful enough.

Footnotes

1. Modifies the behavior of prompting, requesting or handling responses from ai models. ↩

2. Almost over 80% of the code is ai generated. ↩

[tasoo-oos]

1be484a

I saw this commit caused merge conflict.
Don't worry. I will resolve it later.

[tasoo-oos]

I resolved merge conflict with the new commits related to plugin v3 interceptor flag.
While doing that, new flag called 'anthropic_batching_cancel' is made.

[tasoo-oos]

Rebased past commits to work with #1304