Chunk level timeout on streamText

[NeroBlackstone]

Feature Description

Although streamText provides abortSignal option, this is for the entire http request. Can we implement a chuck level timeout (which is obviously more practical)?

I guess this can be achieved with onChunk, but it would be better if the ai sdk had this feature built in

Use Cases

No response

Additional context

No response

[lgrammel]

Can you describe how the chunk timeout should behave? When should it reset, when should it time out?

[NeroBlackstone]

Can you describe how the chunk timeout should behave? When should it reset, when should it time out?

Thank you for reply.

As we know, Server-Sent Events of OpenAI compatible api will return data streaming:

data: {"id":"20250329162729bae32349198643da","choices":[{"index":0,"delta":{"content":"]}"}}],"created":1743236852,"model":"glm-zero-preview","object":"chat.completion.chunk"}

data: {"id":"20250329162729bae32349198643da","choices":[{"index":0,"delta":{"content":"","role":"assistant"}}],"created":1743236852,"model":"glm-zero-preview","object":"chat.completion.chunk"}

data: {"id":"20250329162729bae32349198643da","choices":[{"finish_reason":"stop","index":0,"delta":{}}],"created":1743236852,"model":"glm-zero-preview","object":"chat.completion.chunk"}

..................many data chunks......................

data: [DONE]

I hope to terminate the request when the waiting time between chunks exceeds a certain threshold.

When should it reset

Maybe we can set a timer for the chunk, and when the chunk is received, reset the timer; when timeout, terminates the request.

[NeroBlackstone]

My solution:

  const chunkTimeoutAbortController = new AbortController();
  let inactivityTimerId: NodeJS.Timeout | null = null;

  const resetInactivityTimer = () => {
    if (inactivityTimerId) {
      clearTimeout(inactivityTimerId);
    }
    inactivityTimerId = setTimeout(() => {
      chunkTimeoutAbortController.abort();
    }, CHUNK_TIMEOUT_IN_MS);
  };

const result = streamText({
  onChunk: () => {
      resetInactivityTimer();
    },
    onError: ({ error }) => {
      if (inactivityTimerId) {
        clearTimeout(inactivityTimerId);
      }
      console.error(LogFormat.error('StreamText error:' + error));
    },
abortSignal: chunkTimeoutAbortController.signal, 
onFinish: async ({ response }) => {
      if (inactivityTimerId) {
        clearTimeout(inactivityTimerId);
      }
},
})

resetInactivityTimer();

[lgrammel]

I'll consider this when we rework timeouts and fallback providers