AI Fallback

Automatically switch between AI model providers when one experiences downtime or errors.

Warning

Version compatibility:

• Use ai-fallback version 0 for AI SDK v4.
  
• Use ai-fallback version 1 for AI SDK v5 (currently in beta).

Why?

AI providers can encounter:

• Rate limiting
• Service outages
• API errors
• Capacity issues
• Timeouts

This package ensures reliability by specifying multiple AI models as fallbacks. It automatically switches to the next available model if the primary fails, maintaining application uptime.

---

Installation

npm install ai-fallback@0 # use version 0.x.x for AI sdk v4

---

Models Fallback Reset

Reset to the primary model after a delay (e.g., 1 minute):

const model = createFallback({
    models: [
        anthropic('claude-3-haiku-20240307'), // Use Claude as preferred model
        openai('gpt-3.5-turbo'),
    ],
    onError: (error, modelId) => {
        console.error(`Error with model ${modelId}:`, error)
    },
    modelResetInterval: 60000, // Reset to first model after 1 minute of the first error
})

---

Usage

Create a Fallback Model

import { createFallback } from 'ai-fallback'
import { openai } from '@ai-sdk/openai'
import { anthropic } from '@ai-sdk/anthropic'

const model = createFallback({
    models: [anthropic('claude-3-haiku-20240307'), openai('gpt-3.5-turbo')],
})

---

Text Generation

Generate text with automatic fallback:

const result = await generateText({
    model,
    system: 'You are a helpful assistant.',
    messages: [{ role: 'user', content: 'Count from 1 to 5.' }],
})

---

Streaming Responses

Stream text responses:

const stream = await streamText({
    model,
    system: 'You are a helpful assistant.',
    messages: [{ role: 'user', content: 'Count from 1 to 5.' }],
})

for await (const chunk of stream.textStream) {
    console.log(chunk)
}

---

Structured Output

Stream typed objects using Zod schemas:

import { z } from 'zod'

const stream = await streamObject({
    model,
    system: 'You are a helpful assistant.',
    messages: [
        {
            role: 'user',
            content: 'Give me a person object with name and age properties.',
        },
    ],
    schema: z.object({
        name: z.string(),
        age: z.number(),
    }),
})

for await (const chunk of stream.partialObjectStream) {
    console.log(chunk)
}