feat: add support for copilot models

[nhs000]

solve #2227

Wait for BerriAI/litellm#8577 to be merged

[CLAassistant]

All committers have signed the CLA.

[beetleb]

I think for the claude models, we should keep haiku as the weak models. Having sonnet-3.5 as the weak model for sonnet-3.7 is expensive and overkill.

[nhs000]

@paul-gauthier Since the changes in litellm side were merged at BerriAI/litellm#8577, could you have a look at this to see if I'm missing anythings?
I think the remaining is to bump litellm version once it is merged to the main branch and officially released.
Also I saw some warnings about missing the model information, I guess we need to fix it in litellm side as well?

[beetleb]

I think for the claude models, we should keep haiku as the weak models. Having sonnet-3.5 as the weak model for sonnet-3.7 is expensive and overkill.

@nhs000

Please change the weak models to haiku. sonnet-3.5 costs the same as sonnet-3.7 - there's no point in making 3.5 the weaker model.

[necto]

Please change the weak models to haiku. sonnet-3.5 costs the same as sonnet-3.7 - there's no point in making 3.5 the weaker model.

Sorry to intrude here, are the sonnet costs relevant when used via GH Copilot? Isn't it a flat fee regardless of the model used?

[beetleb]

Please change the weak models to haiku. sonnet-3.5 costs the same as sonnet-3.7 - there's no point in making 3.5 the weaker model.

Sorry to intrude here, are the sonnet costs relevant when used via GH Copilot? Isn't it a flat fee regardless of the model used?

Fair point. One could even argue that the weak model could be the same as the strong model in that case - why have them be different?

My concern is twofold.

The first is simple: Using such a heavy model is overkill for the weak model.

The second is that (some) people who have used proxies to reroute Aider to copilot have gotten warning emails from Microsoft. It's not clear what triggers it - perhaps too much /architect use. I think that lowering the overall burden on their servers would make it less likely to be banned from Copilot. As such, using a much "lighter" model for the weak model makes sense.

[TheBestPessimist]

why is the number of tokens so small? As far as i see both 3.7 and 3.5 accept 200k context: https://docs.anthropic.com/en/docs/about-claude/models/all-models.

do I misunderstand how max_tokens works?