Add r1-as-architect with deepseek-chat-as-editor to polyglot_leaderbo…

[gabrielgrant]

…ard.yml

[CLAassistant]

All committers have signed the CLA.

[gabrielgrant]

note that the actual cost of running this was significantly less than reported ($2.77 vs $6.696), due to running partially during deepseek's off-peak discount hours:

this is a pretty absurd price/performance ratio

[paul-gauthier]

There were a lot of errors logged. Did you investigate?

[gabrielgrant]

I'm not totally sure what types of errors are being reported, but I think the API was unavailable for a period while running the test. I haven't run/deeply read the test code, so wasn't sure if it would resume cleanly, so just left it running repeatedly retrying until it came back online to avoid having to re-run everything

[paul-gauthier]

Unfortunately I can't merge/publish results with so many unexplained errors.

[gabrielgrant]

What types of errors are being reported here? Is it something other than just being disconnected from the API?

[gabrielgrant]

Looking through the logs, there are two types of errors that seem to be showing up a lot:

litellm.APIError: APIError: DeepseekException - peer closed connection without sending complete message body (incomplete
chunked read)
Retrying in 32.0 seconds...

and

litellm.APIError: APIError: DeepseekException - Unable to get json response - Expecting value: line 10 column 1 (char
9), Original Response:









Retrying in 512.0 seconds...