Problem
evals/run.sh passes CLOUD_ML_REGION into the gemini container as LIGHTSPEED_PROVIDER_REGION (introduced in #40). That variable is shared with the claude via Vertex setup, where its usual value is us-east5. The sandbox app maps it into GOOGLE_CLOUD_LOCATION (config.py#L102), so a region intended for claude also pins the gemini SDK.
The default GEMINI_MODEL in run.sh is gemini-3.1-pro-preview, which is currently only served from the global Vertex location. Result: anyone with the claude Vertex environment exported who runs both providers from the same shell gets a working claude container and a broken gemini container.
Evidence
Tested with the lightspeed-agentic-sandbox image and Vertex ADC:
- Gemini container with
CLOUD_ML_REGION unset: resolves SDK=gemini, query round trips, {"success":true,"summary":"pong"}. The SDK routes to a location where the model is available.
- Gemini container with
CLOUD_ML_REGION=us-east5: query fails with 404 NOT_FOUND: Publisher model projects/.../locations/us-east5/publishers/google/models/gemini-3.1-pro-preview was not found.
- Direct Vertex
generateContent probes: gemini-3.1-pro-preview returns 404 in us-east5 and us-central1, 200 only in global. gemini-2.5-pro and gemini-2.5-flash return 200 in us-east5 today, but model to region availability is Google's call and shifts over time, so the fix should not depend on any particular model and region pairing.
Suggested fix
Give gemini its own region variable instead of sharing CLOUD_ML_REGION with claude, for example LIGHTSPEED_PROVIDER_REGION="${GEMINI_REGION:-}" for the gemini case in evals/run.sh. Left empty, the SDK picks a location where the configured model exists (verified working). Anyone needing a specific region can set GEMINI_REGION explicitly.
Context
Follow up from the review discussion on #40: #40 (comment)
Problem
evals/run.shpassesCLOUD_ML_REGIONinto the gemini container asLIGHTSPEED_PROVIDER_REGION(introduced in #40). That variable is shared with the claude via Vertex setup, where its usual value isus-east5. The sandbox app maps it intoGOOGLE_CLOUD_LOCATION(config.py#L102), so a region intended for claude also pins the gemini SDK.The default
GEMINI_MODELinrun.shisgemini-3.1-pro-preview, which is currently only served from theglobalVertex location. Result: anyone with the claude Vertex environment exported who runs both providers from the same shell gets a working claude container and a broken gemini container.Evidence
Tested with the
lightspeed-agentic-sandboximage and Vertex ADC:CLOUD_ML_REGIONunset: resolvesSDK=gemini, query round trips,{"success":true,"summary":"pong"}. The SDK routes to a location where the model is available.CLOUD_ML_REGION=us-east5: query fails with404 NOT_FOUND: Publisher model projects/.../locations/us-east5/publishers/google/models/gemini-3.1-pro-preview was not found.generateContentprobes:gemini-3.1-pro-previewreturns 404 inus-east5andus-central1, 200 only inglobal.gemini-2.5-proandgemini-2.5-flashreturn 200 inus-east5today, but model to region availability is Google's call and shifts over time, so the fix should not depend on any particular model and region pairing.Suggested fix
Give gemini its own region variable instead of sharing
CLOUD_ML_REGIONwith claude, for exampleLIGHTSPEED_PROVIDER_REGION="${GEMINI_REGION:-}"for the gemini case inevals/run.sh. Left empty, the SDK picks a location where the configured model exists (verified working). Anyone needing a specific region can setGEMINI_REGIONexplicitly.Context
Follow up from the review discussion on #40: #40 (comment)