sandbox: ImportProfile provider race persists after #6421 fix under parallel eval

[ggallen]

What happens

After #6421 merged (commit 934df37, fixing #6420), the unsupported provider type or profile error still reproduces when multiple fullsend run processes share a single openshell gateway.

First CI run after the fix (run 32442940867, commit 50055c91c which includes the fix):

Case	Result	Error
001-bug-url-encoding	FAIL	validation failed after 2 iteration(s)
002-needs-info-vague-crash	FAIL	unsupported provider type or profile: fullsend-vertex-ai
003-feature-request	FAIL	validation failed after 2 iteration(s)
004-duplicate-issue	FAIL	validation failed after 2 iteration(s)

Case 002 still hits the provider race. Cases 001/003/004 get past provider creation (improvement — all 4 failed before #6421) but fail on an unrelated output validation issue.

Before #6421: 4/4 cases failed with the provider race.
After #6421: 1/4 cases still fails with the provider race.

The error from case 002's stderr.log:

Error: ensuring provider "vertex-ai": provider create "vertex-ai" failed: exit status 1
  (output: Error: × unsupported provider type or profile: fullsend-vertex-ai)

What should happen

All 4 parallel fullsend run invocations should create providers without racing on profile state. Provider creation should never fail with unsupported provider type or profile when the profile was successfully imported moments earlier.

How to reproduce

1. Merge or rebase onto a branch that includes fix(#6420): make ImportProfile concurrency-safe with hash-based caching #6421 (commit 934df37).
2. Push to a PR that runs functional-tests (triage suite, parallelism=4).
3. Observe that 1 of 4 cases fails at provider creation with the same error as sandbox: ImportProfile delete-then-reimport is not concurrency-safe #6420.

Also reproducing on fullsend-ai/agents PR #908.

Context

This blocks CI on all open PRs across both fullsend-ai/fullsend and fullsend-ai/agents. The functional-tests check is required.

The #6421 fix added hash-based caching to ImportProfile, mirroring ImportProfiles. This reduced the race window but did not eliminate it — concurrent processes can still interleave profile delete and provider create operations.

Relates to #6420 (closed by #6421).

[fullsend-ai-triage]

🤖 Finished Triage · ✅ Success · Started 3:32 AM UTC · Completed 3:35 AM UTC

Commit: 934df37 · View workflow run →

[fullsend-ai-triage]

Root cause

The hash-based cache from #6421 closes the race for cache-hit processes but does not protect the first process (cache miss), which still performs a non-atomic delete → reimport sequence. When all 4 parallel processes start simultaneously and all see no cache file, multiple processes enter the delete+reimport path. A concurrent EnsureProvider call that lands in the window between delete and reimport finds no profile and fails.

This explains the improvement (4/4 → 1/4 failures) — most of the time only one process hits the uncached path while the others see the cache hit from the winner. But under timing pressure, the race still fires.

Recommended fix approaches

1. File-lock the critical section — use flock around the delete+reimport in ImportProfile so only one process mutates a profile at a time. Losers block on the lock, then see a cache hit and skip.
2. Retry in EnsureProvider — treat unsupported provider type or profile as a transient error and retry with short backoff (the profile will exist moments later).
3. Both — locking eliminates the window; retry provides defense-in-depth.

Proposed test case

func TestImportProfileConcurrent(t *testing.T) {
    // Launch N goroutines that each call ImportProfile then EnsureProvider
    // for the same profile id. Assert zero "unsupported provider" errors.
    // Use a test binary or mock that simulates nonzero delete latency to
    // widen the race window.
}

Related

• sandbox: ImportProfile delete-then-reimport is not concurrency-safe #6420 (original race, closed by fix(#6420): make ImportProfile concurrency-safe with hash-based caching #6421)
• Also reproducing on fullsend-ai/agents PR Add .DS_Store to .gitignore to prevent accidental commits #908

---

Labels: This is a high-priority concurrency bug in the sandbox component that blocks CI across multiple repos. Carries over labels from the parent issue #6420.

---

Next steps:

• /fs-code — agent creates a PR to implement this issue
• /fs-code <your instruction> — agent implements with your specific guidance

[fullsend-ai-coder]

🤖 Finished Code · ✅ Success · Started 3:37 AM UTC · Completed 3:49 AM UTC

Commit: 934df37 · View workflow run →