[release/10.0] Change PerfMap CRST to use UNSAFE_ANYMODE

[hoyosjs]

Fixes #128401

main PR #128429

Description

A .NET process on Linux can permanently deadlock when PerfMap is enabled (DOTNET_PerfMapEnabled or via Diagnostics Server IPC). The deadlock is a three-way cycle between GC suspension, CodeFragmentHeap::m_Lock (CRST_UNSAFE_ANYMODE), and PerfMap::s_csPerfMap (CRST_DEFAULT).

Acuiring s_csPerfMap on a coop thread toggles a to preemptive before acquiring the mutex, then back to cooperative after. The post-acquire DisablePreemptiveGC can block indefinitely via RareDisablePreemptiveGC when a GC suspension is in progress if another thread started GC suspension and a that hasn't been suspended tries to hold CodeFragmentHeap lock. That thread cannot reach a GC safe point, so we end up blocking suspension indefinitely.

The fix changes s_csPerfMap to CRST_UNSAFE_ANYMODE, eliminating the GC-mode toggle. All data accessed under this lock is native (FILE*, fd, SString), so holding it in cooperative mode introduces no new GC-safety issues — the I/O was already performed in cooperative mode with the previous default Crst (the toggle only affected the mutex-wait phase, not the work phase). The IO operations could be slow and cause gc pause times, but those are already done in coop mode right now so it's not regressing that case.

Customer Impact

Any .NET application running with PerfMap enabled on Linux (directly or via diagnostic tools like dotnet-trace, dotnet-monitor, or third-party APM agents) can hang permanently under concurrent virtual-call-stub creation and GC pressure. The process becomes unresponsive with no way to recover short of killing it. The only workaround is disabling both DOTNET_PerfMapEnabled=0 and DOTNET_EnableDiagnostics_IPC=0, which disables all perf profiling and diagnostics capabilities.

Regression

Yes, from 8 to 10. Likely around perfmap granularity changes when stub creation got rewritten.

Testing

• Audited all s_csPerfMap holders (Enable, Disable, LogJITCompiledMethod, LogPreCompiledMethod, LogStubs) - all access only native data (FILE*, fd, SString, PCODE) with no managed references. Audited the 4 problematic call sites as well.
• Stress-tested with a repro app with 2000 emitted types across 20 interfaces with concurrent polymorphic dispatch and aggressive GC pressure (DOTNET_GCConserveMemory=9). Usually 5 - 10 runs repro'd the issue against .NET 10, but with this 50 iterations ran to completion.

Risk

Low. The change is a single flag addition to a lock initialization call. CRST_UNSAFE_ANYMODE is well-established and widely used throughout the VM for locks that protect native-only data (e.g., CodeFragmentHeap::m_Lock, m_JumpStubCrst, m_CodeHeapLock). The behavioral change is strictly a removal of an unnecessary GC-mode toggle - the I/O under the lock was already executing in cooperative mode after the post-acquire toggle, so no new GC-latency regression is introduced.

[dotnet-policy-service]

Tagging subscribers to this area: @agocke
See info in area-owners.md if you want to be subscribed.

[Copilot]

Pull request overview

This PR updates the CoreCLR PerfMap synchronization strategy to avoid a GC-mode toggle deadlock that can permanently hang Linux .NET processes when PerfMap is enabled. It does this by making the PerfMap lock safe to acquire without forcing cooperative/preemptive GC transitions, matching other native-only locks used on the relevant execution paths.

Changes:

• Initialize PerfMap::s_csPerfMap with CRST_UNSAFE_ANYMODE to eliminate GC-mode toggling during lock acquisition.
• Relax PerfMap::LogJITCompiledMethod contract from MODE_PREEMPTIVE to MODE_ANY to reflect the updated locking/mode behavior.