Skip to content

feat(libs): let relinked Windows games opt into mimalloc - #6067

Merged
oneandonlydean merged 1 commit into
boykopovar:mainfrom
meh7an:perf/libs-mimalloc-windows
Oct 19, 2026
Merged

oneandonlydean merged 1 commit into
boykopovar:mainfrom
meh7an:perf/libs-mimalloc-windows

Conversation

@meh7an

@meh7an meh7an commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

What

On Windows, a relinked game and its libraries allocate through the CRT, which goes to the Windows heap. On the S2 page of Sonic Origins Plus's main menu, the queue worker spent about 32 samples per frame (APS5 sampler, 15 s) in RtlAllocateHeap, _malloc_base and _free_base.

This builds mimalloc 3.5.4 (a new submodule) as mimalloc.dll, with its Windows redirection (MI_OVERRIDE, MI_WIN_REDIRECT). mimalloc.dll and mimalloc-redirect.dll are copied beside the libraries. With APS5_MIMALLOC set (to any value), the relinker's Windows entry stub loads mimalloc.dll before every other library, so the redirection replaces the CRT allocator for the whole process. Without the variable the stub skips the load and the CRT heap stays, so nothing changes by default. Without mimalloc.dll the stub's load fails quietly, and the CRT heap stays.

  • pe_import_first, a build tool, makes mimalloc-redirect.dll the first import of the copied mimalloc.dll. The redirection needs this, and MinGW's linker does not guarantee the order.
  • RuntimeHeap.c sends mimalloc's own CRT calls (malloc, calloc, realloc, free, strdup) to the process heap through linker wraps, so they never reach the redirected allocator.
    • These calls come from the MinGW runtime that mimalloc.dll links statically: GCC's emulated thread-locals (emutls, which hold mimalloc's own thread-locals), MinGW's thread-exit destructor list, and winpthreads.
    • After the redirection they would enter mimalloc again, before the calling thread's mimalloc state exists.
  • tools/package_release.py ships both DLLs.
  • entry_stub (the relinker's entry stub tests, now linked with relinker_core) pins the new stub code, for an absolute and for a module-relative run path:
    • the stub reads APS5_MIMALLOC with GetEnvironmentVariableW before the allocator load, and an unset variable branches past exactly that load;
    • the stub uses mimalloc.dll's path before the first library's;
    • it loads it with LoadLibraryExW and the library search flags before that library;
    • a run path whose mimalloc.dll path cannot fit the startup buffer fails with the same error as a library path, one character past the longest path that fits.
  • PeImportFirst.cpp is split into a header, its functions and the tool's main. pe_import_first (a test registered on every platform) builds a small PE32+ image:
    • the named import moves to the front of the import directory, whole, and the others keep their order;
    • a missing import, a truncated import directory and a PE32 image fail and leave the file as it was.
  • docs/dev/ENVIRONMENT.md gets the APS5_MIMALLOC row.
  • docs/dev/ARCHITECTURE.md gets a "Windows allocator" section: the two DLLs, the load order in the entry stub, guest versus host allocations, and the wraps.

With mimalloc, the Windows heap's share fell from about 32 samples per frame to 0, and operator new fell from 10.1 to 6.5 (Sonic Origins Plus, on this PR's first base).

The allocator is opt-in after review. On current main, Sons of Sparta (two runs per arm) and three titles measured by @screamm show no frame-time win, and mimalloc adds 80-230 MiB of private memory. So it stays off unless APS5_MIMALLOC asks for it.

Tested

Windows 11 (10.0.26300), WinLibs GCC 15.2.0 (UCRT), Release, NVIDIA GeForce RTX 2080 Ti, driver 617.14.

  • Rebased onto main 5bf2e13 with the opt-in switch.
    • The full ctest suite passes 1479 of 1479, including entry_stub, pe_import_first and release_package.
    • python tools/env_reference.py --check and tools/check_conventions.py pass.
    • In God of War: Sons of Sparta (PPSA28997), a default run loads neither DLL. With APS5_MIMALLOC=1 both load, mimalloc: malloc is redirected. and the v3.5.4 banner appear, and there are no mimalloc errors.
  • Rebased onto main 2fab88d, whose CMakeLists.txt includes cmake/ZArchive.cmake where the mimalloc block goes; both stay. tools/package_release.py also packs fix(release): include the enabled Tracy runtime #5729's Tracy runtime, so the Windows block adds the mimalloc DLLs and the Tracy files follow for every platform, and fix(release): include the enabled Tracy runtime #5729's Tracy packaging test gets the mimalloc DLLs in its Windows fixture. On that main, with this change, the full ctest suite passes 1339 of 1342, including entry_stub, pe_import_first, release_package and windows_import_modules.
    • agc_driver_gpu_indirect_arguments_misaligned_writer_ahead is not run: main itself does not compile it with MinGW.
    • select_tests fails in its setup on Windows without the symlink privilege (WinError 1314 from symlink_to), as on main.
    • agc_driver_store_over_untracked failed once in the parallel run; alone it passes 5 of 5.
    • With the stub's allocator load moved after the libraries, entry_stub fails; with the import move made a no-op, pe_import_first fails.
  • How the redirection was checked, on my fork's dev build (its mimalloc 3.5.4, wraps, entry stub and pe_import_first are this PR's):
    • Sonic Origins Plus ran to the S2 page with MIMALLOC_VERBOSE=1 and MIMALLOC_SHOW_ERRORS=1. mimalloc reported free mode: checked and malloc is redirected.
    • With checked frees, a free of a block mimalloc does not own reports an error. The run reported no error and no warning.
    • In the sampled profiles of that run and of the run before it, RtlAllocateHeap, _malloc_base and _free_base appear in no stack, and mimalloc is in about 9% of them.
      • In the earlier run, 515 of the 516 samples with mimalloc came from libSceAgcDriver and 1 from libc, both entering through libstdc++ (operator new and delete).
    • Guest code does not use the host allocator. libc's guest heap takes its blocks from the guest arena (from malloc only when there is no arena) and frees them the same way, so no block moves between the guest heap and mimalloc. The new ARCHITECTURE.md section says so.
    • The NVIDIA driver's nvoglv64.dll and nvgpucomp64.dll link private copies of mimalloc 3.1.6, which printed their own reports under MIMALLOC_VERBOSE. They do not take part in the redirection.
  • Sonic Origins Plus, main menu, S2 page (the 3D background), 1233x694 window.
    • Captured with APS5_TRACE_SECONDS=200:5 on the same machine in one session, caches warm. Traces attached.
    • Baseline 61ac8fd, this change 02d5af5:
Metric Baseline This PR Change
frame_ms.p50 205.669 178.502 -13.2%
frame_ms.p90 213.496 189.411 -11.3%
mean_frame_ms 206.627 180.391 -12.7%
gpu_busy_ms_per_present 71.647 70.566 -1.5%
Driver.Packet (ms per frame) 203.856 177.710 -12.8%

The FPS shown in the title went from 4.99 to 5.70 (median of six readings).

Note: measuring base. These numbers come from main 0a56aaa, the last main before #5596. On Windows, later mains compare direct imports on every use, which keeps this menu at 3-4 FPS and hides a change of this size. The base also carries these commits from my fork, in both captures:

All of them are on measure/fresh-dev-2026-10-14: the baseline is 61ac8fd and this change is 02d5af5.

The comparison tool marks the window's first present (2229 vs 2486) as a configuration difference: the window opens 200 s into each run. Both summaries set trace_dropped, because the worker filled its event buffer at about 2,700 draws per frame. The phase totals above stay complete.

The baseline runs on the CRT heap. Its mimalloc.dll was removed from the game's libs, so the entry stub's load fails and nothing is redirected.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Unblocks: none
  • GPUs tested: NVIDIA GeForce RTX 2080 Ti, driver 617.14, Windows 11
  • AI-assisted: yes

Traces and summary lines (baseline 61ac8fd, this change 02d5af5):
mimalloc-traces.zip

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 7594 core/relinker/CMakeLists.txt L677-684
core/relinker/elfpatcher/tests/EntryStubTests.cpp L1, L38, L43

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 674 .gitmodules
CMakeLists.txt
tools/package_release.py
PR 1835 .gitmodules
CMakeLists.txt
PR 2780 core/relinker/CMakeLists.txt
PR 3998 CMakeLists.txt
PR 5021 .gitmodules
CMakeLists.txt
core/libs/CMakeLists.txt
core/relinker/CMakeLists.txt
PR 5431 docs/dev/ENVIRONMENT.md
PR 5797 .gitmodules
tools/package_release.py
tools/tests/test_package_release.py
PR 5925 docs/dev/ARCHITECTURE.md
PR 6044 CMakeLists.txt
core/libs/CMakeLists.txt
core/relinker/CMakeLists.txt
docs/dev/ARCHITECTURE.md
PR 6454 docs/dev/ARCHITECTURE.md
docs/dev/ENVIRONMENT.md
PR 6636 docs/dev/ENVIRONMENT.md
PR 6746 CMakeLists.txt
PR 6763 docs/dev/ARCHITECTURE.md
PR 6802 core/relinker/CMakeLists.txt
PR 6879 CMakeLists.txt
core/relinker/CMakeLists.txt
PR 6931 docs/dev/ARCHITECTURE.md
PR 6946 CMakeLists.txt
docs/dev/ARCHITECTURE.md
docs/dev/ENVIRONMENT.md
PR 6947 CMakeLists.txt
docs/dev/ARCHITECTURE.md
docs/dev/ENVIRONMENT.md
PR 6977 core/relinker/CMakeLists.txt
PR 7039 CMakeLists.txt
PR 7173 core/relinker/CMakeLists.txt
PR 7201 CMakeLists.txt
core/libs/CMakeLists.txt
PR 7332 CMakeLists.txt
core/libs/CMakeLists.txt
core/relinker/CMakeLists.txt
docs/dev/ENVIRONMENT.md
PR 7450 core/relinker/CMakeLists.txt
docs/dev/ARCHITECTURE.md
PR 7531 core/relinker/CMakeLists.txt
PR 7604 docs/dev/ENVIRONMENT.md
PR 7642 CMakeLists.txt
docs/dev/ARCHITECTURE.md
docs/dev/ENVIRONMENT.md
PR 7644 docs/dev/ARCHITECTURE.md

@meh7an
meh7an force-pushed the perf/libs-mimalloc-windows branch from f23a370 to 9a0e5fc Compare October 14, 2026 21:59
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 16, 2026
@adriadam10 adriadam10 self-assigned this Oct 16, 2026

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, the measurement and the fallback when mimalloc.dll is missing are clear. This replaces the allocator of every relinked game on Windows, so it needs more than the packaging test:

  • WindowsEntryStubBuilder.cpp:122-131 and :493-514 emit new machine code (a LoadLibraryExW per allocator path, in both the absolute-path and the relative-path branch with the PathCapacity check) with no test. core/relinker/elfpatcher/tests/EntryStubTests.cpp is the place: pin that the stub references mimalloc.dll before the first library in both branches, and that an over-long path throws like the library paths do.
  • core/libs/allocator/PeImportFirst.cpp has no test. It rewrites a PE import directory in place; add a registered test (runs on Linux CI) with a small synthetic PE showing the named import moves to first and the others keep their order, and that a missing import, a truncated directory and a non-PE32+ file fail without touching the file.
  • docs/dev/ARCHITECTURE.md should describe the new process-wide allocator, the load order in the entry stub, and the RuntimeHeap.c wraps (which calls of mimalloc itself go to the process heap and why), under the section it changes.
  • The numbers come from a base with extra commits of your fork, one title, one machine. That is fine as motivation, but please also say how the redirection was checked (e.g. that malloc in a guest-bundled module and in a host library both land in mimalloc, and that free across the two works), since a pointer mixed between mimalloc and the CRT heap would corrupt silently.

Build and the Linux test suite are fine (the failures I saw were load-related and pass alone). The Windows side still needs the full Build once these are in.

@meh7an

meh7an commented Oct 16, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review. All four points are in 8977d03 and the description:

  1. entry_stub builds the Windows stub for an absolute and a module-relative run path. It pins that mimalloc.dll's path, and its LoadLibraryExW with the library search flags, come before the first library. A run path whose mimalloc.dll path cannot fit the startup buffer throws the library paths' error, one character past the longest path that fits.
  2. PeImportFirst.cpp is split into a header, its functions and the tool's main. pe_import_first, registered on every platform, uses a synthetic PE32+ image:
    • the named import moves first, whole, and the others keep their order;
    • a missing import, a truncated import directory and a PE32 image fail without touching the file.
  3. docs/dev/ARCHITECTURE.md has a "Windows allocator" section. The wraps cover the MinGW runtime that mimalloc.dll links statically (emutls, MinGW's thread-exit destructor list, winpthreads), which would otherwise call back into mimalloc before the calling thread's mimalloc state exists.
  4. The redirection check is under Tested:
    • With MIMALLOC_VERBOSE=1 and MIMALLOC_SHOW_ERRORS=1, mimalloc reports malloc is redirected, with checked frees and no error over a run to the menu.
    • The profiles have no CRT heap frames left.
    • Guest code allocates from the guest arena through libc's guest heap, so no block crosses between the guest heap and mimalloc.

Both new tests fail when the behaviour they pin is broken (the allocator loaded after the libraries; an import move that does nothing).

@meh7an
meh7an requested a review from adriadam10 October 16, 2026 23:33

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked the four points against 8977d03:

  • entry_stub now pins mimalloc.dll's path and its LoadLibraryExW call before the first library, for both an absolute and a module-relative run path, and that a path one character past the startup buffer throws.
  • pe_import_first is split into header/functions/main and has a registered test with a synthetic PE32+ image (named import first, others keep order, missing import / truncated directory / PE32 fail without touching the file).
  • ARCHITECTURE.md has the "Windows allocator" section.
  • The redirection check (checked frees, no errors, guest heap on the guest arena) is in the description.

Built on the merged tree; pe_import_first, entry_stub and release_package pass. Merging once the full Windows Build is green.

@oneandonlydean oneandonlydean self-assigned this Oct 18, 2026
@oneandonlydean

Copy link
Copy Markdown
Collaborator

Picking this up while @adriadam10 is away. The code and tests look good to us, and full CI passes on 8977d03. Since this changes the allocator for every relinked Windows game, two things before we merge:

  1. A documented opt-out switch. If a user hits an allocator problem, we need one switch that turns the swap off, so they can rule it in or out without a rebuild. For example, an APS5_NO_MIMALLOC=1 environment variable that makes the entry stub skip loading mimalloc.dll. Please document it in ARCHITECTURE.md next to the mimalloc section. If you'd rather rely on mimalloc's own MIMALLOC_DISABLE_REDIRECT=1, documenting that and confirming it fully restores the CRT heap would also work.
  2. Windows results on current main. Please post results from at least one game that boots and runs with this PR merged into current main, with MIMALLOC_VERBOSE=1 showing "malloc is redirected" and no mimalloc errors. A frame-time comparison against main from the same scene would help. A second GPU vendor would be ideal, if you or a tester has one.

Thanks for the work on this!

@meh7an

meh7an commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @oneandonlydean, both points are done.

1. Opt-out switch. de23d16 adds APS5_NO_MIMALLOC.

  • Set to any value, it makes the entry stub skip loading mimalloc.dll. Nothing is redirected, and the process keeps the C runtime's heap; no other module imports mimalloc.
  • entry_stub_tests checks two things, for both an absolute and a $ORIGIN run path:
    • the stub reads the variable with GetEnvironmentVariableW before the allocator load;
    • a set variable branches past exactly that LoadLibraryExW.
  • It is documented under "Windows allocator" in ARCHITECTURE.md.

2. Results on current main.

  • Build: main 1b0c7cf with this PR merged and the switch on top (built from the working tree before it was committed), MinGW.
  • The full ctest suite passes, 1471/1471.
  • Machine: NVIDIA RTX 2080 Ti, Windows 11, Intel CPU (--to-intel).
  • Game: God of War: Sons of Sparta (PPSA28997, 01.008.001, Unity). It boots and runs.

Allocator checks:

  • With MIMALLOC_VERBOSE=1 MIMALLOC_SHOW_ERRORS=1, the log shows mimalloc: malloc is redirected. with no mimalloc errors or warnings, and mimalloc.dll and mimalloc-redirect.dll are in the process's module list. The NVIDIA driver carries its own private mimalloc v3.1.6 copies, which print their version lines too.
  • The same run with APS5_NO_MIMALLOC=1 has no redirect line, and neither DLL is loaded.

Frame times:

  • Same build, same point after boot with no input, APS5_TRACE_SECONDS=90:20, warm caches, log not redirected.
  • The baseline is APS5_NO_MIMALLOC=1. The PR changes no library source, so that is main's allocator path; one GetEnvironmentVariableW call at startup is the only other difference.
  • From tools/performance_compare.py:
Metric APS5_NO_MIMALLOC=1 mimalloc Change
presents in 20 s 1088 1199 +10.2%
frame_ms p50 18.274 16.683 -8.7%
frame_ms p90 19.488 16.927 -13.1%
frame_ms p99 20.988 17.503 -16.6%
frame_ms max 30.356 17.863 -41.2%
mean frame ms 18.390 16.683 -9.3%
gpu_busy_ms_per_present 8.755 9.110 +4.1%
queue worker 0x0 busy 98.1% 70.3%
  • The mimalloc run sits at the 60 Hz present cap, so the gain is bounded there.
  • An earlier round, the first after warm-up, had shader hitches in both arms, so it is left out of the table: p50 21.87 against 22.21 ms, p99 325.2 against 30.5 ms.
  • Both summaries flag dropped trace events at the default buffer size.
  • I can attach the traces, or rerun against a plain main build, if you want either.

No second GPU vendor here, sorry.

@oneandonlydean

Copy link
Copy Markdown
Collaborator

Thanks, the opt-out looks right. Could you post two more runs per arm (main vs this PR, same scene) so the p99 change isn't resting on one sample, ideally one of them on a second GPU vendor? And could you say what the 325 ms warm-up round is? Please also add an APS5_NO_MIMALLOC row to docs/dev/ENVIRONMENT.md.

@oneandonlydean

Copy link
Copy Markdown
Collaborator

Heads-up: #6078 has merged, so the environment reference check now runs on main. APS5_NO_MIMALLOC (read in WindowsEntryStubBuilder.cpp) needs a row in docs/dev/ENVIRONMENT.md, or the conventions check fails once this PR is merged on top of current main. Could you rebase onto main and add the row (presence flag: skips loading mimalloc.dll so the process keeps the C runtime's heap), then run python tools/env_reference.py --check?

@milescoviello

milescoviello commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

A Linux data point, in case it helps decide whether this should also cover Linux later. I built mimalloc from this PR's submodule commit (f8401befa6, v3.5.4) as libmimalloc.so with its default CMake options and loaded it with LD_PRELOAD. jemalloc 5.3.0 (Ubuntu 24.04's libjemalloc2) is the other comparison. Machine: i7-13700K, RTX 4070 Ti, NVIDIA 595.99.02.

Shader preparation. agc_shader_replay (Release, ANYPS5_ENABLE_SPIRV_TOOLS off, ANYPS5_NO_SHADER_CACHE=1) on main 5bf2e13, replaying 355 dumped MXGP 26 (PPSA30839) shader requests in one process. /usr/bin/time, 2 runs per allocator, interleaved:

Allocator User time Max RSS
glibc 2.39 27.54 s, 29.31 s 595 MB, 601 MB
mimalloc 3.5.4 13.00 s, 13.41 s 656 MB, 658 MB
jemalloc 5.3.0 13.02 s, 13.24 s 613 MB, 610 MB

System time is under 0.5 s in all of them. The largest single compute shader in my dumps (PPSA31162 0x1e401c0000, about 1M SPIR-V words) takes about 1.2 s on glibc and 0.48 s with jemalloc. The 462 .spv files of the 466 requests are byte-identical under all three allocators.

Gameplay. On 10-16 I ran MXGP on a test build from that day (5f469824dc, with my #6917 on top), 130 s with pad input and a warm cache, 3 runs each for glibc and mimalloc and 2 for jemalloc. The frame times in the 10 s trace window did not change measurably: p50 225 to 242 ms with glibc, 222 ms with jemalloc, 221 to 295 ms with mimalloc. The GPU was busy about 120 ms per present in every run, so that scene is GPU-bound here. In a cold-cache run of PPSA31162 on 24 host threads, the preparation threads kept up with the game either way, so the allocator showed up as idle CPU rather than more shaders. I only see a clear difference in CPU time.

If it would be useful once this is merged, I can do the Linux side: the relinked ELF would load libmimalloc.so first, with the same APS5_NO_MIMALLOC switch. That is only a suggestion; I'm not working on it unless you want it.

AI-assisted: yes (Claude Code)

@screamm

screamm commented Oct 18, 2026

Copy link
Copy Markdown
Contributor

Cross-validation on Windows 11 Pro 26220, NVIDIA GeForce RTX 4070 (driver 596.21), Ryzen 7 5700X, WinLibs MinGW-w64 GCC 15.2.0, Release. PR head de23d1600 merged on main 0d078f75d (clean). No second GPU vendor here either.

Tests. pe_import_first and entry_stub 3 of 3 each; release_package passes. With WindowsEntryStubBuilder.cpp from main and the tests kept, entry_stub fails (the stub data does not hold the expected path); restored, it passes. tools/check_conventions.py --base origin/main fails only on the known row: [env-reference] ... APS5_NO_MIMALLOC is read here but has no row in docs/dev/ENVIRONMENT.md. git diff --check is clean. I did not finish a full build or the full suite in this round.

Runs. Three titles, three arms, 3 runs each, interleaved (main, PR, PR with APS5_NO_MIMALLOC=1, repeated), same scene, 65 s, no input. Every run was killed at the limit, with no stop line. main = main's relinker; PR and PR-off = this PR's relinker. All arms use the same libs/ build (the PR changes no library source), so the arms differ only in the entry stub and in whether the two DLLs are present. Env: MIMALLOC_VERBOSE=1 MIMALLOC_SHOW_ERRORS=1 APS5_TRACE_SUMMARY_EVERY=10. Frame metrics are medians of the 10 s summary intervals after the first 10 s. Memory is process private bytes / working set at 30 and 60 s.

  • mimalloc is active in 9 of 9 PR runs: mimalloc: malloc is redirected. and the v3.5.4 banner (twice on RetroArch, once for the launcher and once for the LoadExec child), with mimalloc.dll and mimalloc-redirect.dll in the module list. In the 18 main and PR-off runs there is no redirect line and neither DLL is loaded. There are no mimalloc errors or warnings in any of the 27 runs. The NVIDIA driver's own mimalloc 3.1.6 banners appear in every arm.
title arm frame p50 ms p99 ms private 30 / 60 s MiB WS 60 s MiB
PPSA99005 PS5 OpenGL Showcase (7-9 FPS) main 133.7 (115.4-142.8) 171 (143-242) 3731 / 3830 1500
PR 130.1 (111.2-139.6) 186 (123-267) 3876 / 4045 1521
PR-off 126.3 (111.8-140.0) 181 (126-349) 3694 / 3832 1511
PPSA99169 RetroArch picker (120 Hz paced) main 8.34 8.71 (8.68-27.1) 1193 / 1195 674
PR 8.34 14.96 (8.93-16.0) 1275 / 1277 695
PR-off 8.34 8.95 (8.66-15.5) 1193 / 1195 674
PPSA28319 Middle Ages Ping Pong, Unity (60 Hz paced) main 16.68 17.3 (17.1-19.8) 2091 / 2089 672
PR 16.68 19.0 (17.1-20.5) 2297 / 2324 700
PR-off 16.68 17.8 (17.5-21.9) 2077 / 2076 658

Median (min-max) of 3 runs.

  • Frame time: I can't resolve a difference on this machine. The host was shared with other builds, with a mean CPU load of 22-93 % per run, and that load drives the spread.
    • On PPSA99005, the only title not at a present cap, the PR's p50 is 3-4 ms below main in each round. PR-off, which runs main's CRT heap, is 3-7 ms below main too. PR vs PR-off, the only pair that differs only in the allocator, is -0.4 / +3.8 / -0.6 ms.
    • So the gap to main here looks like run order and load, not mimalloc.
    • The other two titles are capped (8.34 / 16.68 ms in all runs).
    • RetroArch's p99 was above 14 ms in 2 of 3 PR runs, against 1 of 3 for main and for PR-off. With three runs on a shared host I can't call that a regression. In the quietest round all three arms were at 8.7-8.95 ms.
  • Memory: this is the clear effect. With mimalloc, private bytes are higher by +145 / +215 MiB (PPSA99005), +82 / +82 MiB (RetroArch picker) and +206 / +235 MiB (PPSA28319) at 30 / 60 s. Working set is +21 to +28 MiB at 60 s.
  • Opt-out: APS5_NO_MIMALLOC=1 matches main within -37 to +2 MiB private, and its frame times are in main's range. That supports using it as the baseline, as in your GoW table.
  • Missing DLL: with mimalloc-redirect.dll removed from libs/, the stub's load fails without any message and the title runs on the CRT heap. That matches ARCHITECTURE.md ("fails quietly"). A line under the existing dependency diagnostics would make an incomplete libs/ visible.
  • Source note: mimalloc.dll is built from the submodule. mimalloc-redirect.dll is the prebuilt binary from the submodule's bin/ (byte-identical copy); upstream does not publish its source.

Not checked: your titles (GoW Sons of Sparta, Sonic Origins Plus) aren't available here; AMD/Intel; Linux; sessions longer than 65 s; the full ctest suite.

Written with Claude Code assistance (validation by a subagent, reviewed in this session).

@meh7an

meh7an commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @oneandonlydean.

  • Rebased onto main 5bf2e13, with the APS5_NO_MIMALLOC row in docs/dev/ENVIRONMENT.md.
  • python tools/env_reference.py --check passes (437 names, all documented).
  • The conventions check and the full ctest suite (1479/1479) pass on the rebased branch.

The 325 ms round was warm-up, not the allocator:

  • Its top phases include first-time Vulkan pipeline creation inside the window: Vulkan.GraphicsPipelinePrebuild, Vulkan.GraphicsPipeline and Vulkan.GraphicsPipelineLibrary, about 9 ms per frame each.
  • Its mimalloc counterpart spent 5.7 ms per frame in Driver.Draw.shader_compile_and_link.
  • None of these show up once the caches are warm.

More runs. Sons of Sparta again, same scene and window: APS5_TRACE_SECONDS=90:20, no input, warm caches, log not redirected, RTX 2080 Ti.

Run Presents p50 p90 p99 Max Worker 0x0 busy
main + PR, APS5_NO_MIMALLOC=1 (1) 1199 16.683 16.924 17.289 17.840 69.4%
main + PR, APS5_NO_MIMALLOC=1 (2) 1199 16.683 16.925 17.275 18.002 69.1%
main + PR, mimalloc (1) 1199 16.683 16.917 17.389 17.669 70.4%
main + PR, mimalloc (2) 1151 17.306 18.011 18.808 21.157 98.3%
  • On current main this scene sits at the 60 Hz present cap with or without mimalloc, so it shows no frame-time gain.
  • The gap in my previous comment does not hold up. Runs in either arm sometimes drop into a mode with the worker about 98% busy: the earlier baseline round 2, and mimalloc run 2 here. That is what the earlier comparison caught.
  • Sons of Sparta is neutral for this PR.

Plain main 5bf2e13 could not complete a traced run on this title:

  • One run stalled near 17 FPS without writing the summary, and another aborted after 39 s.
  • A rerun with the log captured aborted after 57 s with clock_gettime: GetProcessTimes failed: Invalid access to memory location, an uncaught std::system_error from CpuTimeNanos in core/libs/prx/libkernel/Time/Time.cpp.
  • Untraced, plain main ran 120 s at 60 FPS.
  • This looks like a separate bug in main. I don't claim the PR fixes it: main + PR with APS5_NO_MIMALLOC=1, the same C runtime heap, completed both of its runs.

Still no second GPU vendor here, sorry.

@oneandonlydean

Copy link
Copy Markdown
Collaborator

Thanks for the honest re-measure, and thanks @screamm for the three-title runs. Putting them together: no Windows run shows a frame-time win (your title is neutral now that the worker-busy mode is accounted for), and screamm's runs show p99 worse on two paced titles (14.96 vs 8.71 ms, 19.0 vs 17.3 ms) and +80-230 MiB private memory, with APS5_NO_MIMALLOC=1 matching main.

So we'd rather not swap the allocator for every relinked Windows game by default. Could you invert the switch, so mimalloc loads only when it's asked for (for example APS5_MIMALLOC=1), with the ENVIRONMENT.md row updated to match? Then anyone who wants to try it can, at no cost to everyone else, and we'll rerun CI and merge.

@github-actions github-actions Bot added perf Performance change perf numbers missing perf pull request without a before and after summary line labels Oct 18, 2026
@meh7an
meh7an force-pushed the perf/libs-mimalloc-windows branch from 8e2c330 to 9824c46 Compare October 18, 2026 21:15
@meh7an meh7an changed the title perf(libs): use mimalloc as the allocator of relinked Windows games feat(libs): let relinked Windows games opt into mimalloc Oct 18, 2026
@meh7an

meh7an commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

Done, thanks both, and thanks @screamm for the three-title cross-check.

  • mimalloc is opt-in now. The entry stub loads mimalloc.dll only when APS5_MIMALLOC is set (to any value); without it the stub skips the load, and the process keeps the C runtime's heap.
    • The ENVIRONMENT.md row and ARCHITECTURE.md's "Windows allocator" section say so.
    • APS5_NO_MIMALLOC is gone.
    • entry_stub checks the new branch for both run paths.
  • The PR is squashed into one commit on main 5bf2e13, and the title and description now match.
    • The full ctest suite passes, 1479/1479.
    • python tools/env_reference.py --check and the conventions check pass.
  • Sons of Sparta:
    • A default run loads neither DLL.
    • With APS5_MIMALLOC=1 both load, the log shows the redirect line and the v3.5.4 banner, and there are no mimalloc errors.
  • An unrelated note: main's clock_gettime abort (GetProcessTimes failed: Invalid access to memory location) also hit one untraced APS5_MIMALLOC=1 run, after the redirect.
    • So it is tied neither to tracing nor to the allocator.
    • It looks like a separate main bug, worth an issue of its own.

@github-actions github-actions Bot removed perf Performance change perf numbers missing perf pull request without a before and after summary line labels Oct 18, 2026
@oneandonlydean
oneandonlydean merged commit 60fb807 into boykopovar:main Oct 19, 2026
13 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants