jit.profile / jit.allocprof: hardenings and jit.util.gcstat - #669
Closed
damiansirbu wants to merge 7 commits into
Closed
jit.profile / jit.allocprof: hardenings and jit.util.gcstat#669damiansirbu wants to merge 7 commits into
damiansirbu wants to merge 7 commits into
Conversation
jit_profile_callback ran exit(EXIT_FAILURE) after a failed lua_pcall, so one error in the sampling callback killed the whole process. The engine panic handler (CScriptEngine::lua_panic) already logs the stack and returns, so the error stays visible. Drop the exit, clear the callback thread stack, and keep profiling. The failed sample is dropped.
luaJIT_profile_stop cleared ps->g before profile_timer_stop, so the Windows timer thread could enter profile_trigger, read the now-null g, and dereference it. Join the timer thread first, then clear the state. No trigger runs after the join.
…frames whole allocprof_record cut the stack key mid-frame at the buffer bound, so a truncated frame read as an unattributable stack and lost bytes to "unknown". Cut at the last ';' instead, so a partial frame drops whole. Raise the CPU dumpstack buffer 2048->8192 and the alloc stack key 512->4096 so a 64-deep capture holds the whole stack without mid-frame truncation.
The CPU sampler slept a fixed interval, so a mod running on a fixed schedule could phase-lock to it and be systematically over- or under-sampled. Draw each sleep from an exponential around the mean, the same way the allocation profiler already draws its sample distance, so the sample times are never periodic.
The leaf+stack aggregation tables were 80MB of static image data (4096+16384 slots x 4104B) after the key-size increase. Allocate them with calloc on the first jit.allocprof.start instead: a process that never profiles holds no table memory. The tables stay alive after stop because consumers dump after allocprof.stop(); reset and slot reads are NULL-guarded for calls before the first start.
The allocation profiler measures bytes but cannot show GC schedule health:
how close the collector is to its next step and how far behind it runs.
Those live in GCState (threshold, estimate, debt) and were not Lua-readable;
collectgarbage("count") exposes only total.
Add jit.util.gcstat() returning {total, threshold, estimate, debt} in bytes,
mirroring jit.util.traceinfo's table shape. Double-encoded, so an MSize past
2 GB does not wrap the int32 field. Read-only: it exposes engine state and
computes nothing, so it never changes VM behaviour. Manifest entry added.
The profiler can show GC time share but not how often the collector runs. Add a monotonic cycle counter incremented at both GCSpause end-transitions in gc_onestep (the single chokepoint every collection passes through, so it catches allocation-driven, parallel, and explicit collects alike), and return it as a fifth gcstat field. The counter is a file-scope global in lj_gc.c, not a GCState field, so it does not shift global_State layout or the save format.
Contributor
Author
|
Superseded: moved the branch to a clean fork. Reopening from there. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resubmit of #663. Its base branch
prwas deleted, so a reopen was not possible. This is the same content against the currentall-in-one-vs2022-wpo.Hardenings on top of the merged #641 profilers, from running them as the backend of an in-game profiling UI on a full GAMMA install:
jit.util.gcstat: threshold, estimate, debt, and a completed-cycle count, for reading collector health from LuaTesting
Single-thread and multi-thread, 2-minute combat captures on a vanilla-based install and on full GAMMA.
0% of samples at the depth-64 cap, the VM-state split reconciled with the per-stack weights, and gcstat coherent across a fight (threshold at or above total, estimate at or below total, debt bounded).