Repository navigation
Conversation
Changed emit from trace -> Event.t -> unit to trace -> ~ring_id:int -> ~ts:int64 -> ~name:string -> ~kind:Event.kind -> unit, passing fields directly instead of boxing them into a record on every event. This should avoid one Event.t allocation per event. Event.t type is preserved since kind is still needed as an extensible type.
Provides a direct path for counter events that takes ~value:int instead of ~kind:(Counter value).
This is a preliminary patch for a library change to come, to ensure that the emitted traces have the same information, and that the new library works properly with the kind of traces that olly emits. The test calls Olly_format_fuchsia directly with every kind of event olly emits, and decodes the resulting trace with a small FXT reader (test/dump_fxt_trace.ml). The FXT reader normalizes the trace to be agnostic to the library's table allocation policy (each event is fully printed with referentes resolved). The resulting trace is committed as a cram test.
Key design choices: - C stubs (fxt_put_event_header, fxt_put_arg_header_i32/i64) pack multi-field headers into int64 words using C bit manipulation — zero Int64 boxing - String interning (32K table) — repeated phase names like "major", "minor" are emitted once as string records, then referenced by index. Makes traces ~50% smaller and avoids redundant string writes - Thread interning (256 table) — domain thread refs registered once, referenced by index - Single Bytes.t buffer (64KB) flushed to out_channel — no Buf_chain, no locks, no pool (olly is single-threaded) - %caml_bytes_set64u compiler intrinsic for timestamp writes — single instruction, no allocation
Defines a global [max_domains] constant, set to the previously value of 128 that was hardcoded in two places (gc-stats, trace backends). Olly now fails earlier, at events file loading instead of on the first domain index over the limit encountered.
This value is consistent with the new hard limit of OCaml since 5.3, and now possible with the new FXT lib. The footprint of olly is increased, mostly in the tables allocated at loadctime by gc-stats (domain_times, domain_gc_times, domain_minor_words, domain_promoted_words, domain_major_words) and by the FXT backend, accounting for an extra ~500 KB, and at final display time (print_percentiles), with a few MB of short-lived allocations for the text and JSON reports. The final print allocations could be dropped using a high watermark of used domain indices.
Tested in both plain and flambda ocaml 5.5.2, the ocaml assembly is simple and does not allocate. OCaml appears to have good enough unboxing for these patterns (simple expressions including calls to of_int as arg to the %caml_bytes_set64u primitive).
In perfetto, events have a pid and a tid. Spans are associated to a pair (tid, pid), and counters are associated pid, disregarding thetid. The JSON trace backend outputs (tid = pid = ring_id) on all events, resulting in a pid track section for each ring_id, each containing a track for this ring_id's spans and a track for each counter emitted from this ring_id. The FXT backend outputs (pid = 0, tid = ring_id). resulting in a single pid section, with one track per thread for the spans, and a single track for each counter where the same counters from different ring_ids are merged. Domain local counters get all smashed on a single track, resulting in a meaningless plot (for intance, major_heap_words looks like a square signal, at each slice, the last domain to emit its counter event "wins" the race for the value of the plot until the next slice). This patch gives the behaviour of JSON to FXT.
ngorogiannis
approved these changes
Nov 18, 2026
| let emit_counter trace ~ring_id ~ts ~name ~value = | ||
| let t_ref = trace.doms.(ring_id) and time_ns = ts in | ||
| Trace.Event.Counter.encode trace.buf ~t_ref ~name ~time_ns | ||
| ~args:[ ("v", A_int value) ] |
Collaborator
There was a problem hiding this comment.
Regarding the remark in the PR summary that the changes might not lead to zero allocations: Initially I thought, here's about 3 allocations ([], (), A_int) that are traded for one (Counter).
However, the call is changed to drop the new allocations, two commits later.
Author
There was a problem hiding this comment.
Right. If we need to enforce zero alloc, we should have a principled approach to this, for instance with a test that preheats the lib and then does a series of calls, checking the GC allocated words before and after.
| [@@noalloc] | ||
|
|
||
| external put_event_header : | ||
| Bytes.t -> int -> int -> int -> int -> int -> int -> unit |
Collaborator
There was a problem hiding this comment.
Labels would be great here, to distinguish all the ints. Ditto for the next 2 functions.
The library is packing headers in 64-bit words and then writing them with intrinsic "%caml_bytes_set64u". This intrinsic write in platform native byte order, not little endian as expected by the format. This patch swaps the byte order using intrinsic "%bswap_int64" before the write (as the stdlib does for the safe version). Should make the FXT trace check tests pass on s390x. The generated code for header writing functions is still zero-alloc.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR extracts the Fuchsia library part of PR #89 by @tmcgilchrist and adds some related tests and fixes.
The patch was initially written in a performance improvement PR, but the current motivation is to make olly more standalone by dropping the external dependencies on
traceandtrace-fuchsia.Edits
Short summary of the patches
two preliminary patches to cherry-pick the main patch with less changes
A cram test that generates a trace with the external FXT library mocking the FXT features and format used by olly, printed in a normalized form, to serve as validation for the new implem / regression tests.
The main patch by @tmcgilchrist
The second record header output by the custom FXT lib (to set the timestamp unit) was not of the expected form, and the produced traces did not display properly in Perfetto. This patch fixes the record header.
The original PR was bumping the max_domain supported by olly to meet the OCaml value increased by 5.3 (128 --> 4096), since the new FXT library makes the change easier.
This PR extracts this change in two patches : one to fail more gracefully in case the monitored program has an unsupported value, and another that just bumps the value.
This patch is OPTIONAL, the new lib was using C stubs to avoid int64 boxing, and I was curious to see if OCaml was really that bad at unboxing. Turns out OCaml can producted non allocating code (see APPENDIX), but it's a case of "it works on my computer" (tested on 5.5.2, with and without flambds).
This is not strictly related, but I think it's worth putting in that batch of patches. The FXT trace for the same program is not as readable as the JSON one in Perfetto because some tracks are squashed. This patch aligns the FXT choices with the JSON ones. Maybe there was other reasons leading to this choice.
One of the original patches I forgot and
Remarks
- the FXT library definesexternal put_raw_64_le : Bytes.t -> int -> int64 -> unit = "%caml_bytes_set64u"which is misleading, i think this intrinsic uses native endianness, also we could rename it "unsafe", it's more accurate than "raw".AI disclosure
The FXT dump tool was written by Claude, then fully reviewed against the official format doc at fuchsia.dev, tweaked and commented by me.
I also used Claude's help for the dune file updates and the Unix commands in the cram tests.
All the english is my own broken english.
APPENDIX