Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions lib/olly_common/cli.ml
Original file line number Diff line number Diff line change
Expand Up @@ -41,14 +41,14 @@ let proc_stat_freq_option =
RSS of the monitored process are sampled. Sampling runs on a dedicated \
domain, independently of $(b,--freq). The value must be positive. Where \
the resident pages of the runtime events ring buffer can be attributed to \
it, currently on Linux only, the peak is the maximum over samples of the \
RSS less the ring: only the current RSS can be decomposed that way, so a \
it, on Linux and OSX, the peak is the maximum over samples of the RSS \
less the ring: only the current RSS can be decomposed that way, so a \
shorter interval lowers the chance of missing a transient peak at the \
cost of more sampling overhead, a genuine accuracy/overhead tradeoff. \
Where they cannot, on OSX and FreeBSD and wherever the ring's mapping \
could not be found, Linux falls back to the exact peak the kernel \
maintains, which each sample can only raise, while OSX and FreeBSD report \
only the current RSS and the peak is again approximated from the samples."
Where they cannot, on FreeBSD and wherever the ring's mapping could not \
be found, Linux falls back to the exact peak the kernel maintains, which \
each sample can only raise, while OSX and FreeBSD report only the current \
RSS and the peak is again approximated from the samples."
in
Arg.(
value
Expand Down
102 changes: 91 additions & 11 deletions lib/olly_common/max_rss_stubs.c
Original file line number Diff line number Diff line change
Expand Up @@ -22,16 +22,13 @@
* One function rather than two because the numbers are not
* independent on all platforms.
*
* Neither number is a peak: the platform that implements this reports
* Neither number is a peak: both platforms that implement this report
* the RSS as it is at the sample, and the caller tracks the peak of the
* difference. That is forced rather than chosen — see the Linux arm.
*
* Only Linux identifies the ring so far. On macOS the per-region
* breakdown of proc_pidinfo(PROC_PIDREGIONPATHINFO) would locate it; on
* FreeBSD, struct kinfo_proc has ki_rssize (RSS) and ki_size (total VM)
* but not a heap/stack split, and libprocstat would give the per-mapping
* breakdown needed. Neither is attempted here, so both leave the ring
* in.
* On FreeBSD, struct kinfo_proc has ki_rssize (RSS) and ki_size (total
* VM) but not a heap/stack split; libprocstat would give the per-mapping
* breakdown needed to locate the ring.
*/

#include <caml/alloc.h>
Expand Down Expand Up @@ -104,7 +101,7 @@ static void status_rss_kb(int pid, long *rss_kb, long *hwm_kb) {
* once the file has been unlinked.
*
* One mmap is one VMA, but mprotect and madvise can split it, so scan
* over the consecutive matching ones. */
* over the consecutive matching ones as the macOS arm does. */
static int ring_range(int pid, const char *ring_file, unsigned long *lo,
unsigned long *hi) {
char path[64];
Expand Down Expand Up @@ -258,17 +255,100 @@ static struct rss_sample platform_sample(int pid, const char *ring_file) {
#elif defined(__APPLE__)

#include <libproc.h>
#include <sys/resource.h>
#include <sys/proc_info.h>
#include <sys/stat.h>
#include <unistd.h>

/* Resident kB of the VM regions of [pid] that are backed by the vnode
* (dev, ino) and lie at or above [from], or -1 if there is no such
* region. On success [*found_at] is the address of the first one.
*
* Regions are matched on their backing vnode rather than on
* prp_vip.vip_path, which holds only the "tail end" of long paths and
* need not be spelled the way the caller spelled it. A large mapping
* can be split across several adjacent regions, so sum over all of the
* consecutive matching ones. */
static long vnode_resident_kb(int pid, uint64_t from, uint32_t dev,
uint64_t ino, uint64_t *found_at) {
long page_kb = getpagesize() / 1024;
uint64_t addr = from;
long total = 0;
int found = 0;

for (;;) {
struct proc_regionwithpathinfo r;
uint64_t next;

if (proc_pidinfo(pid, PROC_PIDREGIONPATHINFO, addr, &r, sizeof(r)) <= 0)
break; /* past the last region, or the process is gone */
if (r.prp_vip.vip_vi.vi_stat.vst_dev == dev &&
r.prp_vip.vip_vi.vi_stat.vst_ino == ino) {
if (!found) {
found = 1;
*found_at = r.prp_prinfo.pri_address;
}
total += (long)r.prp_prinfo.pri_pages_resident * page_kb;
} else if (found)
break; /* past the last matching region */

next = r.prp_prinfo.pri_address + r.prp_prinfo.pri_size;
if (next <= addr)
break; /* no progress: bail out rather than spin */

addr = next;
}
return found ? total : -1;
}

/* Where the ring was last seen. Only the poller domain calls in here,
* for one process per run. */
static int ring_hint_pid = 0;
static uint64_t ring_hint_addr = 0;

/* Counting a region's resident pages costs the kernel a walk of its page
* list (~0.35ms for a 256MB-resident ring), and reaching the region at
* all costs a walk of every region below it (~1ms). The ring's mapping
* never moves, so remember where it was and start there next time,
* falling back to the full walk if it is no longer at that address. */
static long ring_kb(int pid, const char *ring_file) {
struct stat st;

/* [stat] rather than a string comparison against the region's path:
* this resolves symlinks, so the directory olly was given need not be
* the canonical one. */
if (stat(ring_file, &st) != 0)
return -1;

uint32_t dev = (uint32_t)st.st_dev;
uint64_t ino = (uint64_t)st.st_ino;
uint64_t hint = (ring_hint_pid == pid) ? ring_hint_addr : 0;
uint64_t found_at = 0;
long total = -1;

if (hint != 0)
total = vnode_resident_kb(pid, hint, dev, ino, &found_at);
if (total < 0)
total = vnode_resident_kb(pid, 0, dev, ino, &found_at);

if (total >= 0) {
ring_hint_pid = pid;
ring_hint_addr = found_at;
}
return total;
}

/* proc_pidinfo(PROC_PIDREGIONPATHINFO) reports the per-region breakdown
* that would locate the ring; until it is used, the ring is left in. */
static struct rss_sample platform_sample(int pid, const char *ring_file) {
struct rss_sample s = {-1, -1, -1};
struct proc_taskinfo ti;
struct rusage ru;

(void)ring_file;
if (ring_file)
s.ring_kb = ring_kb(pid, ring_file);
if (proc_pidinfo(pid, PROC_PIDTASKINFO, 0, &ti, sizeof(ti)) > 0)
s.rss_kb = (long)(ti.pti_resident_size / 1024);
if(getrusage(RUSAGE_SELF, &ru) == 0)
s.hwm_kb = ru.ru_maxrss / 1024;
return s;
}

Expand Down
7 changes: 4 additions & 3 deletions lib/olly_common/process_poller.ml
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,10 @@ let sample_peak_rss ~pid ~ring_file ~peak_rss ~excludes_ring =
if ring_kb > 0 then
(* On Linux the ring's pages are counted from the traced process's own
page tables, the same ones its RSS is drawn from, so there they
cannot exceed it. A platform that counts them some other way could
report pages the traced process never faulted in; clamp rather than
report a negative footprint. *)
cannot exceed it. On macOS the count comes from the backing region,
and the ring is shared with this process, so in principle it could
hold resident pages the traced process never faulted in; clamp
rather than report a negative footprint. *)
max 0 (rss_kb - ring_kb)
else rss_kb
in
Expand Down
2 changes: 1 addition & 1 deletion lib/olly_gc_stats/olly_gc_stats.ml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ let gc_stats_cmd =
runtime events ring buffer is mapped into the child, and is \
routinely much larger than the program's own live memory, so its \
resident pages are excluded from this figure where the platform \
permits it. Currently that is Linux only; elsewhere the figure \
permits it. Currently that is Linux and macOS; elsewhere the figure \
still includes the ring, which the human-readable output says and \
the json output reports as $(b,max_rss_excludes_ring)." );
`Blocks help_secs;
Expand Down
44 changes: 21 additions & 23 deletions test/cram/olly.t
Original file line number Diff line number Diff line change
Expand Up @@ -96,16 +96,15 @@ Trace subcommand help:
peak RSS of the monitored process are sampled. Sampling runs on a
dedicated domain, independently of --freq. The value must be
positive. Where the resident pages of the runtime events ring
buffer can be attributed to it, currently on Linux only, the peak
is the maximum over samples of the RSS less the ring: only the
current RSS can be decomposed that way, so a shorter interval
lowers the chance of missing a transient peak at the cost of more
sampling overhead, a genuine accuracy/overhead tradeoff. Where
they cannot, on OSX and FreeBSD and wherever the ring's mapping
could not be found, Linux falls back to the exact peak the kernel
maintains, which each sample can only raise, while OSX and FreeBSD
report only the current RSS and the peak is again approximated
from the samples.
buffer can be attributed to it, on Linux and OSX, the peak is the
maximum over samples of the RSS less the ring: only the current
RSS can be decomposed that way, so a shorter interval lowers the
chance of missing a transient peak at the cost of more sampling
overhead, a genuine accuracy/overhead tradeoff. Where they cannot,
on FreeBSD and wherever the ring's mapping could not be found,
Linux falls back to the exact peak the kernel maintains, which
each sample can only raise, while OSX and FreeBSD report only the
current RSS and the peak is again approximated from the samples.

COMMON OPTIONS
These options are common to all commands.
Expand Down Expand Up @@ -183,9 +182,9 @@ GC stats subcommand help:
ring buffer is mapped into the child, and is routinely much larger
than the program's own live memory, so its resident pages are
excluded from this figure where the platform permits it. Currently
that is Linux only; elsewhere the figure still includes the ring,
which the human-readable output says and the json output reports
as max_rss_excludes_ring.
that is Linux and macOS; elsewhere the figure still includes the
ring, which the human-readable output says and the json output
reports as max_rss_excludes_ring.

ARGUMENTS
EXECUTABLE
Expand Down Expand Up @@ -225,16 +224,15 @@ GC stats subcommand help:
peak RSS of the monitored process are sampled. Sampling runs on a
dedicated domain, independently of --freq. The value must be
positive. Where the resident pages of the runtime events ring
buffer can be attributed to it, currently on Linux only, the peak
is the maximum over samples of the RSS less the ring: only the
current RSS can be decomposed that way, so a shorter interval
lowers the chance of missing a transient peak at the cost of more
sampling overhead, a genuine accuracy/overhead tradeoff. Where
they cannot, on OSX and FreeBSD and wherever the ring's mapping
could not be found, Linux falls back to the exact peak the kernel
maintains, which each sample can only raise, while OSX and FreeBSD
report only the current RSS and the peak is again approximated
from the samples.
buffer can be attributed to it, on Linux and OSX, the peak is the
maximum over samples of the RSS less the ring: only the current
RSS can be decomposed that way, so a shorter interval lowers the
chance of missing a transient peak at the cost of more sampling
overhead, a genuine accuracy/overhead tradeoff. Where they cannot,
on FreeBSD and wherever the ring's mapping could not be found,
Linux falls back to the exact peak the kernel maintains, which
each sample can only raise, while OSX and FreeBSD report only the
current RSS and the peak is again approximated from the samples.

COMMON OPTIONS
These options are common to all commands.
Expand Down
Loading