BAAS-35403 track object with largest memory in vm value stack#130
Conversation
| return | ||
| } | ||
|
|
||
| valueMemUsage, _ := v.MemUsage(vm.r.stackMemUsageContext) |
There was a problem hiding this comment.
[minor] Although it rarely happens, what should we expect if v.MemUsage returns an error?
There was a problem hiding this comment.
Ah yeah, I'll add a comment saying that we will swallow the error. I didn't want to thread the logger through here for now since I don't see anywhere that err would ever be set to non-nil. Though I could be missing it
jwongmongodb
left a comment
There was a problem hiding this comment.
looks good for a first step. Do we want to create a test case in DEV with some fixed data in our atlas clusters from a range of documents totaling more than 300mb? We can have a scheduled event for every 1 minute, if that's not frequent enough, you can make it a database trigger with multiple scheduled triggers to insert more actions on the main trigger to do the same range scan (using a bunch of memory). We can compare the before and after memory usage etc.
Imo that would be overkill for now since we just log the value and take no action. But when we start enforcing this value, we can add a unit test that will specifically capture the corrected memory total in our function executions. Once we turn on the flag for candidate apps, we can use the logging to determine how effective this change is |
|
Made a change to clear the visitTracker each time we check memory since I noticed that we don't revise objects that may have changed between the last track. This is a bit of a performance hit, and we could try to optimize this, but looking at the |
This PR does the following things:
Add a flag to the runtime so we can feature flag the memory tracking change below
Track the object with the largest memory used on the goja value stack. Theoretically the Function objects should contain references to other objects we can check memory for. However the largest memory I see is from an Arguments object which might be from some sub function or call happening under the hood when the program finishes. Instead of skipping these types, we will just check member all types since I noticed some incorrect tracked sizes in the
BenchmarkVmMemTrackingbenchmarkAdded benchmarks to test both an entire program execution via the VM and test the vm stack push independently to isolate the mem tracking differences. The benchmarks have some varying results so pasting 5 runs of each below for some reference data. While the first benchmark shows that tracking memory this way does have some overhead. But the second benchmark shows that it's negligible with objects 10k+ bytes in size