Serious slowdown when filter calls GetFrame(0) in its constructor

[pinterf]

We came across this problem during the updated ConvertToPlanarRGB tests.
Theoretically, the ConvertBits in this script is a NOP instruction, since nno bit depth conversion is performed.

However, if it calls GetFrame(0) in its constructor for frame-property detection, serious slowdown happens. When I removed the constructor-initiated GetFrame(0), it went back to normal speed. As a sidenote: ConvertToPlanarRGB is calling a similar GetFrame(0) but this has no slowing effect.

Blankclip(60000,width=1600, height=1600, pixel_type="YUV444P8")
ConvertToPlanarRGB(bits=8)
ConvertBits(8) # tested with and without this line.

The applied bits matched the whole flow: 8, 16 and 32 were used.

Benchmarks suggested that memory-cache size might also be involved in the problem. (<1% differences are only measurement errors: Avsmeter64 was used to get the fps data.)

# X,Y: Without-ConvertBits(n), With-ConvertBits(n) [fps]. ConvertBits constructor calls GetFrame(0)
# size:        400x400      600x600       800x800     1200x1200   1600x1600   2000x2000
# 8 bits       27376,27000  13200,13100  7600,7600    3351,3280   1720,1509    837,795
# 16 bits      24000,24000  11235,11200  6600,6100    1854,1420    671,599     380,353
# float        13900,14000    6300,5900  2600,1800     626, 545    285,270     178,171

Blank clip is single frame, and is generated once, then it goes into cache and remains there. Checked, true, we always get the very same single memory address for that precalculated frame. (BlankClip/Colorbars, all the same).

The in-constructor call GetFrame(0) is a usual way for obtaining clip-wide properties such as color matrix, full/limited/narrow range setting, etc.., assuming that such properties in frame #0 won't change across the whole clip. So the precalculations and the dispatched functions can be chosen once, during the filter instance creation.

Who is the culprit? Avisynth frame caching system? Or unfortunate memory layout? Memory? Mutexes waiting for each other?

Bad frame physical addresses? I read about that the cache lines and the address lookup and translation may depend on magic boundaries in processors. Logged the read and write pointers of ConvertToPlanarRGB for the different runs. But some thousand runs (automated, logged and analyzed) did not strengthen this possible reason.

At this point the free AI access was over :) Quickly subscribed my first AI month :)

I was adviced that what if I try GetFrame(100)? And yes, the slowdown was immediately over.

Reverted back to GetFrame(0) and after another advice I logged the first couple GetReadPtr and GetWritePtr pointers in ConvertToPlanarRGB and looked at it - this time within the same session. For the quick case (without ConvertBits) all addresses were the same. In the the slow cases the GetWritePtr pointers were altering between two addresses. And when ConvertBits's GetFrame(0) was removed, the addresses became identical again.

Those addresses came from env->NewVideoFrame, which in turn obtains the addresses from the buffer-reusing-helper, a so called Frame Registry.
Allocations based on Frame Registry are fast, really they are mostly not allocations. Frame Registry mechanism avoids OS re-allocations by giving back an unused (reference count = 0) frame/video frame buffer. Unused frames and buffer do not get freed up immediately, only in case of lack-of-memory.

So our write-pointer flip-flopped between two addresses at even-odd frames at the slow case. Analyzing the timings and the reference counts and even developing a last-released-first-reserved video frame buffer re-use logic, it still failed and was alternating between two addresses. (Btw: why it makes slower if writing into X address in even frames, Y address in odd frames instead of always writing into X address? This is subject to further investigation. Cache eviction?)

So far our good AI was helpful to detect what is not causing the problem.

The codebase is too complex and huge to pass an AI.

In the final solution I then tried to eliminate the creation of per-filter AvsCache instances prematurely, during the filter instantiation phase. Calling child->GetFrame(0) will construct a Cache object for the child (in our case, for ConvertToPlanarRGB), and fills it with frame #0 content, which somehow interacts with the future :)
I made another change: a No-Op filter simply returns one of its child/parameter clip unaltered. (e.g. during a clip1.ConvertBits(16) we detect that clip1 bit-depth is already 16-bits and return it.) In this case we prevent clip1 from getting yet another MTGuard and CacheGuard object.

[pinterf]

Reopened.
The "solution" resulted in very-very long setup times (5+ mins instead of seconds) in complex scripts e.g. m_qtgmc based ones. Use of caches are inevitable in the script preparation phase as well.

[DTL2020]

"Btw: why it makes slower if writing into X address in even frames, Y address in odd frames instead of always writing into X address? "

Performance of the memory subsystem on writes greatly depends on the size of X and Y buffers.

1. If sizeof(X)+sizeof(Y) significantly below CPU cache size (L1 or L2 or L3 and store performance depends on performance of these caches) - all writes may be done into caches with overwrite of old data and may never be flashed to host RAM. It is the best performance case.

2. If sizeof(X)+sizeof(Y) is significantly above CPU cache size - the performance will be limited by host RAM write performance. Because CPUs do not have enough cache lines to store data with the right to overwrite without flashing to RAM. After the X-buffer finished writing (to cache and possibly part to RAM) - the Y buffer is completely flushed to RAM and its cache lines overwritten by X buffer data (virtual address range). After the CPU starts to write the next Y buffer - it must first flash to RAM all non-saved (dirty) cachelines with X buffer data (and Y buffer write operations are stalled). The rate of preparing free cache lines is about host RAM write performance only.

If it possible in filtergraph - better performance solution is to write all (or a limited group) of X(even ?)-frames to X-buffer first (really only last write matter if there is no consumer to read each frame) and next write all Y(odd ?)-frames to Y-buffer. This will faster overwrite more cachelines without time to flash dirty cache lines to RAM and to stall write operations until flashing to RAM makes some cachelines free.

Also this is simple 1 core CPU model. With multi-core CPUs we also have some service cache coherency traffic between local cores caches (not shared ?). So it also may be better to process single cache lines addresses ranges with 1 core only (if 2 or more cores have same virtual addresses in its local L1/L2 caches - the additional service traffic may take bus at least to broadcast write operation in these address range and update data in the cache lines of other cores).

[pinterf]

Meanwhile I checked it with Intel VTune.

Also analyzed the cache and video frame registry and video frame buffer allocation and release life cycle.

Now I daresay that this is not really a slowdown. It depends. The glass is half full or half empty.

I think our quick situation is just the result of our special test script. The quick version is a very-very lucky memory and internal video frame cache life cycle coincidence.

Which - in this form - is not possible neither to detect, nor to heal.

Quickness assumes that the caches are evicted (and thus release the very same videoframe buffer reference) in such a way, that the next filter will always get the very same pointer address. Quick version always returned the same GetWritePtr for each frames.
Slow version returned two addresses, alternating, one for even, one for odd frames. Depending on which VideoFrameBuffer was losing its reference due to a previous evicted cache entry.

Similar to that you cannot say the OS that you want to get the same address for a malloc, that you just freed some milliseconds and processes ago, you cannot tell Avisynth you special needs. Thing, working in the background won't let it do.

[DTL2020]

About cache-friendly memory management google AI wrote:
A cache-friendly, Last-Free-First-Return (LFFR) memory allocator in C uses a LIFO (stack) structure, specifically a linked list of free blocks, to prioritize reusing recently freed memory. This approach keeps data "hot" in CPU cache. Implementing this requires malloc for initial pools and manual pointer manipulation to return recently freed blocks to the head of the list.
Cache-Friendly Memory Allocation Principles
Locality of Reference: Reusing the most recently freed memory keeps it in the CPU cache, reducing cache misses.
Avoid Fragmentation: Small, consistent allocations (like for node-based structures) benefit from specialized allocators (pool allocators).
Avoid System Calls: The standard malloc()/free() can be slow and fragmented; custom LIFO allocators are faster.
Implementation Concept (LFFR)
A LIFO-based, or stack-based, allocator holds a head pointer to the most recently freed block.
Allocation (alloc):
If head is not NULL, pop the block from the top of the stack and return it.
If head is NULL, allocate a new memory block using malloc.
Deallocation (free):
Push the freed block onto the top of the head pointer list.
Conceptual C Code Example
c
#include <stdlib.h>
#include <stdio.h>

typedef struct Block {
struct Block* next;
} Block;

Block* free_list = NULL;

// Cache-friendly allocation (LIFO)
void* cache_friendly_alloc(size_t size) {
if (free_list) {
Block* b = free_list;
free_list = free_list->next;
return (void*)b; // Return recently freed block
}
return malloc(size > sizeof(Block) ? size : sizeof(Block));
}

// LFFR Deallocation
void cache_friendly_free(void* ptr) {
if (!ptr) return;
Block* b = (Block*)ptr;
b->next = free_list;
free_list = b; // Push to top of stack
}
Key Considerations
Thread Safety: The example above is not thread-safe. For multi-threaded applications, use thread-local storage or spinlocks.
Use-cases: This is ideal for fixed-size object allocators, such as in game engines or real-time systems, but not for varying size requests.
Dangling Pointers: As with normal free(), ensure pointers to freed memory are set to NULL.

May be something like this already implemented in the internal AVS+ memory allocation ?

[pinterf]

(The ALTERNATIVE_VFB_TIMESTAMP was (is) the implementation the "return most recently feed" logic. It worked fine, but the alternating addresses still happened. It was because the videoframebuffer we'd expect was not yet freed, since it was referenced by the cache. By the time it was freed, the odd-frame videoframebuffer was inserted into the cache and the cache related to the even frame was evicted. even and odd frames were received always the last-before-one videoframebuffer, stepping into a forever-async loop (stabil chaos attractor had been reached :) ) F DTL2020 ***@***.***> ezt írta (időpont: 2026. febr. 15., V, 21:47):

…

*DTL2020* left a comment (AviSynth/AviSynthPlus#476) <#476 (comment)> About cache-friendly memory management google AI wrote: A cache-friendly, Last-Free-First-Return (LFFR) memory allocator in C uses a LIFO (stack) structure, specifically a linked list of free blocks, to prioritize reusing recently freed memory. This approach keeps data "hot" in CPU cache. Implementing this requires malloc for initial pools and manual pointer manipulation to return recently freed blocks to the head of the list. Cache-Friendly Memory Allocation Principles Locality of Reference: Reusing the most recently freed memory keeps it in the CPU cache, reducing cache misses. Avoid Fragmentation: Small, consistent allocations (like for node-based structures) benefit from specialized allocators (pool allocators). Avoid System Calls: The standard malloc()/free() can be slow and fragmented; custom LIFO allocators are faster. Implementation Concept (LFFR) A LIFO-based, or stack-based, allocator holds a head pointer to the most recently freed block. Allocation (alloc): If head is not NULL, pop the block from the top of the stack and return it. If head is NULL, allocate a new memory block using malloc. Deallocation (free): Push the freed block onto the top of the head pointer list. Conceptual C Code Example c #include <stdlib.h> #include <stdio.h> typedef struct Block { struct Block* next; } Block; Block* free_list = NULL; // Cache-friendly allocation (LIFO) void* cache_friendly_alloc(size_t size) { if (free_list) { Block* b = free_list; free_list = free_list->next; return (void*)b; // Return recently freed block } return malloc(size > sizeof(Block) ? size : sizeof(Block)); } // LFFR Deallocation void cache_friendly_free(void* ptr) { if (!ptr) return; Block* b = (Block*)ptr; b->next = free_list; free_list = b; // Push to top of stack } Key Considerations Thread Safety: The example above is not thread-safe. For multi-threaded applications, use thread-local storage or spinlocks. Use-cases: This is ideal for fixed-size object allocators, such as in game engines or real-time systems, but not for varying size requests. Dangling Pointers: As with normal free(), ensure pointers to freed memory are set to NULL. — Reply to this email directly, view it on GitHub <#476 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABS5GMJPJKDLBQUYT3EWP3D4MDLOHAVCNFSM6AAAAACUVK3GOOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTSMBVGE3DEMZYGY> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>