Moved file memory leak

[darintay]

The inotify observer has a memory leak, specifically around moved files. I think I've tracked it down to this _moved_from_events dictionary in inotify_c.py. There is a clear_move_records() function that never appears to be called by anything.

See code below for a simple repro. Run as-is and it will gradually use up more and more memory. If you uncomment the 'clear_move_records' line, it will not.

Unfortunately I'm not familiar enough with the code to know when would be an appropriate time to actually call this function, or if it should be getting cleaned up in some other way.

from watchdog.events import PatternMatchingEventHandler
from watchdog.observers import Observer
from threading import Thread
from pathlib import Path


def thread_main():
    filename = "file1"
    Path(filename).write_text("dummy")
    for i in range(1000000):
        new_filename = f"{'a'*100}.{i}"
        Path(filename).rename(new_filename)
        filename = new_filename


observer = Observer()


class TestHandler(PatternMatchingEventHandler):
    def on_moved(self, event):
        # Uncomment this line to 'fix' the memory leak
        # list(observer.emitters)[0]._inotify._inotify.clear_move_records()
        print("MOVED")


def main():
    th = Thread(target=thread_main)
    th.start()
    handler = TestHandler(patterns=["*"])
    observer.schedule(handler, ".")
    observer.start()

    th.join()


if __name__ == "__main__":
    main()

[conara]

I experience the same issue, so I am wondering whether the workaround is a good option to mitigate this problem.

So adding list(observer.emitters)[0]._inotify._inotify.clear_move_records() to the on_moved callback.