Skip to content

bug: high memory consumption when the topic has a large number of tasks #680

Open
@Logioniz

Description

@Logioniz

topic_test.zip

Bug Report

YDB Python SDK version: 3.21.2

Environment

OS: ubuntu
Python: 3.12
CPU arch: x86_64

Current behavior:

A lot of RAM consumption by topic reader. The consumption is 100 times more than I have limited.

Expected behavior:

I expect it to be at least 20 or 30 times more than I set, because python objects are wasteful, but not 100 times.

Steps to reproduce:

I wrote simple example to reproduce this behaviour.
Need to start the producer for 1 minute and accumulate tasks in the topic. After 1 minute start consumer.
The producer will start consuming a large amount of memory within a minute.
To run example need to install deps in requirements.txt and set settings in settings.py.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions