Export `get_all_awaited_by` API into public Python API.

[corona10]

Currently get_all_awaited_by API is provided part of _remote_debugging module.
But the module is private currently, and there are some usecase for this API from aiomonitor.

So it would be useful if we could export it as part of the asyncio API, such as 'asyncio.get_all_awaited_by'.

Please let me know if there is concern to export this API as public. @kumaraditya303 @pablogsal

cc @achimnol

[achimnol]

aiomonitor provides TUI and GUI to continuously monitor the alive tasks and their task chains.
I'd like to use get_all_awaited_by() to enhance the performance and fidelity of the information, such as async contexts shown in the pstree command.

For now, I wish I could have a programmable, public access to get_all_awaited_by() which would be compatible within the minor release (e.g., 3.14 series).

In addition, for scalability when there are thousands of running tasks while repeating the queries by a few seconds interval, I'd like to also be able to read the task tree of a single task (or a set of tasks) for detailed views of individual tasks in aiomonitor, via exact task IDs or task name patterns.

• refs Async Call-Stack Reconstruction #91048

[pablogsal]

Sadly I think that for 3.14 I don't think we are ready to make these APIs public because we haven't prepare them for public consumption of anything other than proving it works and having a simple thing that dumps the task graph.

Making these APIs public now will corner us too quickly and may prevent optimising them or creating a better format.

[corona10]

Well, I am not saying about 3.14, maybe we can add it for 3.15. But we should define the specification of API and what attributes are actually needed, because without those API, they should hack too many things of internal CPython implementation.
Currently, they can use asyncio.tools._get_awaited_by_tasks, but it does not look safe in the future or minor release.

[achimnol]

Yes, I know 3.14 is already feature-frozen and the current private API is unstable. It will be a good starting point to experiment with, and in 3.15 we could introduce a more stable API.

[pablogsal]

My recommendation would be to play with the private version in 3.14 even if it can change and then help us design a better interface they can be performant for general use cases so we can expose it in 3.15.

This should not only include requirements but also architecture and performance considerations

[achimnol]

FYI, I'm testing the current private API here:
https://discuss.python.org/t/testing-asyncio-tools-get-all-awaited-by/92805

[achimnol]

My initial impressions:

• The returned tuple fields should be named explicitly (e.g., using dataclasses or named tuples) to make them backward-compatible when the schema changes in the future.
• It would be nice to have a subscription interface to "stream" the newly created tasks and terminated tasks, which should make less overheads when there are thousands of concurrent tasks.
  • I'm not sure how this could be implemented in the current "remote memory read" mechanism.

[kumaraditya303]

I would like to keep the API private for now as it relies heavily on the internal implementation detail of per thread tasks lists which I added in 3.14. It is fine to rely on private API for experimenting as it is unlikely it would change in 3.14 now but can change in 3.15.