Sharing data between prefork threads

[mnettlingham]

I've just about finished my first mojo app, so should be considered a beginner. It's a restful API using the OpenAPI plugin.
It works fine under morbo, but there are a couple of components which are causing issues when running via the prefork server.

I have a caching layer (self-written), and I also collate stats (Net::Prometheus) regarding both requests per user, and cache usage. Under prefork, these are local to each thread, meaning that we cache everything once per thread, slowing things down, and increasing the memory usage. Stats are pulled by an external service via a /stats endpoint, and obviously it gets different sets of stats depending on the thread serving the response.

Both the caching and stats are currently stored at the main 'Mojo::Base' level

I'd rather avoid moving to an external cache (e.g. redis), and also would rather not push the stats to something like an influx instance, in order to return the collated stats.

So is there a way of persisting data (in-memory) that can be shared between these threads?

[s1037989]

Check out Mojolicious::Plugin::Status, by the same author as Mojolicious itself. Sounds like it's doing remarkably similar things to what you're doing.

…

On Wed, Nov 15, 2023, 5:01 AM mnettlingham ***@***.***> wrote: I've just about finished my first mojo app, so should be considered a beginner. It's a restful API using the OpenAPI plugin. It works fine under morbo, but there are a couple of components which are causing issues when running via the prefork server. I have a caching layer (self-written), and I also collate stats (Net::Prometheus) regarding both requests per user, and cache usage. Under prefork, these are local to each thread, meaning that we cache everything once per thread, slowing things down, and increasing the memory usage. Stats are pulled by an external service via a /stats endpoint, and obviously it gets different sets of stats depending on the thread serving the response. Both the caching and stats are currently stored at the main 'Mojo::Base' level I'd rather avoid moving to an external cache (e.g. redis), and also would rather not push the stats to something like an influx instance, in order to return the collated stats. So is there a way of persisting data (in-memory) that can be shared between these threads? — Reply to this email directly, view it on GitHub <#2131>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAD6K6RGKJVCFODGWDVXRC3YESOIJAVCNFSM6AAAAAA7MJRP7SVHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZVHA2TIMBUHA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[mnettlingham]

Thanks - that is useful, but my stats need to be in a specific format, as they're consumed by a service that collates stats across all services. I might have a dig into the code, though, as that is clearly able to interrogate all child processes.

[mnettlingham]

Looks like Mojo::MemoryMap might be a good place to start. Appreciate the hint!