Skip to content

Commit 3acb660

Browse files
committed
a clue in the right direction
Signed-off-by: clux <[email protected]>
1 parent e1dad01 commit 3acb660

File tree

2 files changed

+6
-3
lines changed

2 files changed

+6
-3
lines changed

docs/blog/posts/2024-06-11-reflector-memory.md

+5-3
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,7 @@ On top of that, the buffer in the `watcher` was not always released (quote from
9797
> The default system allocator never returns the memory to the OS after the burst, even if the objects are dropped. Since the initial list fetch happens sporadically you get a higher RSS usage together with the memory spike. Solving the burst will solve this problem, and reflectors and watchers can be started in parallel without worrying of OOM killers.
9898
> The allocator does not return the memory to the OS since it treats it as a cache. This is mitigated by using jemalloc with some tuning, however, you still get the memory burst so our solution was to use jemalloc + start the watchers sequentially. As you can imagine it's not ideal.
9999
100-
So in the end you have memory performance that is actually holding on to between 2x and 3x the actual store size at all times.
100+
So in the end you might actually be holding on to between 2x and 3x the actual store size at all times.
101101

102102
!!! note "watcher guarantee was designed for the store guarantee"
103103

@@ -150,9 +150,11 @@ The initial [synthetic benchmarks](https://github.com/kube-rs/kube/pull/1494#iss
150150

151151
!!! warning "Ad-hoc Benchmarks"
152152

153-
Whether the ad-hoc synthetic benchmarks are in any way realistic going forwards remains to be seen. How much you can get likely depends on a range of factors from allocator to usage patterns.
153+
Whether the ad-hoc synthetic benchmarks are in any way realistic going forwards remains to be seen. How much you can get likely depends on a range of factors from allocator choice to usage patterns.
154154

155-
__So far__, we have seen controllers with a basically unchanged profile, some with small improvements in the 10-20% range, but I have gotten one [50% drop in a real-world controller](https://github.com/kube-rs/kube/pull/1494#issuecomment-2126694967) (ironically, the one I used to test the change). So YMMV, particularly if you are doing a lot of other stuff, but please [reach out](https://discord.gg/tokio) with more results.
155+
__So far__, we have seen controllers with a basically unchanged profile, some with small improvements in the 10-20% range, and one [50% drop in a real-world controller](https://github.com/kube-rs/kube/pull/1494#issuecomment-2126694967) (ironically, the one I used to test the change). The default [Config::page_size] of 500 does seem to undermine the optimization somewhat. Try setting the page size to 50 to get a bigger effect.
156+
157+
YMMV, particularly if you are doing a lot of other stuff, but please [reach out](https://discord.gg/tokio) with more results.
156158

157159
## Thoughts for the future
158160

includes/links.md

+1
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@
2323
[ListParams]: https://docs.rs/kube/latest/kube/api/struct.ListParams.html
2424
[watcher::Config]: https://docs.rs/kube/latest/kube/runtime/watcher/struct.Config.html
2525
[watcher::Error]: https://docs.rs/kube/latest/kube/runtime/watcher/enum.Error.html
26+
[Config::page_size]: https://docs.rs/kube/latest/kube/runtime/watcher/struct.Config.html#method.page_size
2627
[controller::Config]: https://docs.rs/kube/latest/kube/runtime/controller/struct.Config.html
2728
[any_semantic]: https://docs.rs/kube/latest/kube/runtime/watcher/struct.Config.html#method.any_semantic
2829
[Object]: https://docs.rs/kube/latest/kube/core/struct.Object.html

0 commit comments

Comments
 (0)