Fluent Bit v4.0.1 in Buildroot?? - Need support!

[ThomasDevoogdt]

Hi, I would like to bump the fluent-bit package in buildroot, see https://github.com/buildroot/buildroot/tree/master/package/fluent-bit.

Currently, a lot of custom patches are taken along to get things build. I would like to upstream as much as possible. I opened a bunch of pull requests to get this done, but I see little to no movement on all of them. Please, help me out to get these landed. It would make my life easier if these are just getting proper review and attention. And it would make fluent-bit available in a widely used platform.

Here is an (ordered list of importance) of pull requests that I really want to see landed in e.g. v4.0.1:

• plugins: kafka: fix cmake cross compile error #9600
• build: use the system provided libmsgpack and libsqlite3 if found #9572
• lib: luajit-cmake: CMakeLists.txt: do not require a CXX compiler #9815
• lib: nghttp2: CMakeLists.txt: do not require a CXX compiler #9816
• lib: kafka: CMakeLists.txt: do not require a CXX compiler  #9817
• workflows: add sanity check for toolchains w/ CXX support #9277
• build: only compile simdutf if there is CXX support #10277
• plugins: fix compilation if FLB_SQLDB (sqlite3) is disabled #10239

Here is a list of issues I still don't got any answer on:

• Building without FLB_SQLDB (sqlite3) support completely broken. #9757

@edsiper

[leonardo-albertovich]

While I support the idea I think we need to be cautious about the approach. Modifying dependency provided files means someone would have to keep merging those patches when we update the libraries or a regression could be caused (which wouldn't be caught by our tests).

[leonardo-albertovich]

Maybe we should include those patches in a way that allows us to apply them on top of the original files as part of the build process?

[ThomasDevoogdt]

Maybe we should include those patches in a way that allows us to apply them on top of the original files as part of the build process?

These patches in buildroot have all a reference to a pull request on this repo. Some are indeed merged, but others not. I'm just reaching out here to have a look to the patches which are not yet merged. I summarized them above. I also try to upstream patches further than only this repo, e.g. PR #9816 fixes something in nghttp2, but the actual fix is already upstreamed, I'm just waiting for review.

[leonardo-albertovich]

I understand what you mean but I brought up the issue to try to steer things in the right direction to avoid future problems regardless of there are pre existing changes.

[ThomasDevoogdt]

@leonardo-albertovich, thx for the support, but what I really need is someone that review and merges my pull requests. Can you ping the right people to do so? It's quite hard to get anything merged in this repo. And I really want to reduce my patch list in buildroot. I listed all relevant pull requests in the issue description. Just go over them, one by one. Thanks in advance!

[ThomasDevoogdt]

@edsiper I rebased

• build: use the system provided libmsgpack and libsqlite3 if found #9572
• workflows: add sanity check for toolchains w/ CXX support #9277

Please have a look at them. FYI, Buildroot has been bumped to v4.0.0 in commit buildroot/buildroot@8181727.

[leonardo-albertovich]

@leonardo-albertovich, thx for the support, but what I really need is someone that review and merges my pull requests. Can you ping the right people to do so? It's quite hard to get anything merged in this repo. And I really want to reduce my patch list in buildroot. I listed all relevant pull requests in the issue description. Just go over them, one by one. Thanks in advance!

I understand your frustration but as a maintainer I need to take the implications of the changes you want to make in consideration to minimize the possibility of them causing problems in the future and as I said, changing files from dependencies will either shift the burden of maintaining those changes to whoever is working on the project in the future or be the root cause of a regression if whoever updates the dependency next time doesn't notice that there were customizations.

[edsiper]

Hi, thanks for the contribution, currently we dont allow direct changes in our dependencies so I agree with @leonardo-albertovich .

Looking forward, we just upgraded a bunch of dependencies here: #10195

would you please check if #10195 solves the issue with dependencies ?

[ThomasDevoogdt]

@edsiper @leonardo-albertovich Yes, I understand that you don't want to update any dependency. But almost all my PRs are about compile fixes which are either in fluent-bit itself, or in a dependency, but already upstreamed.

• plugins: kafka: fix cmake cross compile error #9600 is purely in fluent-bit, and fixes cross-compilation.
• build: use the system provided libmsgpack and libsqlite3 if found #9572 is about using system dependencies rather than build-in dependencies, is purely in fluent-bit. (This PR comes with a workflow file, already rebased to work with the new Ubuntu container.
• workflows: add sanity check for toolchains w/ CXX support #9277 is the only PR which fixes stuff in the dependencies itself. But note that there, almost all commits are upstreamed. And, also these one comes with a workflow, so there is no real chance on regression.

It would be nice if at least the first one gets merged, and ideally also the second.

would you please check if #10195 solves the issue with dependencies ?

I checked, and I could close #9816, thanks!

[ThomasDevoogdt]

Hi @edsiper, @leonardo-albertovich, @cosmo0920, and others,

In the meantime, I could drastically reduce my custom patch set, thanks to the v4.0.2 release. (Thanks for that!)
See buildroot/buildroot@84d9cf3.

There are still some PRs open which could drop the remaining patches that I have to carry along.

• plugins: kafka: fix cmake cross compile error #9600
• plugins: fix compilation if FLB_SQLDB (sqlite3) is disabled #10239
• lib: luajit: only require a C compiler #10318
• lib: zstd: only require a C compiler #10316
• lib: miniz: only require a C compiler #10315

It would also be nice if the PR bellow could be merged. I know, one patch is not yet upstream, but it comes with a workflow file to enforce functionality.

• workflows: add sanity check for toolchains w/ CXX support #9277

[leonardo-albertovich]

Great, I can see that two out of three library patches have been upstreamed already so we'll definitely update the dependencies ASAP.

• plugins: kafka: fix cmake cross compile error #9600 is out of my ballpark and it's ok IMO
• plugins: fix compilation if FLB_SQLDB (sqlite3) is disabled #10239 needs changes
• workflows: add sanity check for toolchains w/ CXX support #9277 mixes workflow changes with dependency changes so it cannot be merged as is, I've left a note there

Thank you.