out_azure_kusto: OTEL traces not getting logged at all with in_opentelemetry plugin

[dceravigupta]

I'm trying to send my application's OTEL traces to an Azure Kusto database using Fluent Bit, but it seems they are being dropped somewhere along the way. Nothing is getting logged in Kusto, and Fluent Bit isn't showing any errors, even in verbose mode.

I tried using stdout instead of azurekusto and everything seems to work, so seems like some issue with azure kusto plugin only.

Any guidance on how can I debug this further?

[patrick-stephens]

Can you follow the full issue template and provide the relevant config and version information? It's really hard to help without it.

I'm guessing maybe the traces are being ingested as non-log type data and the output plugin only supports log type data but it's hard to say without more details.

[dceravigupta]

I reviewed the code and noticed that traces are not currently enabled for the out_azure_kusto plugin. Even if we enable traces, there's another issue: the in_opentelemetry plugin sends traces in batches, with each record consists of multiple spans. We’ll need to flatten these spans into individual records before sending them to Azure Kusto, as it doesn’t natively support the OpenTelemetry traces schema.

@patrick-stephens is there anything available in Fluent Bit that can be used to flatten OpenTelemetry records into individual spans? Or would this need to be implemented directly within the out_azure_kusto plugin?

[patrick-stephens]

I think you will have to implement it

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.