Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logs ingested with OTeL not showing detected_level and Detected Fields #486

Closed
omri-shilton opened this issue Jun 25, 2024 · 10 comments
Closed

Comments

@omri-shilton
Copy link

We use OTeL for our logs and it works great.
In the regular explore it automatically structures the attributes as metadata and identifies the level.
In explore:
image

In explore logs it doesnt:
image
image

Are we doing something wrong?

@cyriltovena
Copy link
Collaborator

Structured metadata used by OTEL is not yet supported in Explore Logs but we're working on this actively. #310

@omri-shilton
Copy link
Author

omri-shilton commented Jun 26, 2024

Structured metadata used by OTEL is not yet supported in Explore Logs but we're working on this actively. #310

Will this include the level? What label does 'detected_level' search for?

@svennergr
Copy link
Contributor

Will this include the level? What label does 'detected_level' search for?

Just to be sure, which Loki version are you using and is the discover_log_levels turned on?

In this case it should get the error level based on the severity_number label:
https://github.com/grafana/loki/blob/23f200603f313a111667ca39d95a35455b32c233/pkg/distributor/distributor.go#L895-L916

@omri-shilton
Copy link
Author

Will this include the level? What label does 'detected_level' search for?

Just to be sure, which Loki version are you using and is the discover_log_levels turned on?

In this case it should get the error level based on the severity_number label: https://github.com/grafana/loki/blob/23f200603f313a111667ca39d95a35455b32c233/pkg/distributor/distributor.go#L895-L916

loki version 3.0.0 and yes, im using the default value which is true for discover_log_levels

@svennergr
Copy link
Contributor

Thanks @omri-shilton. In Loki 3.0.0 we wrote the level label instead of detected_level. The PR that changed this behavior is grafana/loki#12651 and is unfortunately not yet in a released version.

@svennergr
Copy link
Contributor

FYI, I'll keep this issue open as we might consider being backwards compatible. Will update you accordingly, @omri-shilton.

@omri-shilton
Copy link
Author

@svennergr is this still related to the fact that im using OTeL? because I dont have a label level or detected_level. i didnt configure anything special other than the fact to listen to otel logs and im sending them like instructed here: https://grafana.com/docs/enterprise-logs/latest/send-data/otel/

@svennergr
Copy link
Contributor

Yea, it's still related. I see the level label, which Loki automatically detected here:
image

@omri-shilton
Copy link
Author

Got it, im waiting for the PR on that fix. for now it doesnt even seem to have a PR.

@gtk-grafana
Copy link
Contributor

Should be fixed by #736

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants