-
Notifications
You must be signed in to change notification settings - Fork 127
Add "No data visible in Kibana" - EDOT troubleshooting #2258
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
32ab1b5
Add "No data visible in Kibana" - EDOT troubleshooting
alexandra5000 f26df08
Remove empty urls from commented-out sections
alexandra5000 143f6b7
Apply comments
alexandra5000 4808782
Fix typo
alexandra5000 02359c3
Apply comment
alexandra5000 c504340
Merge remote-tracking branch 'upstream/main' into no-data-visible
alexandra5000 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,112 @@ | ||
--- | ||
navigation_title: No data visible in Kibana | ||
description: Learn what to check when no data (logs, metrics, traces) appears in Kibana after setting up EDOT. | ||
applies_to: | ||
stack: | ||
serverless: | ||
observability: | ||
product: | ||
edot_collector: ga | ||
products: | ||
- id: cloud-serverless | ||
- id: observability | ||
- id: edot-collector | ||
--- | ||
|
||
# No logs, metrics, or traces visible in Kibana | ||
|
||
If the EDOT Collector or SDKs appear to be running, but you see no logs, metrics, or traces in the {{kib}} UI, try to use these solutions to identify and resolve the issue. | ||
|
||
## Symptoms | ||
|
||
* Collector process is consuming CPU/memory but no telemetry data is visible in {{kib}} | ||
* APM services don’t show up in {{kib}} | ||
* Dashboards appear empty or partially loaded | ||
* The Collector is running without crash logs | ||
|
||
## Causes | ||
|
||
This issue is typically caused by one or more of the following: | ||
|
||
* Incorrect export endpoint | ||
* Missing or invalid API key/token | ||
* Network issues, such as proxy misconfigurations | ||
* TLS verification failures | ||
* Misconfigured pipelines or disabled signals | ||
* Incomplete setup — to capture all telemetry data, you must use the EDOT Collector together with an appropriate EDOT SDK | ||
|
||
## Resolution | ||
|
||
Use the following checks to identify and fix common configuration or connectivity issues that can prevent telemetry data from reaching {{kib}}. | ||
|
||
### Check export endpoint | ||
|
||
Make sure the Collector is configured to send data to the correct Elastic endpoint. | ||
|
||
To confirm: | ||
|
||
1. Open {{kib}} for your deployment. | ||
2. In the top right, select **Add data**. | ||
3. Select **Application** > **OpenTelemetry**. | ||
4. In the **APM Agents** panel, locate **OpenTelemetry**. | ||
5. Ensure the `endpoint` in your Collector configuration matches exactly. | ||
|
||
If you’re using the managed OTLP endpoint, confirm the region and cluster ID are correct. | ||
|
||
### Verify authentication headers | ||
|
||
Ensure the Collector or SDK includes an API key in the `Authorization` header: | ||
|
||
```yaml | ||
headers: | ||
Authorization: ApiKey <your-api-key> | ||
``` | ||
|
||
If you’re using environment variables, confirm the key is set correctly in the runtime context. | ||
|
||
### Review logs for export errors | ||
|
||
Common log messages include: | ||
|
||
``` | ||
permanent error: rpc error: code = Unavailable desc = connection refused | ||
``` | ||
|
||
Also look for: | ||
|
||
* TLS handshake failures | ||
* Invalid character errors, which may indicate proxy or HTML redirect instead of JSON | ||
|
||
Increase verbosity using `--log-level=debug` for deeper insights. <!--Refer to [Enable debug logging] for more information.--> | ||
|
||
### Test network connectivity | ||
|
||
You can validate connectivity using `curl`: | ||
|
||
```bash | ||
curl -v https://<endpoint> -H "Authorization: ApiKey <your-key>" | ||
``` | ||
|
||
Or use `telnet` or `nc` to verify port 443 is reachable. | ||
|
||
<!--### Check proxy environment variables | ||
|
||
Ensure environment variables are correctly set in your deployment. Refer to [EDOT proxy settings] for more information relevant to your configuration. | ||
|
||
In Kubernetes or container environments, pass these as `env:` entries. | ||
--> | ||
|
||
### Validate signal configuration | ||
|
||
Check that each pipeline is defined properly in your configuration: | ||
|
||
```yaml | ||
service: | ||
pipelines: | ||
traces: | ||
receivers: [otlp] | ||
processors: [...] | ||
exporters: [otlp] | ||
``` | ||
|
||
If only logs are configured, metrics and traces will not be sent. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.