-
Couldn't load subscription status.
- Fork 504
feat: add Agentless Hello World integration #15729
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
nkvoll
wants to merge
9
commits into
main
Choose a base branch
from
add-agentless-hello-world-package
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+386
−0
Open
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
071873c
feat: add Agentless Hello World integration
nkvoll 062af80
fix: update pull request link in changelog for initial release
nkvoll 48a3802
fix ecs reference
nkvoll c8daacf
add pipeline test
nkvoll 0d69977
add pipeline and system tests
nkvoll 4a8bdf2
Update packages/agentless_hello_world/data_stream/generic/elasticsear…
nkvoll 8b39cac
Update packages/agentless_hello_world/_dev/build/build.yml
nkvoll ac4b894
only store the status code
nkvoll 0c4599e
remove message field
nkvoll File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| dependencies: | ||
| ecs: | ||
| reference: [email protected] |
10 changes: 10 additions & 0 deletions
10
packages/agentless_hello_world/_dev/deploy/docker/config.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| rules: | ||
| - path: / | ||
| methods: ["GET"] | ||
| responses: | ||
| - status_code: 418 | ||
| headers: | ||
| Content-Type: | ||
| - "application/json" | ||
| body: |- | ||
| {"this_is": "ignored"} |
14 changes: 14 additions & 0 deletions
14
packages/agentless_hello_world/_dev/deploy/docker/docker-compose.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| services: | ||
| epr_mock: | ||
| image: docker.elastic.co/observability/stream:v0.18.0 | ||
| hostname: epr_mock | ||
| ports: | ||
| - 8080 | ||
| volumes: | ||
| - ./config.yml:/config.yml | ||
| environment: | ||
| PORT: "8080" | ||
| command: | ||
| - http-server | ||
| - --addr=:8080 | ||
| - --config=/config.yml |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| - version: "0.1.0" | ||
| changes: | ||
| - description: Initial release. | ||
| type: enhancement | ||
| link: https://github.com/elastic/integrations/pull/15729 |
7 changes: 7 additions & 0 deletions
7
packages/agentless_hello_world/data_stream/generic/_dev/test/pipeline/test-hello-world.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| { | ||
| "events": [ | ||
| { | ||
| "message": "{\"status_code\":200}" | ||
| } | ||
| ] | ||
| } |
1 change: 1 addition & 0 deletions
1
...tless_hello_world/data_stream/generic/_dev/test/pipeline/test-hello-world.json-config.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| fields: {} |
23 changes: 23 additions & 0 deletions
23
...ss_hello_world/data_stream/generic/_dev/test/pipeline/test-hello-world.json-expected.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| { | ||
| "expected": [ | ||
| { | ||
| "agentless_hello_world": { | ||
| "generic": { | ||
| "status_code": 200 | ||
| } | ||
| }, | ||
| "ecs": { | ||
| "version": "9.1.0" | ||
| }, | ||
| "event": { | ||
| "category": [ | ||
| "web" | ||
| ], | ||
| "kind": "event", | ||
| "type": [ | ||
| "info" | ||
| ] | ||
| } | ||
| } | ||
| ] | ||
| } |
7 changes: 7 additions & 0 deletions
7
packages/agentless_hello_world/data_stream/generic/_dev/test/system/test-default-config.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| input: cel | ||
| service: epr_mock | ||
| data_stream: | ||
| vars: | ||
| url: http://{{Hostname}}:{{Port}} | ||
| assert: | ||
| hit_count: 1 |
18 changes: 18 additions & 0 deletions
18
packages/agentless_hello_world/data_stream/generic/agent/stream/cel.yml.hbs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| config_version: 2 | ||
| interval: 20s | ||
| resource.timeout: 15s | ||
| resource.url: "{{url}}" | ||
| state: | ||
| url: "{{url}}" | ||
| program: | | ||
| request("GET", state.url) | ||
| .do_request() | ||
| .as(resp, { | ||
| "events": [{ | ||
| "message": { | ||
| "status_code": resp.StatusCode | ||
| }.encode_json() | ||
| }] | ||
| }) | ||
| tags: | ||
| - agentless-hello-world |
88 changes: 88 additions & 0 deletions
88
packages/agentless_hello_world/data_stream/generic/elasticsearch/ingest_pipeline/default.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,88 @@ | ||
| --- | ||
| description: Pipeline for processing Agentless Hello World generic logs. | ||
| processors: | ||
| - set: | ||
| field: ecs.version | ||
| value: '9.1.0' | ||
| - remove: | ||
| field: | ||
| - organization | ||
| - division | ||
| - team | ||
| ignore_missing: true | ||
| if: ctx.organization instanceof String && ctx.division instanceof String && ctx.team instanceof String | ||
| tag: remove_agentless_tags | ||
| description: >- | ||
| Removes the fields added by Agentless as metadata, as they can collide with ECS fields. | ||
| - terminate: | ||
| tag: data_collection_error | ||
| if: ctx.error?.message != null && ctx.message == null && ctx.event?.original == null | ||
| - rename: | ||
| field: message | ||
| target_field: event.original | ||
| ignore_missing: true | ||
| if: ctx.event?.original == null | ||
| - remove: | ||
| field: message | ||
| tag: remove_message | ||
| ignore_missing: true | ||
| description: The `message` field is no longer required if the document has an `event.original` field. | ||
| if: ctx.event?.original != null | ||
| - json: | ||
| field: event.original | ||
| target_field: agentless_hello_world.generic | ||
| on_failure: | ||
| - set: | ||
| field: error.type | ||
| value: "json_parse_error" | ||
| - set: | ||
| field: error.message | ||
| value: "{{{ _ingest.on_failure_message }}}" | ||
| - set: | ||
| field: event.kind | ||
| value: event | ||
| - set: | ||
| field: event.type | ||
| value: [info] | ||
| - set: | ||
| field: event.category | ||
| value: [web] | ||
| - remove: | ||
| field: event.original | ||
| if: ctx.tags == null || !(ctx.tags.contains('preserve_original_event')) | ||
| ignore_failure: true | ||
| ignore_missing: true | ||
| - script: | ||
| lang: painless | ||
| description: This script processor iterates over the whole document to remove fields with null values. | ||
| source: | | ||
| void handleMap(Map map) { | ||
| for (def x : map.values()) { | ||
| if (x instanceof Map) { | ||
| handleMap(x); | ||
| } else if (x instanceof List) { | ||
| handleList(x); | ||
| } | ||
| } | ||
| map.values().removeIf(v -> v == null); | ||
| } | ||
| void handleList(List list) { | ||
| for (def x : list) { | ||
| if (x instanceof Map) { | ||
| handleMap(x); | ||
| } else if (x instanceof List) { | ||
| handleList(x); | ||
| } | ||
| } | ||
| } | ||
| handleMap(ctx); | ||
| on_failure: | ||
| - set: | ||
| field: event.kind | ||
| value: pipeline_error | ||
| - append: | ||
| field: error.message | ||
| value: >- | ||
| Processor '{{{ _ingest.on_failure_processor_type }}}' | ||
| {{#_ingest.on_failure_processor_tag}}with tag '{{{ _ingest.on_failure_processor_tag }}}' | ||
| {{/_ingest.on_failure_processor_tag}}failed with message '{{{ _ingest.on_failure_message }}}' | ||
23 changes: 23 additions & 0 deletions
23
packages/agentless_hello_world/data_stream/generic/fields/base-fields.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| - name: input.type | ||
| type: keyword | ||
| description: Input type | ||
| - name: data_stream.type | ||
| type: constant_keyword | ||
| description: Data stream type. | ||
| - name: data_stream.dataset | ||
| type: constant_keyword | ||
| description: Data stream dataset. | ||
| - name: data_stream.namespace | ||
| type: constant_keyword | ||
| description: Data stream namespace. | ||
| - name: event.module | ||
| type: constant_keyword | ||
| description: Event module | ||
| value: agentless_hello_world | ||
| - name: event.dataset | ||
| type: constant_keyword | ||
| description: Event dataset | ||
| value: agentless_hello_world.generic | ||
| - name: '@timestamp' | ||
| type: date | ||
| description: Event timestamp. |
6 changes: 6 additions & 0 deletions
6
packages/agentless_hello_world/data_stream/generic/fields/fields.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| - name: agentless_hello_world.generic | ||
| type: group | ||
| fields: | ||
| - name: status_code | ||
| type: long | ||
| description: HTTP Status Code |
15 changes: 15 additions & 0 deletions
15
packages/agentless_hello_world/data_stream/generic/manifest.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| title: Generic logs | ||
| type: logs | ||
| streams: | ||
| - input: cel | ||
| title: Generic logs | ||
| description: Collect generic logs from EPR endpoint. | ||
| template_path: cel.yml.hbs | ||
| vars: | ||
| - name: url | ||
| type: text | ||
| title: EPR URL | ||
| description: URL of the EPR endpoint (internal use only, for testing) | ||
| default: https://epr.elastic.co | ||
| required: false | ||
| show_user: false |
69 changes: 69 additions & 0 deletions
69
packages/agentless_hello_world/data_stream/generic/sample_event.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| { | ||
| "@timestamp": "2025-10-23T11:25:00.349Z", | ||
| "agent": { | ||
| "ephemeral_id": "2f0402ea-00e1-47fa-944c-1e34d91fdc2f", | ||
| "id": "0909c464-2093-4f85-8bf7-b11593587146", | ||
| "name": "elastic-agent-93305", | ||
| "type": "filebeat", | ||
| "version": "9.1.3" | ||
| }, | ||
| "agentless_hello_world": { | ||
| "generic": { | ||
| "status_code": 418 | ||
| } | ||
| }, | ||
| "data_stream": { | ||
| "dataset": "agentless_hello_world.generic", | ||
| "namespace": "88559", | ||
| "type": "logs" | ||
| }, | ||
| "ecs": { | ||
| "version": "9.1.0" | ||
| }, | ||
| "elastic_agent": { | ||
| "id": "0909c464-2093-4f85-8bf7-b11593587146", | ||
| "snapshot": false, | ||
| "version": "9.1.3" | ||
| }, | ||
| "event": { | ||
| "agent_id_status": "verified", | ||
| "category": [ | ||
| "web" | ||
| ], | ||
| "dataset": "agentless_hello_world.generic", | ||
| "ingested": "2025-10-23T11:25:03Z", | ||
| "kind": "event", | ||
| "module": "agentless_hello_world", | ||
| "type": [ | ||
| "info" | ||
| ] | ||
| }, | ||
| "host": { | ||
| "architecture": "aarch64", | ||
| "containerized": false, | ||
| "hostname": "elastic-agent-93305", | ||
| "ip": [ | ||
| "172.30.0.2", | ||
| "172.18.0.4" | ||
| ], | ||
| "mac": [ | ||
| "36-F7-E4-8A-31-61", | ||
| "B2-C0-07-A9-21-9B" | ||
| ], | ||
| "name": "elastic-agent-93305", | ||
| "os": { | ||
| "family": "", | ||
| "kernel": "6.10.14-linuxkit", | ||
| "name": "Wolfi", | ||
| "platform": "wolfi", | ||
| "type": "linux", | ||
| "version": "20230201" | ||
| } | ||
| }, | ||
| "input": { | ||
| "type": "cel" | ||
| }, | ||
| "tags": [ | ||
| "agentless-hello-world" | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| # Agentless Hello World | ||
|
|
||
| This is a sample integration designed to exercise the Agentless infrastructure. It periodically fetches data from `https://epr.elastic.co` every minute to demonstrate basic agentless functionality. | ||
|
|
||
| ## Overview | ||
|
|
||
| The Agentless Hello World integration is a minimal example that: | ||
| - Fetches data from the Elastic Package Registry (EPR) endpoint | ||
| - Runs every 1 minute | ||
| - Requires no user configuration | ||
|
|
||
| ## Configuration | ||
|
|
||
| This integration requires no configuration from the user. All settings are pre-configured: | ||
| - **Endpoint**: `https://epr.elastic.co` | ||
| - **Interval**: 1 minute | ||
| - **Deployment mode**: Agentless by default | ||
|
|
||
| ## Data Collection | ||
|
|
||
| The integration makes HTTP GET requests to `https://epr.elastic.co` and stores: | ||
| - **status_code**: HTTP Status Code for the response. | ||
|
|
||
| ## Requirements | ||
|
|
||
| ### Agentless-enabled integration | ||
|
|
||
| Agentless integrations allow you to collect data without having to manage Elastic Agent in your cloud. They make manual agent deployment unnecessary, so you can focus on your data instead of the agent that collects it. For more information, refer to [Agentless integrations](https://www.elastic.co/guide/en/serverless/current/security-agentless-integrations.html) and the [Agentless integrations FAQ](https://www.elastic.co/guide/en/serverless/current/agentless-integration-troubleshooting.html). | ||
|
|
||
| Agentless deployments are only supported in Elastic Serverless and Elastic Cloud environments. This functionality is in beta and is subject to change. Beta features are not subject to the support SLA of official GA features. | ||
|
|
||
| ## Logs | ||
|
|
||
| ### Generic | ||
|
|
||
| The generic data stream collects responses from the EPR endpoint. | ||
|
|
||
| **ECS Field Reference** | ||
|
|
||
| Please refer to the following document for detailed information on ECS fields: | ||
| - [Elastic Common Schema](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) | ||
|
|
||
| **Exported fields** | ||
|
|
||
| | Field | Description | Type | | ||
| |---|---|---| | ||
| | @timestamp | Event timestamp. | date | | ||
| | agentless_hello_world.generic.status_code | The HTTP Status Code of the response. | long | | ||
| | data_stream.dataset | Data stream dataset. | constant_keyword | | ||
| | data_stream.namespace | Data stream namespace. | constant_keyword | | ||
| | data_stream.type | Data stream type. | constant_keyword | | ||
| | event.dataset | Event dataset | constant_keyword | | ||
| | event.module | Event module | constant_keyword | |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.