Skip to content

feat(falcon): paginate event search results with a 100-event page cap#224

Closed
m-mizutani wants to merge 3 commits into
mainfrom
worktree-vivid-hatching-pretzel
Closed

feat(falcon): paginate event search results with a 100-event page cap#224
m-mizutani wants to merge 3 commits into
mainfrom
worktree-vivid-hatching-pretzel

Conversation

@m-mizutani

Copy link
Copy Markdown
Contributor

Summary

falcon_search_events previously returned the entire EDR event result set straight to the agent, flooding the LLM context with potentially thousands of events. This change paginates the results: at most 100 events are returned per call, along with the total result-set size, and later pages are served from a stored snapshot without re-running the query.

Background (investigation)

  • CrowdStrike NGSIEM only exposes the asynchronous queryjobs API — there is no streaming/export endpoint and no mimeType parameter.
  • A filter (non-aggregate) query returns at most 200 events by default; more (up to 20,000 on Falcon LogScale Cloud) requires | tail(N) in the query string. The exact match count is best obtained via | count().
  • The query job poll returns the cumulative result set on every poll, so accumulating events across polls duplicated them — a latent bug in the previous implementation.

Changes

  • Add limit (default 100, max 100), offset, and result_set_id parameters to falcon_search_events. Responses now include total, offset, returned, has_more, and result_set_id.
  • On a new search, snapshot the full result set to the shared Cloud Storage bucket as newline-delimited JSON (streamed one event at a time, under falcon/events/). Subsequent pages are read back via result_set_id without re-querying — stable across calls and multi-instance safe (no in-process state, matching Warren's stateless design).
  • Tool description guides the model to use | tail(N) for more events and | count() for exact totals; the tool itself does not modify the query.
  • Fix the cumulative-poll duplication bug (use only the final done response) and the metadataResultmetaData response-key typo. When the API reports metaData.eventCount, surface it as total_matched.
  • Wire the shared storage client/prefix into the Falcon agent via a new optional agents.StorageAware interface; no new CLI flags (reuses --storage-bucket / --storage-prefix). Without storage, only the first page is returned.

Tests

  • Updated event-search tests and added coverage for snapshot pagination (no re-query on later pages), limit clamping, total_matched from metadata, missing result set, the cumulative no-duplication fix, and the pagination helpers.
  • go vet ./..., golangci-lint run, gosec, and go test ./... all pass.

🤖 Generated with Claude Code

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.

@m-mizutani

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces stable pagination and result-set snapshotting for Falcon event searches by utilizing a shared storage client. It defines a new StorageAware interface to inject storage configurations into agent factories, implements NDJSON-based snapshotting in the Falcon agent, and updates the CLI commands to support this integration. The review feedback highlights a valuable performance optimization: instead of decoding the entire NDJSON snapshot into memory within decodeNDJSON, the implementation can be optimized to decode only the requested page of events and discard the rest, thereby reducing memory overhead and garbage collection pressure for large result sets.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread pkg/agents/falcon/tool.go Outdated
Comment thread pkg/agents/falcon/tool.go Outdated
@m-mizutani m-mizutani closed this Jun 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant