feat(logs): send JSON via dict-encoded sets of keys by TheSafo · Pull Request #49404 · DataDog/datadog-agent

TheSafo · 2026-04-15T17:03:42Z

What does this PR do?

Motivation

Describe how you validated your changes

Additional Notes

TheSafo · 2026-04-15T17:04:10Z

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

agent-platform-auto-pr · 2026-04-15T17:12:05Z

Go Package Import Differences

Baseline: a903409
Comparison: 3154df3

binary	os	arch	change
agent	linux	amd64	+11, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust/flatbuffers/patterns +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc +github.com/google/flatbuffers/go
agent	linux	arm64	+11, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust/flatbuffers/patterns +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc +github.com/google/flatbuffers/go
agent	windows	amd64	+11, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust/flatbuffers/patterns +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc +github.com/google/flatbuffers/go
agent	darwin	amd64	+11, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust/flatbuffers/patterns +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc +github.com/google/flatbuffers/go
agent	darwin	arm64	+11, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust/flatbuffers/patterns +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc +github.com/google/flatbuffers/go
iot-agent	linux	amd64	+9, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc
iot-agent	linux	arm64	+9, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc
heroku-agent	linux	amd64	+11, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust/flatbuffers/patterns +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc +github.com/google/flatbuffers/go
cluster-agent	linux	amd64	+9, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc
cluster-agent	linux	arm64	+9, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc
cluster-agent-cloudfoundry	linux	amd64	+1, -0 +github.com/DataDog/agent-payload/v5/statefulpb
cluster-agent-cloudfoundry	linux	arm64	+1, -0 +github.com/DataDog/agent-payload/v5/statefulpb
dogstatsd	linux	amd64	+1, -0 +github.com/DataDog/agent-payload/v5/statefulpb
dogstatsd	linux	arm64	+1, -0 +github.com/DataDog/agent-payload/v5/statefulpb
process-agent	linux	amd64	+1, -0 +github.com/DataDog/agent-payload/v5/statefulpb
process-agent	linux	arm64	+1, -0 +github.com/DataDog/agent-payload/v5/statefulpb
process-agent	windows	amd64	+1, -0 +github.com/DataDog/agent-payload/v5/statefulpb
process-agent	darwin	amd64	+1, -0 +github.com/DataDog/agent-payload/v5/statefulpb
process-agent	darwin	arm64	+1, -0 +github.com/DataDog/agent-payload/v5/statefulpb
heroku-process-agent	linux	amd64	+1, -0 +github.com/DataDog/agent-payload/v5/statefulpb
security-agent	linux	amd64	+10, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc +github.com/DataDog/datadog-agent/pkg/trace/log
security-agent	linux	arm64	+10, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc +github.com/DataDog/datadog-agent/pkg/trace/log
security-agent	windows	amd64	+10, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc +github.com/DataDog/datadog-agent/pkg/trace/log
system-probe	linux	amd64	+10, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc +github.com/DataDog/datadog-agent/pkg/trace/log
system-probe	linux	arm64	+10, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc +github.com/DataDog/datadog-agent/pkg/trace/log
system-probe	windows	amd64	+10, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc +github.com/DataDog/datadog-agent/pkg/trace/log
system-probe	darwin	amd64	+10, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc +github.com/DataDog/datadog-agent/pkg/trace/log
system-probe	darwin	arm64	+10, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc +github.com/DataDog/datadog-agent/pkg/trace/log
otel-agent	linux	amd64	+9, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc
otel-agent	linux	arm64	+9, -0 +github.com/DataDog/agent-payload/v5/statefulpb +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering +github.com/DataDog/datadog-agent/pkg/logs/patterns/clustering/merging +github.com/DataDog/datadog-agent/pkg/logs/patterns/eviction +github.com/DataDog/datadog-agent/pkg/logs/patterns/processor +github.com/DataDog/datadog-agent/pkg/logs/patterns/tags +github.com/DataDog/datadog-agent/pkg/logs/patterns/token +github.com/DataDog/datadog-agent/pkg/logs/patterns/tokenizer/rust +github.com/DataDog/datadog-agent/pkg/logs/sender/grpc
privateactionrunner	linux	amd64	+1, -0 +github.com/DataDog/agent-payload/v5/statefulpb
privateactionrunner	linux	arm64	+1, -0 +github.com/DataDog/agent-payload/v5/statefulpb
privateactionrunner	windows	amd64	+1, -0 +github.com/DataDog/agent-payload/v5/statefulpb
privateactionrunner	darwin	amd64	+1, -0 +github.com/DataDog/agent-payload/v5/statefulpb
privateactionrunner	darwin	arm64	+1, -0 +github.com/DataDog/agent-payload/v5/statefulpb

TheSafo force-pushed the jsaf/json-encoding-updates branch 2 times, most recently from 9024ee9 to 5713ae7 Compare April 15, 2026 19:32

TheSafo mentioned this pull request Apr 16, 2026

fix(logs): rebuild libpatterns.so targeting glibc 2.30 (was 2.34) #49446

Draft

TheSafo force-pushed the jsaf/json-encoding-updates branch from 5713ae7 to 28670da Compare April 16, 2026 13:39

json updates

3154df3

TheSafo force-pushed the jsaf/json-encoding-updates branch from 28670da to 3154df3 Compare April 16, 2026 22:14

This was referenced Apr 16, 2026

fix(logs): send delete on tag eviction #49493

Draft

feat(logs): periodically remove unused entry from dict #49494

Draft

DDuongNguyen mentioned this pull request Apr 17, 2026

feat(logs): max template size limit + json_as_raw raw bypass #49552

Draft

This was referenced Apr 20, 2026

local proto plus servic #49628

Draft

better json #49672

Draft

fix(logs): staging fidelity fixes — tabs, newlines, Unicode positions, UTF-8 safety #49810

Draft

TheSafo changed the title ~~json updates~~ feat(logs): send JSON via keysets Apr 24, 2026

TheSafo changed the title ~~feat(logs): send JSON via keysets~~ feat(logs): send JSON via dict-encoded sets of keys Apr 24, 2026

This was referenced Apr 25, 2026

utf8 #49898

Draft

updated deletion #49961

Draft

prune inflight better #50034

Draft

dict_admission #50036

Draft

don't send deletes, tighten TTL #50047

Draft

tag manager addition #50049

Draft

This was referenced Apr 29, 2026

lazy snapshot sending #50120

Draft

ttl in config #50121

Draft

fix delta encoding #50133

Draft

flat-log-encoding #50321

Draft

extedned json nesting #50347

Draft

jsaf/flatten-json-encoding-values #50349

Draft

json presence #50402

Draft

This was referenced May 8, 2026

inflight fixes #50530

Draft

jsaf/updated-inflight-eviction #50557

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(logs): send JSON via dict-encoded sets of keys#49404

feat(logs): send JSON via dict-encoded sets of keys#49404
TheSafo wants to merge 1 commit intojsaf/dual-send-stackedfrom
jsaf/json-encoding-updates

TheSafo commented Apr 15, 2026

Uh oh!

TheSafo commented Apr 15, 2026 •

edited

Loading

Uh oh!

agent-platform-auto-pr Bot commented Apr 15, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

TheSafo commented Apr 15, 2026

What does this PR do?

Motivation

Describe how you validated your changes

Additional Notes

Uh oh!

TheSafo commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

agent-platform-auto-pr Bot commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Go Package Import Differences

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

TheSafo commented Apr 15, 2026 •

edited

Loading

agent-platform-auto-pr Bot commented Apr 15, 2026 •

edited

Loading