docs: add PPL language reference with data-grounded examples#142
Closed
anirudha wants to merge 6 commits intoopensearch-project:mainfrom
Closed
docs: add PPL language reference with data-grounded examples#142anirudha wants to merge 6 commits intoopensearch-project:mainfrom
anirudha wants to merge 6 commits intoopensearch-project:mainfrom
Conversation
Add a full PPL (Piped Processing Language) documentation section to the Observability Stack docs, positioning PPL as the native query language for logs and traces. New pages: - PPL overview with comparison to KQL and EQL - Command reference summary (50+ commands) - 27 detailed per-command reference pages with Description, Syntax, Arguments, Usage notes, Basic/Extended examples, and See also - Function reference (200+ functions across 13 categories) - Observability examples with live playground links for OTel data Commands documented individually: search, where, fields, eval, rename, fillnull, expand, flatten, stats, eventstats, streamstats, timechart, trendline, sort, head, dedup, top, rare, parse, grok, rex, patterns, spath, join, lookup, ml, describe Updated: - Sidebar with categorized PPL command navigation - Main docs index with PPL section and LinkCard - Investigate page with links to new PPL reference - README with PPL section and example query Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace every generic example (accounts, gender, age, etc.) across all 27 PPL command pages with real observability data from logs-otel-v1* and otel-v1-apm-span-* indices. All queries validated against the local OpenSearch PPL API endpoint. Also remove duplicate PPL Commands/Functions entries from the Reference sidebar section. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Escape single quotes in RISON encoding (%27 → !%27) to prevent premature termination of query strings containing PPL literals - Widen time range from now-15m to now-6h so playground shows data Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Reorder sidebar: Overview, Get Started, Send Data, PPL, Discover, Agent Observability, Application Monitoring, Dashboards & Visualize, Alerting, Agent Health, SDKs/MCP & Clients, Claude Code. Rename "Alerting & Detection" to "Alerting" and "Reference" to "SDKs, MCP & Clients". Replace all em dashes with hyphens across 71 doc files. Fix anchor links broken by the em dash removal. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Rewrite grok/rex/parse/spath examples with verified patterns from real OTel data (Envoy access logs, Kafka broker logs) instead of fabricated Apache log patterns - Fix expand/flatten docs to use OTel indices instead of fabricated my-index - Add Data Prepper flat schema notes to expand/flatten - Fix timechart trace example with timefield=startTime - Fix head.md dedup+head example that was missing dedup - Fix search.md operator precedence note (PPL OR>AND differs from SQL) - Add stats earliest()/latest() example, where BETWEEN example - Fix 22 broken See Also links across 17 command docs to point to specific command pages instead of generic index - Add masterclass pipeline examples to examples.md (service health scorecard, GenAI cost analysis, Envoy log parsing, error pattern discovery, cross-signal log-trace correlation) - Add advanced analytics section (eventstats, streamstats, trendline) - Generate 28 new playground URLs for all added examples - Build validates with all internal links valid Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
docs: add PPL language reference with data-grounded examples
anirudha
requested review from
goyamegh,
kylehounslow,
ps48 and
vamsimanohar
as code owners
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #142 +/- ##
=======================================
Coverage 18.51% 18.51%
=======================================
Files 3 3
Lines 54 54
Branches 18 19 +1
=======================================
Hits 10 10
Misses 44 44 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Collaborator
Author
|
Closing in favor of a new PR with squashed commit and DCO sign-off. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add comprehensive PPL (Piped Processing Language) documentation section to the Observability Stack docs, targeting Splunk SREs evaluating PPL as a query language for OpenSearch observability.
logs-otel-v1*andotel-v1-apm-span-*indices - no fabricated dataNew pages
ppl/index.mdppl/commands.mdppl/commands/*.mdppl/functions.mdppl/examples.mdPer-command pages
Search & Filter:
search,whereFields & Transformation:
fields,eval,rename,fillnull,expand,flattenAggregation & Statistics:
stats,eventstats,streamstats,timechart,trendlineSorting & Limiting:
sort,head,dedup,top,rareText Extraction:
parse,grok,rex,patterns,spathData Combination:
join,lookupMachine Learning:
mlMetadata:
describeEach command page follows a consistent structure:
Examples page highlights
Other changes
Data grounding
All text extraction examples (grok, rex, parse, spath) were tested against actual log bodies in the cluster:
frontend-proxy:[timestamp] "METHOD /path HTTP/1.1" status ...[ComponentName id=N] message ...User action product: IDKey PPL behavioral findings documented:
parserequires full-string match (implicitly anchored);rexdoes partial matching%{DATA}cause "Duplicate key" errorsTest plan
npm run buildpasses with all internal links validated (starlight-links-validator)!%27for single quotes)my-index,accounts, Apache CLF) remains in any example🤖 Generated with Claude Code