diff --git a/apl/entities/entity-names.mdx b/apl/entities/entity-names.mdx index 05430d28..675b1b3e 100644 --- a/apl/entities/entity-names.mdx +++ b/apl/entities/entity-names.mdx @@ -36,6 +36,6 @@ Quote an identifier in your APL query if any of the following is true: - Dash (`-`) - The identifier name is identical to one of the reserved keywords of the APL query language. For example, `project` or `where`. -If any of the above is true, you must quote the identifier by putting it in quotation marks (`'` or `"`) and square brackets (`[]`). For example, `['my-field']`. +If any of the above is true, you must quote the identifier by enclosing it in quotation marks (`'` or `"`) and square brackets (`[]`). For example, `['my-field']`. If none of the above is true, you don’t need to quote the identifier in your APL query. For example, `myfield`. In this case, quoting the identifier name is optional. \ No newline at end of file diff --git a/apl/introduction.mdx b/apl/introduction.mdx index 76afe845..7efc9e2f 100644 --- a/apl/introduction.mdx +++ b/apl/introduction.mdx @@ -6,45 +6,92 @@ icon: door-open tags: ['axiom documentation', 'documentation', 'axiom', 'APL', 'axiom processing language', 'data explorer', 'getiing started guide', 'summarize', 'filter'] --- -## Introduction +import Prerequisites from "/snippets/minimal-prerequisites.mdx" The Axiom Processing Language (APL) is a query language that is perfect for getting deeper insights from your data. Whether logs, events, analytics, or similar, APL provides the flexibility to filter, manipulate, and summarize your data exactly the way you need it. -## Get started + -Go to the Query tab and click one of your datasets to get started. The APL editor has full auto-completion so you can poke around or you can get a better understanding of all the features by using the reference menu to the left of this page. +## Build an APL query -## APL query structure +APL queries consist of the following: -At a minimum, a query consists of source data reference (name of a dataset) and zero or more query operators applied in sequence. Individual operators are delimited using the pipe character (`|`). +- **Data source:** The most common data source is one of your Axiom datasets. +- **Operators:** Operators filter, manipulate, and summarize your data. -APL query has the following structure: +Delimit operators with the pipe character (`|`). + +A typical APL query has the following structure: ```kusto -DataSource -| operator ... -| operator ... +DatasetName +| Operator ... +| Operator ... ``` -Where: +- `DatasetName` is the name of the dataset you want to query. +- `Operator` is an operation you apply to the data. -- DataSource is the name of the dataset you want to query -- Operator is a function that will be applied to the data + +Apart from Axiom datasets, you can use other data sources: +- External data sources using the [externaldata](/apl/tabular-operators/externaldata-operator) operator. +- Specify a data table in the APL query itself using the `let` statement. + -Let’s look at an example query. +## Example query ```kusto ['github-issue-comment-event'] -| extend bot = actor contains "-bot" or actor contains "[bot]" -| where bot == true +| extend isBot = actor contains '-bot' or actor contains '[bot]' +| where isBot == true | summarize count() by bin_auto(_time), actor ``` -The query above begins with reference to a dataset called **github-issue-comment-event** and contains several operators, [extend](/apl/tabular-operators/extend-operator), [where](/apl/tabular-operators/where-operator), and [summarize](/apl/tabular-operators/summarize-operator), each separated by a `pipe`. The extend operator creates the **bot** column in the returned result, and sets its values depending on the value of the actor column, the **where** operator filters out the value of the **bot** to a branch of rows and then produce a chart from the aggregation using the **summarize** operator. +[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'github-issue-comment-event'%5D%20%7C%20extend%20isBot%20%3D%20actor%20contains%20'-bot'%20or%20actor%20contains%20'%5Bbot%5D'%20%7C%20where%20isBot%20%3D%3D%20true%20%7C%20summarize%20count()%20by%20bin_auto(_time)%2C%20actor%22%7D) + +The query above uses a dataset called `github-issue-comment-event` as its data source. It uses the follwing operators: + +- [extend](/apl/tabular-operators/extend-operator) adds a new field `isBot` to the query results. It sets the values of the new field to true if the values of the `actor` field in the original dataset contain `-bot` or `[bot]`. +- [where](/apl/tabular-operators/where-operator) filters for the values of the `isBot` field. It only returns rows where the value is true. +- [summarize](/apl/tabular-operators/summarize-operator) aggregates the data and produces a chart. + +Each operator is separated using the pipe character (`|`). + +## Example result + +As a result, the query returns a chart and a table. The table counts the different values of the `actor` field where `isBot` is true, and the chart displays the distribution of these counts over time. + +| actor | count_ | +|---------------------|--------| +| github-actions[bot] | 487 | +| sonarqubecloud[bot] | 208 | +| dependabot[bot] | 148 | +| vercel[bot] | 91 | +| codecov[bot] | 63 | +| openshift-ci[bot] | 52 | +| coderabbitai[bot] | 43 | +| netlify[bot] | 37 | + + +The query results are a representation of your data based on your request. The query doesn’t change the original dataset. + + +## Quote dataset and field names + +If the name of a dataset or field contains at least one of the following special characters, quote the name in your APL query: + - Space (` `) + - Dot (`.`) + - Dash (`-`) + +To quote the dataset or field in your APL query, enclose its name with quotation marks (`'` or `"`) and square brackets (`[]`). For example, `['my-field']`. + +For more information on rules about naming and quoting entities, see [Entity names](/apl/entities/entity-names). -The most common kind of query statement is a tabular expression statement. Tabular statements contain operators, each of which starts with a tabular `input` and returns a tabular `output.` +## What's next -- Explore the [tabular operators](/apl/tabular-operators/extend-operator) we support. -- Check out our [entity names and identifier naming rules](/apl/entities/entity-names). +Check out the [list of sample queries](/apl/tutorial) or explore the supported operators and functions: -Axiom Processing Language supplies a set of system [data types](/apl/data-types/scalar-data-types) that define all the types of [data](/apl/data-types/null-values) that can be used with Axiom Processing Language. +- [Scalar functions](/apl/scalar-functions/) +- [Aggregation functions](/apl/aggregation-function/) +- [Tabular operators](/apl/tabular-operators/) +- [Scalar operators](/apl/scalar-operators/) \ No newline at end of file diff --git a/apl/tutorial.mdx b/apl/tutorial.mdx index 8617dd6b..064946a6 100644 --- a/apl/tutorial.mdx +++ b/apl/tutorial.mdx @@ -5,41 +5,24 @@ icon: vial tags: ['axiom documentation', 'documentation', 'axiom', 'APL', 'tutorial', 'apl overview', 'tabular operators', 'scalar functions', 'aggregation functions'] --- -In this tutorial, you’ll explore how to use APL in Axiom’s Query tab to run queries using Tabular Operators, Scalar Functions, and Aggregation Functions. +This page shows you how to query your data using APL through a wide range of sample queries. You can try out each example in the [Axiom Playground](https://play.axiom.co/axiom-play-qf1k/query). -## Prerequisites +For an introduction to APL and to the structure of an APL query, see [Introduction to APL](/apl/introduction). -- Sign up and log in to [Axiom Account](https://app.axiom.co/register) -- Ingest data into your dataset or you can run queries on [Play Sandbox](https://axiom.co/play) +## Summarize data -## Overview of APL +[summarize](/apl/tabular-operators/summarize-operator) produces a table that aggregates the content of the dataset. Use the [aggregation functions](/apl/aggregation-function/statistical-functions) with the `summarize` operator to produce different fields. -Every query, starts with a dataset embedded in **square brackets**, with the starting expression being a tabular operator statement. The query’s tabular expression statements produce the results of the query. - -Before you can start writing tabular operators or any function, the pipe (`|`) delimiter starts the query statements as they flow from one function to another. - - - - - -## Commonly used Operators - -To run queries on each function or operator in this tutorial, click the **Run in Playground** button. - -[summarize](/apl/tabular-operators/summarize-operator): Produces a table that aggregates the content of the dataset. - -The following query returns the count of events by **time** +The following query counts events by time bins. ```kusto -['github-push-event'] +['sample-http-logs'] | summarize count() by bin_auto(_time) ``` -[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27github-push-event%27%5D%5Cn%7C%20summarize%20count%28%29%20by%20bin_auto%28_time%29%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) - -You can use the [aggregation functions](/apl/aggregation-function/statistical-functions) with the **summarize operator** to produce different columns. +[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20summarize%20count%28%29%20by%20bin_auto%28_time%29%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Top 10 GitHub push events by maximum push id +The example below summarizes the top 10 GitHub push events by maximum push ID. ```kusto ['github-push-event'] @@ -49,7 +32,7 @@ You can use the [aggregation functions](/apl/aggregation-function/statistical-fu [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27github-push-event%27%5D%5Cn%7C%20summarize%20max_if%20%3D%20maxif%28push_id%2C%20true%29%20by%20size%5Cn%7C%20top%2010%20by%20max_if%20desc%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Distinct City count by server datacenter +The example below summarizes the distinct city count by server datacenter. ```kusto ['sample-http-logs'] @@ -58,17 +41,13 @@ You can use the [aggregation functions](/apl/aggregation-function/statistical-fu [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20summarize%20cities%20%3D%20dcount%28%5B%27geo.city%27%5D%29%20by%20server_datacenter%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -The result of a summarize operation has: +## Tabular operators -- A row for every combination of by values +### where -- Each column named in by +[where](/apl/tabular-operators/where-operator) filters the content of the dataset that meets a condition when executed. -- A column for each expression - -[where](/apl/tabular-operators/where-operator): Filters the content of the dataset that meets a **condition** when executed. - -The following query filters the data by **method** and **content_type**: +The following query filters the data by `method` and `content_type`: ```kusto ['sample-http-logs'] @@ -78,7 +57,9 @@ The following query filters the data by **method** and **content_type**: [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20where%20method%20%3D%3D%20%5C%22GET%5C%22%20and%20content_type%20%3D%3D%20%5C%22application%2Foctet-stream%5C%22%5Cn%7C%20project%20method%20%2C%20content_type%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -[count](/apl/tabular-operators/count-operator): Returns the number of events from the input dataset. +### count + +[count](/apl/tabular-operators/count-operator) returns the number of events from the input dataset. ```kusto ['sample-http-logs'] @@ -87,17 +68,9 @@ The following query filters the data by **method** and **content_type**: [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20count%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) +### project -[Summarize](/apl/tabular-operators/summarize-operator) count by time bins in sample HTTP logs - -```kusto -['sample-http-logs'] -| summarize count() by bin_auto(_time) -``` - -[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20summarize%20count%28%29%20by%20bin_auto%28_time%29%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) - -[project](/apl/tabular-operators/project-operator): Selects a subset of columns. +[project](/apl/tabular-operators/project-operator) selects a subset of columns. ```kusto ['sample-http-logs'] @@ -106,7 +79,9 @@ The following query filters the data by **method** and **content_type**: [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20project%20content_type%2C%20%5B%27geo.country%27%5D%2C%20method%2C%20resp_body_size_bytes%2C%20resp_header_size_bytes%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -[take](/apl/tabular-operators/take-operator): Returns up to the specified number of rows. +### take + +[take](/apl/tabular-operators/take-operator) returns up to the specified number of rows. ```kusto ['sample-http-logs'] @@ -115,7 +90,9 @@ The following query filters the data by **method** and **content_type**: [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20take%20100%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -The **limit** operator is an alias to the **take** operator. +### limit + +The `limit` operator is an alias to the `take` operator. ```kusto ['sample-http-logs'] @@ -124,11 +101,11 @@ The **limit** operator is an alias to the **take** operator. [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20limit%2010%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Scalar Functions +## Scalar functions -#### [parse_json()](/apl/scalar-functions/string-functions#parse-json) +### parse_json -The following query extracts the JSON elements from an array: +[parse_json](/apl/scalar-functions/string-functions#parse-json) extracts the JSON elements from an array. ```kusto ['sample-http-logs'] @@ -137,7 +114,9 @@ The following query extracts the JSON elements from an array: [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20project%20parsed_json%20%3D%20parse_json%28%20%5C%22config_jsonified_metrics%5C%22%29%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -#### [replace_string()](/apl/scalar-functions/string-functions#parse-json): Replaces all string matches with another string. +### replace_string + +[replace_string](/apl/scalar-functions/string-functions#parse-json) replaces all string matches with another string. ```kusto ['sample-http-logs'] @@ -147,7 +126,9 @@ The following query extracts the JSON elements from an array: [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20extend%20replaced_string%20%3D%20replace_string%28%20%5C%22creator%5C%22%2C%20%5C%22method%5C%22%2C%20%5C%22machala%5C%22%20%29%5Cn%7C%20project%20replaced_string%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -#### [split()](/apl/scalar-functions/string-functions#split): Splits a given string according to a given delimiter and returns a string array. +### split + +[split](/apl/scalar-functions/string-functions#split) splits a given string according to a given delimiter and returns a string array. ```kusto ['sample-http-logs'] @@ -157,7 +138,9 @@ The following query extracts the JSON elements from an array: [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20project%20split_str%20%3D%20split%28%5C%22method_content_metrics%5C%22%2C%20%5C%22_%5C%22%29%5Cn%7C%20take%2020%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -#### [strcat_delim()](/apl/scalar-functions/string-functions#strcat-delim): Concatenates a string array into a string with a given delimiter. +### strcat_delim + +[strcat_delim](/apl/scalar-functions/string-functions#strcat-delim) concatenates a string array into a string with a given delimiter. ```kusto ['sample-http-logs'] @@ -166,7 +149,9 @@ The following query extracts the JSON elements from an array: [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20project%20strcat%20%3D%20strcat_delim%28%5C%22%3A%5C%22%2C%20%5B%27geo.city%27%5D%2C%20resp_body_size_bytes%29%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -#### [indexof()](/apl/scalar-functions/string-functions#indexof): Reports the zero-based index of the first occurrence of a specified string within the input string. +### indexof + +[indexof](/apl/scalar-functions/string-functions#indexof) reports the zero-based index of the first occurrence of a specified string within the input string. ```kusto ['sample-http-logs'] @@ -175,7 +160,9 @@ The following query extracts the JSON elements from an array: [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20extend%20based_index%20%3D%20%20indexof%28%20%5B%27geo.country%27%5D%2C%20content_type%2C%2045%2C%2060%2C%20resp_body_size_bytes%20%29%2C%20specified_time%20%3D%20bin%28resp_header_size_bytes%2C%2030%29%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Regex Examples +## Regex examples + +**Remove leading characters** ```kusto ['sample-http-logs'] @@ -184,7 +171,7 @@ The following query extracts the JSON elements from an array: [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20project%20remove_cutset%20%3D%20trim_start_regex%28%5C%22%5B%5Ea-zA-Z%5D%5C%22%2C%20content_type%20%29%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Finding logs from a specific City +**Find logs from a city** ```kusto ['sample-http-logs'] @@ -193,7 +180,7 @@ The following query extracts the JSON elements from an array: [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20where%20tostring%28%5B%27geo.city%27%5D%29%20matches%20regex%20%5C%22%5ECamaqu%C3%A3%24%5C%22%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Identifying logs from a specific user agent +**Identify logs from a user agent** ```kusto ['sample-http-logs'] @@ -202,7 +189,7 @@ The following query extracts the JSON elements from an array: [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20where%20tostring%28user_agent%29%20matches%20regex%20%5C%22Mozilla%2F5.0%5C%22%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Finding logs with response body size in a certain range +**Find logs with response body size in a certain range** ```kusto ['sample-http-logs'] @@ -211,7 +198,7 @@ The following query extracts the JSON elements from an array: [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20where%20toint%28resp_body_size_bytes%29%20%3E%3D%204000%20and%20toint%28resp_body_size_bytes%29%20%3C%3D%205000%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Finding logs with user agents containing Windows NT +**Find logs with user agents containing Windows NT** ```kusto ['sample-http-logs'] @@ -220,7 +207,7 @@ The following query extracts the JSON elements from an array: [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?qid=m8yNkSVVjGq-s0z19c) -## Finding logs with specific response header size +**Find logs with specific response header size** ```kusto ['sample-http-logs'] @@ -229,7 +216,7 @@ The following query extracts the JSON elements from an array: [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20where%20toint%28resp_header_size_bytes%29%20%3D%3D%2031%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Finding logs with specific request duration +**Find logs with specific request duration** ```kusto ['sample-http-logs'] @@ -238,7 +225,7 @@ The following query extracts the JSON elements from an array: [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20where%20toreal%28req_duration_ms%29%20%3C%201%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Finding logs where TLS is enabled and method is POST +**Find logs where TLS is enabled and method is POST** ```kusto ['sample-http-logs'] @@ -249,7 +236,9 @@ The following query extracts the JSON elements from an array: ## Array functions -#### [array_concat()](/apl/scalar-functions/array-functions#array_concat): Concatenates a number of dynamic arrays to a single array. +### array_concat + +[array_concat](/apl/scalar-functions/array-functions#array_concat) concatenates a number of dynamic arrays to a single array. ```kusto ['sample-http-logs'] @@ -259,7 +248,9 @@ The following query extracts the JSON elements from an array: [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20extend%20concatenate%20%3D%20array_concat%28%20dynamic%28%5B5%2C4%2C3%2C87%2C45%2C2%2C3%2C45%5D%29%29%5Cn%7C%20project%20concatenate%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -#### [array_sum()](/apl/scalar-functions/array-functions#array-sum): Calculates the sum of elements in a dynamic array. +### array_sum + +[array_sum](/apl/scalar-functions/array-functions#array-sum) calculates the sum of elements in a dynamic array. ```kusto ['sample-http-logs'] @@ -271,7 +262,9 @@ The following query extracts the JSON elements from an array: ## Conversion functions -#### [todatetime()](/apl/scalar-functions/conversion-functions#todatetime): Converts input to datetime scalar. +### todatetime + +[todatetime](/apl/scalar-functions/conversion-functions#todatetime) converts input to datetime scalar. ```kusto ['sample-http-logs'] @@ -280,7 +273,9 @@ The following query extracts the JSON elements from an array: [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20extend%20dated_time%20%3D%20todatetime%28%5C%222026-08-16%5C%22%29%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -#### [dynamic_to_json()](/apl/scalar-functions/conversion-functions#dynamic-to-json): Converts a scalar value of type dynamic to a canonical string representation. +### dynamic_to_json + +[dynamic_to_json](/apl/scalar-functions/conversion-functions#dynamic-to-json) converts a scalar value of type dynamic to a canonical string representation. ```kusto ['sample-http-logs'] @@ -289,11 +284,17 @@ The following query extracts the JSON elements from an array: [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20extend%20dynamic_string%20%3D%20dynamic_to_json%28dynamic%28%5B10%2C20%2C30%2C40%20%5D%29%29%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## String Operators +## Scalar operators + +APL supports a wide range of scalar operators: + +- [String operators](/apl/scalar-operators/string-operators) +- [Logical operators](/apl/scalar-operators/logical-operators) +- [Numerical operators](/apl/scalar-operators/numerical-operators) -[We support various query string](/apl/scalar-operators/string-operators), [logical](/apl/scalar-operators/logical-operators) and [numerical operators](/apl/scalar-operators/numerical-operators). +### contains -In the query below, we use the **contains** operator, to find the strings that contain the string **-bot** and **[bot]**: +The query below uses the `contains` operator to find the strings that contain the string `-bot` and `[bot]`: ```kusto ['github-issue-comment-event'] @@ -315,13 +316,11 @@ In the query below, we use the **contains** operator, to find the strings that c [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20extend%20user_status%20%3D%20status%20contains%20%5C%22200%5C%22%20%2C%20agent_flow%20%3D%20user_agent%20contains%20%5C%22%28Windows%20NT%206.4%3B%20AppleWebKit%2F537.36%20Chrome%2F41.0.2225.0%20Safari%2F537.36%5C%22%5Cn%7C%20where%20user_status%20%3D%3D%20true%5Cn%7C%20summarize%20count%28%29%20by%20bin_auto%28_time%29%2C%20status%5Cn%7C%20take%2015%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Hash Functions +## Hash functions -- [hash_md5()](/apl/scalar-functions/hash-functions#hash-md5): Returns an MD5 hash value for the input value. - -- [hash_sha256()](/apl/scalar-functions/hash-functions#hash-sha256): Returns a sha256 hash value for the input value. - -- [hash_sha1()](/apl/scalar-functions/hash-functions#hash-sha1): Returns a sha1 hash value for the input value. +- [hash_md5](/apl/scalar-functions/hash-functions#hash-md5) returns an MD5 hash value for the input value. +- [hash_sha256](/apl/scalar-functions/hash-functions#hash-sha256) returns a sha256 hash value for the input value. +- [hash_sha1](/apl/scalar-functions/hash-functions#hash-sha1) returns a sha1 hash value for the input value. ```kusto ['sample-http-logs'] @@ -331,69 +330,69 @@ In the query below, we use the **contains** operator, to find the strings that c [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20extend%20sha_256%20%3D%20hash_md5%28%20%5C%22resp_header_size_bytes%5C%22%20%29%2C%20sha_1%20%3D%20hash_sha1%28%20content_type%29%2C%20md5%20%3D%20hash_md5%28%20method%29%2C%20sha512%20%3D%20hash_sha512%28%20%5C%22resp_header_size_bytes%5C%22%20%29%5Cn%7C%20project%20sha_256%2C%20sha_1%2C%20md5%2C%20sha512%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## List all unique groups +## Rounding functions + +- [floor()](/apl/scalar-functions/rounding-functions#floor) calculates the largest integer less than, or equal to, the specified numeric expression. +- [ceiling()](/apl/scalar-functions/rounding-functions#ceiling) calculates the smallest integer greater than, or equal to, the specified numeric expression. +- [bin()](/apl/scalar-functions/rounding-functions#bin) rounds values down to an integer multiple of a given bin size. ```kusto ['sample-http-logs'] -| distinct ['id'], is_tls +| extend largest_integer_less = floor( resp_header_size_bytes ), smallest_integer_greater = ceiling( req_duration_ms ), integer_multiple = bin( resp_body_size_bytes, 5 ) +| project largest_integer_less, smallest_integer_greater, integer_multiple ``` -[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'sample-http-logs'%5D%5Cn%7C%20distinct%20%5B'id'%5D%2C%20is_tls%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) +[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20extend%20largest_integer_less%20%3D%20floor%28%20resp_header_size_bytes%20%29%2C%20smallest_integer_greater%20%3D%20ceiling%28%20req_duration_ms%20%29%2C%20integer_multiple%20%3D%20bin%28%20resp_body_size_bytes%2C%205%20%29%5Cn%7C%20project%20largest_integer_less%2C%20smallest_integer_greater%2C%20integer_multiple%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Count of all events per service +**Truncate decimals using round function** ```kusto ['sample-http-logs'] -| summarize Count = count() by server_datacenter -| order by Count desc +| project rounded_value = round(req_duration_ms, 2) ``` -[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'sample-http-logs'%5D%5Cn%7C%20summarize%20Count%20%3D%20count%28%29%20by%20server_datacenter%5Cn%7C%20order%20by%20Count%20desc%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) +[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'sample-http-logs'%5D%5Cn%7C%20project%20rounded_value%20%3D%20round%28req_duration_ms%2C%202%29%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Change the time clause +**Truncate decimals using floor function** ```kusto -['github-issues-event'] -| where _time == ago(1m) -| summarize count(), sum(['milestone.number']) by _time=bin(_time, 1m) +['sample-http-logs'] +| project floor_value = floor(resp_body_size_bytes), ceiling_value = ceiling(req_duration_ms) ``` -[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27github-issues-event%27%5D%5Cn%7C%20where%20_time%20%3D%3D%20ago%281m%29%5Cn%7C%20summarize%20count%28%29%2C%20sum%28%5B%27milestone.number%27%5D%29%20by%20_time%3Dbin%28_time%2C%201m%29%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) - -## Rounding functions -- [floor()](/apl/scalar-functions/rounding-functions#floor): Calculates the largest integer less than, or equal to, the specified numeric expression. +[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'sample-http-logs'%5D%5Cn%7C%20project%20floor_value%20%3D%20floor%28resp_body_size_bytes%29%2C%20ceiling_value%20%3D%20ceiling%28req_duration_ms%29%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -- [ceiling()](/apl/scalar-functions/rounding-functions#ceiling): Calculates the smallest integer greater than, or equal to, the specified numeric expression. +## Other examples -- [bin()](/apl/scalar-functions/rounding-functions#bin): Rounds values down to an integer multiple of a given bin size. +**List all unique groups** ```kusto ['sample-http-logs'] -| extend largest_integer_less = floor( resp_header_size_bytes ), smallest_integer_greater = ceiling( req_duration_ms ), integer_multiple = bin( resp_body_size_bytes, 5 ) -| project largest_integer_less, smallest_integer_greater, integer_multiple +| distinct ['id'], is_tls ``` -[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20extend%20largest_integer_less%20%3D%20floor%28%20resp_header_size_bytes%20%29%2C%20smallest_integer_greater%20%3D%20ceiling%28%20req_duration_ms%20%29%2C%20integer_multiple%20%3D%20bin%28%20resp_body_size_bytes%2C%205%20%29%5Cn%7C%20project%20largest_integer_less%2C%20smallest_integer_greater%2C%20integer_multiple%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) +[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'sample-http-logs'%5D%5Cn%7C%20distinct%20%5B'id'%5D%2C%20is_tls%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Truncate decimals using round function +**Count of all events per service** ```kusto ['sample-http-logs'] -| project rounded_value = round(req_duration_ms, 2) +| summarize Count = count() by server_datacenter +| order by Count desc ``` -[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'sample-http-logs'%5D%5Cn%7C%20project%20rounded_value%20%3D%20round%28req_duration_ms%2C%202%29%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) +[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'sample-http-logs'%5D%5Cn%7C%20summarize%20Count%20%3D%20count%28%29%20by%20server_datacenter%5Cn%7C%20order%20by%20Count%20desc%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Truncate decimals using floor function +**Change the time clause** ```kusto -['sample-http-logs'] -| project floor_value = floor(resp_body_size_bytes), ceiling_value = ceiling(req_duration_ms) +['github-issues-event'] +| where _time == ago(1m) +| summarize count(), sum(['milestone.number']) by _time=bin(_time, 1m) ``` +[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27github-issues-event%27%5D%5Cn%7C%20where%20_time%20%3D%3D%20ago%281m%29%5Cn%7C%20summarize%20count%28%29%2C%20sum%28%5B%27milestone.number%27%5D%29%20by%20_time%3Dbin%28_time%2C%201m%29%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'sample-http-logs'%5D%5Cn%7C%20project%20floor_value%20%3D%20floor%28resp_body_size_bytes%29%2C%20ceiling_value%20%3D%20ceiling%28req_duration_ms%29%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) - -## HTTP 5xx responses (day wise) for the last 7 days - one bar per day +**HTTP 5xx responses for the last 7 days, one bar per day** ```kusto ['sample-http-logs'] @@ -404,7 +403,7 @@ In the query below, we use the **contains** operator, to find the strings that c ``` [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'sample-http-logs'%5D%20%7C%20where%20_time%20%3E%20ago(7d)%20%7C%20where%20req_duration_ms%20%3E%3D%205%20and%20req_duration_ms%20%3C%206%20%7C%20summarize%20count()%2C%20histogram(resp_header_size_bytes%2C%2020)%20by%20bin(_time%2C%201d)%20%7C%20order%20by%20_time%20desc%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%227d%22%7D%7D) -## Implement a remapper on remote address logs +**Implement a remapper on remote address logs** ```kusto ['sample-http-logs'] @@ -413,9 +412,7 @@ In the query below, we use the **contains** operator, to find the strings that c [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'sample-http-logs'%5D%5Cn%7C%20extend%20RemappedStatus%20%3D%20case%28req_duration_ms%20%3E%3D%200.57%2C%20%5C%22new%20data%5C%22%2C%20resp_body_size_bytes%20%3E%3D%201000%2C%20%5C%22size%20bytes%5C%22%2C%20resp_header_size_bytes%20%3D%3D%2040%2C%20%5C%22header%20values%5C%22%2C%20%5C%22doesntmatch%5C%22%29%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Advanced aggregations - -In this section, you will learn how to run queries using different functions and operators. +**Advanced aggregations** ```kusto ['sample-http-logs'] @@ -427,7 +424,7 @@ In this section, you will learn how to run queries using different functions and [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20extend%20prospect%20%3D%20%5B%27geo.city%27%5D%20contains%20%5C%22Okayama%5C%22%20or%20uri%20contains%20%5C%22%2Fapi%2Fv1%2Fmessages%2Fback%5C%22%5Cn%7C%20extend%20possibility%20%3D%20server_datacenter%20contains%20%5C%22GRU%5C%22%20or%20status%20contains%20%5C%22301%5C%22%5Cn%7C%20summarize%20count%28%29%2C%20topk%28%20user_agent%2C%206%20%29%20by%20bin%28_time%2C%2010d%29%2C%20%5B%27geo.country%27%5D%5Cn%7C%20take%204%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Searching map fields +**Search map fields** ```kusto ['otel-demo-traces'] @@ -439,7 +436,7 @@ In this section, you will learn how to run queries using different functions and [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'otel-demo-traces'%5D%5Cn%7C%20where%20isnotnull%28%20%5B'attributes.custom'%5D%29%5Cn%7C%20extend%20extra%20%3D%20tostring%28%5B'attributes.custom'%5D%29%5Cn%7C%20search%20extra%3A%5C%220PUK6V6EV0%5C%22%5Cn%7C%20project%20_time%2C%20trace_id%2C%20name%2C%20%5B'attributes.custom'%5D%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Configure Processing rules +**Configure processing rules** ```kusto ['sample-http-logs'] @@ -449,7 +446,7 @@ In this section, you will learn how to run queries using different functions and [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20where%20_sysTime%20%3E%20ago%281d%29%5Cn%7C%20summarize%20count%28%29%20by%20method%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%221d%22%7D%7D) -## Return different values based on the evaluation of a condition +**Return different values based on the evaluation of a condition** ```kusto ['sample-http-logs'] @@ -458,7 +455,7 @@ In this section, you will learn how to run queries using different functions and [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20extend%20MemoryUsageStatus%20%3D%20iff%28req_duration_ms%20%3E%2010000%2C%20%27Highest%27%2C%20%27Normal%27%29%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Working with different operators +**Working with different operators** ```kusto ['hn'] @@ -484,7 +481,7 @@ In this section, you will learn how to run queries using different functions and [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%5Cn%7C%20summarize%20flow%20%3D%20dcount%28%20content_type%29%20by%20%5B%27geo.country%27%5D%5Cn%7C%20take%2050%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Get the JSON into a property bag using parse-json +**Get the JSON into a property bag using parse-json** ```kusto example @@ -493,7 +490,7 @@ example | project service, parsed_log.level, parsed_log.message ``` -## Get average response using project keep function +**Get average response using project-keep** ```kusto ['sample-http-logs'] @@ -507,7 +504,7 @@ example [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'sample-http-logs'%5D%5Cn%7C%20where%20%5B'geo.country'%5D%20%3D%3D%20%5C%22United%20States%5C%22%20or%20%5B'id'%5D%20%3D%3D%20%5C%22b2b1f597-0385-4fed-a911-140facb757ef%5C%22%5Cn%7C%20extend%20systematic_view%20%3D%20ceiling%28%20resp_header_size_bytes%20%29%5Cn%7C%20extend%20resp_avg%20%3D%20cos%28%20resp_body_size_bytes%20%29%5Cn%7C%20project-away%20systematic_view%5Cn%7C%20project-keep%20resp_avg%5Cn%7C%20take%205%22%7D) -## Combine multiple percentiles into a single chart in APL +**Combine multiple percentiles into a single chart** ```kusto ['sample-http-logs'] @@ -516,7 +513,7 @@ example [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%20%7C%20summarize%20percentiles_array(req_duration_ms%2C%2050%2C%2075%2C%2090)%20by%20bin_auto(_time)%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Combine mathematical functions +**Combine mathematical functions** ```kusto ['sample-http-logs'] @@ -537,7 +534,7 @@ example [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27github-issues-event%27%5D%5Cn%7C%20where%20actor%20%21endswith%20%5C%22%5Bbot%5D%5C%22%5Cn%7C%20where%20repo%20startswith%20%5C%22kubernetes%2F%5C%22%5Cn%7C%20where%20action%20%3D%3D%20%5C%22opened%5C%22%5Cn%7C%20summarize%20count%28%29%20by%20bin_auto%28_time%29%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Change global configuration attributes +**Change global configuration attributes** ```kusto ['sample-http-logs'] @@ -546,7 +543,7 @@ example [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%20%7C%20extend%20status%20%3D%20coalesce(status%2C%20%5C%22info%5C%22)%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Set defualt value on event field +**Set defualt value on event field** ```kusto ['sample-http-logs'] @@ -559,7 +556,7 @@ example [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27sample-http-logs%27%5D%20%7C%20project%20status%20%3D%20case(isnotnull(status)%20and%20status%20!%3D%20%5C%22%5C%22%2C%20content_type%2C%20%5C%22info%5C%22)%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2230d%22%7D%7D) -## Extract nested payment amount from custom attributes map field +**Extract nested payment amount from custom attributes map field** ```kusto ['otel-demo-traces'] @@ -570,7 +567,7 @@ example [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'otel-demo-traces'%5D%20%7C%20extend%20amount%20%3D%20%5B'attributes.custom'%5D%5B'app.payment.amount'%5D%20%7C%20where%20isnotnull(%20amount)%20%7C%20project%20_time%2C%20trace_id%2C%20name%2C%20amount%2C%20%5B'attributes.custom'%5D%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2290d%22%7D%7D) -## Filtering GitHub issues by label identifier +**Filtering GitHub issues by label identifier** ```kusto ['github-issues-event'] @@ -581,7 +578,7 @@ example [Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'github-issues-event'%5D%20%7C%20extend%20data%20%3D%20tostring(labels)%20%7C%20where%20labels%20contains%20'd73a4a'%22%2C%22queryOptions%22%3A%7B%22quickRange%22%3A%2290d%22%7D%7D) -## Aggregate trace counts by HTTP method attribute in custom map +**Aggregate trace counts by HTTP method attribute in custom map** ```kusto ['otel-demo-traces'] diff --git a/doc-assets/shots/overview-of-apl-introduction.png b/doc-assets/shots/overview-of-apl-introduction.png deleted file mode 100644 index 8d4bad71..00000000 Binary files a/doc-assets/shots/overview-of-apl-introduction.png and /dev/null differ diff --git a/restapi/ingest.mdx b/restapi/ingest.mdx index 5b3eacff..eaccba70 100644 --- a/restapi/ingest.mdx +++ b/restapi/ingest.mdx @@ -1,65 +1,47 @@ --- -title: "Send data via Axiom API" -description: "Learn how to send and load data into Axiom using the API." +title: "Send data to Axiom via API" +description: "This page explains how to send to Axiom using the API." sidebarTitle: Send data tags: ['axiom documentation', 'documentation', 'axiom', 'axiom api', 'rest api', 'rest', 'authorization', 'headers', 'send data', 'ingesting', 'json', 'arrays', 'nested arrays', 'objects', 'strings', 'csv', 'response'] --- -This API allows you to send and load data into Axiom. You can use different methods to ingest logs depending on your requirements and log format. +import Prerequisites from "/snippets/standard-prerequisites.mdx" +import ReplaceDatasetToken from "/snippets/replace-dataset-token.mdx" -## Authorization and headers +The Axiom REST API accepts the following data formats: -The only expected header is `Authorization: Bearer` which is your to token to authenticate the request. For more information, see [Tokens](/reference/tokens). +- [JSON](#send-data-in-json-format) +- [NDJSON](#send-data-in-ndjson-format) +- [CSV](#send-data-in-csv-format) -## Using Axiom JS library to ingest data +This page explains how to send data to Axiom via cURL commands in each of these formats, and how to send data with the [Axiom Node.js library](#send-data-with-axiom-node-js). -Axiom maintains the [axiom-js](https://github.com/axiomhq/axiom-js) to provide official Javascript bindings for the Axiom API. +For more information on other ingest options, see [Send data](send-data/ingest). -Install using `npm install`: +For an introduction to the basics of the Axiom API and to the authentication options, see [Introduction to Axiom API](/restapi/introduction). -```shell -npm install @axiomhq/js -``` - -If you use the [Axiom CLI](https://github.com/axiomhq/cli), run `eval $(axiom config export -f)` to configure your environment variables. - -Otherwise, create an [API token](/reference/tokens) and export it as `AXIOM_TOKEN`. - -You can also configure the client using options passed to the constructor of the Client: - -```ts -const client = new Client({ - token: process.env.AXIOM_TOKEN -}); -``` - -Create and use a client like this: - -```ts -import { Axiom } from '@axiomhq/js'; +The API requests on this page use the ingest data endpoint. For more information, see the [API reference](/restapi/endpoints/ingestIntoDataset). + -async function main() { - const axiom = new Axiom({ - token: process.env.AXIOM_TOKEN - }); +## Send data in JSON format - await axiom.ingest('my-dataset', [{ foo: 'bar' }]); +To send data to Axiom in JSON format: +1. Encode the events as JSON objects. +1. Enter the array of JSON objects into the body of the API request. +1. Optional: In the body of the request, set optional parameters such as `timestamp-field` and `timestamp-format`. For more information, see the [ingest data API reference](/restapi/endpoints/ingestIntoDataset). +1. Set the `Content-Type` header to `application/json`. +1. Set the `Authorization` header to `Bearer API_TOKEN`. Replace `API_TOKEN` with the Axiom API token you have generated. +1. Send the POST request to `https://api.axiom.co/v1/datasets/DATASET_NAME/ingest`. Replace `DATASET_NAME` with the name of the Axiom dataset where you want to send data. - const res = await axiom.query(`['my-dataset'] | where foo == 'bar' | limit 100`); -} -``` +### Example with grouped events -These examples send an API event to Axiom. Before getting started with Axiom API, you need to create a [Dataset](/reference/datasets) and [API Token](/reference/tokens). +The following example request contains grouped events. The structure of the JSON payload has the scheme of `[ { "labels": { "key1": "value1", "key2": "value2" } }, ]` where the array contains one or more JSON objects describing events. -## Ingest Events using JSON - -The following example request contains grouped events. The structure of the `JSON` payload should have the scheme of `[ { "labels": { "key1": "value1", "key2": "value12" } }, ]`, in which the array comprises of one or more JSON objects describing Events. - -### Example Request using JSON +**Example request** ```bash -curl -X 'POST' 'https://api.axiom.co/v1/datasets/{dataset_name}/ingest' \ +curl -X 'POST' 'https://api.axiom.co/v1/datasets/DATASET_NAME/ingest' \ -H 'Authorization: Bearer API_TOKEN' \ -H 'Content-Type: application/json' \ -d '[ @@ -74,9 +56,9 @@ curl -X 'POST' 'https://api.axiom.co/v1/datasets/{dataset_name}/ingest' \ ]' ``` -### Example Response + -A successful POST Request returns a `200` response code JSON with details: +**Example response** ```json { @@ -85,144 +67,178 @@ A successful POST Request returns a `200` response code JSON with details: "failures": [], "processedBytes": 219, "blocksCreated": 0, - "walLength": 8 + "walLength": 2 } ``` -### Example Request using Nested Arrays +### Example with nested arrays + +**Example request** ```bash -curl -X 'POST' 'https://api.axiom.co/v1/datasets/{dataset_name}/ingest' \ - -H 'Authorization: Bearer API_TOKEN' \ - -H 'Content-Type: application/json' \ - -d '[ - { - "axiom": [{ - "logging":[{ - "observability":[{ - "location":[{ - "credentials":[{ - "datasets":[{ - "first_name":"axiom", - "last_name":"logging", - "location":"global" - }], - "work":[{ - "details":"https://app.axiom.co/", - "tutorials":"https://www.axiom.co/blog", - "changelog":"https://www.axiom.co/changelog", - "documentation": "https://www.axiom.co/docs" - }] - }], - "social_media":[{ - "details":[{ - "twitter":"https://twitter.com/AxiomFM", - "linkedin":"https://linkedin.com/company/axiomhq", - "github":"https://github.com/axiomhq" +curl -X 'POST' 'https://api.axiom.co/v1/datasets/DATASET_NAME/ingest' \ + -H 'Authorization: Bearer API_TOKEN' \ + -H 'Content-Type: application/json' \ + -d '[ + { + "axiom": [{ + "logging":[{ + "observability":[{ + "location":[{ + "credentials":[{ + "datasets":[{ + "first_name":"axiom", + "last_name":"logging", + "location":"global" + }], + "work":[{ + "details":"https://app.axiom.co/", + "tutorials":"https://www.axiom.co/blog", + "changelog":"https://www.axiom.co/changelog", + "documentation": "https://www.axiom.co/docs" + }] }], - "features":[{ - "datasets":"view logs", - "stream":"live_tail", - "explorer":"queries" + "social_media":[{ + "details":[{ + "twitter":"https://twitter.com/AxiomFM", + "linkedin":"https://linkedin.com/company/axiomhq", + "github":"https://github.com/axiomhq" + }], + "features":[{ + "datasets":"view logs", + "stream":"live_tail", + "explorer":"queries" + }] }] }] + }], + "logs":[{ + "apl": "functions" }] }], - "logs":[{ - "apl": "functions" - }] - }], - "storage":[{}] - }]} - ]' + "storage":[{}] + }]} + ]' ``` -### Example Response + -A successful POST Request returns a `200` response code JSON with details: +**Example response** ```json { - "ingested":1, - "failed":0, - "failures":[], - "processedBytes":1509, - "blocksCreated":0, - "walLength":6 + "ingested":1, + "failed":0, + "failures":[], + "processedBytes":1587, + "blocksCreated":0, + "walLength":3 } ``` -### Example Request using Objects, Strings, and Arrays +### Example with objects, strings, and arrays + +**Example request** ```bash -curl -X 'POST' 'https://api.axiom.co/v1/datasets/{dataset_name}/ingest' \ - -H 'Authorization: Bearer API_TOKEN' \ - -H 'Content-Type: application/json' \ - -d '[{ "axiom": { - "logging": { - "observability": [ - { "apl": 23, "function": "tostring" }, - { "apl": 24, "operator": "summarize" } - ], - "axiom": [ - { "stream": "livetail", "datasets": [4, 0, 16], "logging": "observability", "metrics": 8, "dashboard": 10, "alerting": "kubernetes" } - ] - }, - "apl": { - "reference": - [[80, 12], [30, 40]] +curl -X 'POST' 'https://api.axiom.co/v1/datasets/DATASET_NAME/ingest' \ + -H 'Authorization: Bearer API_TOKEN' \ + -H 'Content-Type: application/json' \ + -d '[{ "axiom": { + "logging": { + "observability": [ + { "apl": 23, "function": "tostring" }, + { "apl": 24, "operator": "summarize" } + ], + "axiom": [ + { "stream": "livetail", "datasets": [4, 0, 16], "logging": "observability", "metrics": 8, "dashboard": 10, "alerting": "kubernetes" } + ] + }, + "apl": { + "reference": + [[80, 12], [30, 40]] + } } - } -}]' + }]' ``` -### Example Response + -A successful POST Request returns a `200` response code JSON with details: +**Example response** ```json { - "ingested":1, - "failed":0, - "failures":[], - "processedBytes":432, - "blocksCreated":0, - "walLength":7 + "ingested":1, + "failed":0, + "failures":[], + "processedBytes":432, + "blocksCreated":0, + "walLength":4 } ``` -### Example Response +## Send data in NDJSON format + +To send data to Axiom in NDJSON format: +1. Encode the events as JSON objects. +1. Enter each JSON object in a separate line into the body of the API request. +1. Optional: In the body of the request, set optional parameters such as `timestamp-field` and `timestamp-format`. For more information, see the [ingest data API reference](/restapi/endpoints/ingestIntoDataset). +1. Set the `Content-Type` header to `application/x-ndjson`. +1. Set the `Authorization` header to `Bearer API_TOKEN`. Replace `API_TOKEN` with the Axiom API token you have generated. +1. Send the POST request to `https://api.axiom.co/v1/datasets/DATASET_NAME/ingest`. Replace `DATASET_NAME` with the name of the Axiom dataset where you want to send data. -A successful POST Request returns a `200` response code JSON with details: +**Example request** + +```bash +curl -X 'POST' 'https://api.axiom.co/v1/datasets/DATASET_NAME/ingest' \ + -H 'Authorization: Bearer API_TOKEN' \ + -H 'Content-Type: application/x-ndjson' \ + -d '{"id":1,"name":"machala"} + {"id":2,"name":"axiom"} + {"id":3,"name":"apl"} + {"index": {"_index": "products"}} + {"timestamp": "2016-06-06T12:00:00+02:00", "attributes": {"key1": "value1","key2": "value2"}} + {"queryString": "count()"}' +``` + + + +**Example response** ```json { "ingested": 6, "failed": 0, "failures": [], - "processedBytes": 236, + "processedBytes": 266, "blocksCreated": 0, "walLength": 6 } ``` -## Ingest Events using CSV +## Send data in CSV format -The following example request contains events. The structure of the `CSV` payload uses a comma to separate values `'value1, value2, value3'`. +To send data to Axiom in JSON format: +1. Encode the events in CSV format. The first line specifies the field names separated by commas. Subsequent new lines specify the values separated by commas. +1. Enter the CSV representation in the body of the API request. +1. Optional: In the body of the request, set optional parameters such as `timestamp-field` and `timestamp-format`. For more information, see the [ingest data API reference](/restapi/endpoints/ingestIntoDataset). +1. Set the `Content-Type` header to `text/csv`. +1. Set the `Authorization` header to `Bearer API_TOKEN`. Replace `API_TOKEN` with the Axiom API token you have generated. +1. Send the POST request to `https://api.axiom.co/v1/datasets/DATASET_NAME/ingest`. Replace `DATASET_NAME` with the name of the Axiom dataset where you want to send data. -### Example Request using CSV +**Example request** ```bash -curl -X 'POST' 'https://api.axiom.co/v1/datasets/{dataset_name}/ingest' \ - -H 'Authorization: Bearer API_TOKEN' \ - -H 'Content-Type: text/csv' \ - -d 'user, name - foo, bar' +curl -X 'POST' 'https://api.axiom.co/v1/datasets/DATASET_NAME/ingest' \ + -H 'Authorization: Bearer API_TOKEN' \ + -H 'Content-Type: text/csv' \ + -d 'user, name + foo, bar' ``` -### Example Response + -A successful POST Request returns a 200 response code JSON with details: +**Example response** ```json { @@ -235,4 +251,21 @@ A successful POST Request returns a 200 response code JSON with details: } ``` -Datasets names are usually case sensitive, Dataset names must be between 1-128 characters, and may only contain ASCII alphanumeric characters and the '-' character. +## Send data with Axiom Node.js + +1. [Install and configure](/guides/javascript#use-axiomhq-js) the Axiom Node.js library. +1. Encode the events as JSON objects. +1. Pass the dataset name and the array of JSON objects to the `axiom.ingest` function. + + ```ts + axiom.ingest('DATASET_NAME', [{ foo: 'bar' }]); + await axiom.flush(); + ``` + + + +For more information on other libraries you can use to query data, see [Send data](send-data/ingest). + +## What’s next + +After ingesting data to Axiom, you can [query it via API](/restapi/query) or the [Axiom app UI](/query-data/explore). \ No newline at end of file diff --git a/restapi/introduction.mdx b/restapi/introduction.mdx index fa9cac85..2a9056f9 100644 --- a/restapi/introduction.mdx +++ b/restapi/introduction.mdx @@ -24,7 +24,7 @@ Axiom API follows the REST architectural style and uses JSON for serialization. For example, the following curl command ingests data to an Axiom dataset: ```bash -curl -X 'POST' 'https://api.axiom.co/v1/datasets/{dataset_name}/ingest' \ +curl -X 'POST' 'https://api.axiom.co/v1/datasets/DATASET_NAME/ingest' \ -H 'Authorization: Bearer API_TOKEN' \ -H 'Content-Type: application/json' \ -d '[ @@ -34,6 +34,8 @@ curl -X 'POST' 'https://api.axiom.co/v1/datasets/{dataset_name}/ingest' \ ]' ``` +For more information, see [Send data to Axiom via API](/restapi/ingest) and [Ingest data endpoint](/restapi/endpoints/ingestIntoDataset). + ## Regions All examples in the Axiom API reference use the base domain `https://api.axiom.co`, which is the default for the US region. If your organization uses the EU region, change the base domain in the examples to `https://api.eu.axiom.co`. @@ -54,14 +56,14 @@ To prove that API requests come from you, you must include forms of authenticati If you use an API token for authentication, include the API token in the `Authorization` header. ```bash -Authorization: Bearer {token} +Authorization: Bearer API_TOKEN ``` If you use a PAT for authentication, include the PAT in the `Authorization` header and the org ID in the `x-axiom-org-id` header. For more information, see [Determine org ID](/reference/tokens#determine-org-id). ```bash -Authorization: Bearer {token} -x-axiom-org-id: {org_id} +Authorization: Bearer API_TOKEN +x-axiom-org-id: ORG_ID ``` If authentication is unsuccessful for a request, Axiom returns the error status code `403`. @@ -79,3 +81,8 @@ Below is a list of the types of data used within the Axiom API: | **Float** | A number with decimals. | 15.67 | | **Map** | A data structure with a list of values assigned to a unique key. | \{ "key": "value" \} | | **List** | A data structure with only a list of values separated by a comma. | ["value", 4567, 45.67] | + +## What's next + +- [Ingest data via API](/restapi/ingest) +- [Query data via API](/restapi/query) \ No newline at end of file diff --git a/restapi/query.mdx b/restapi/query.mdx index 2ee9abbf..d2f56651 100644 --- a/restapi/query.mdx +++ b/restapi/query.mdx @@ -1,494 +1,334 @@ --- title: 'Query data via Axiom API' -description: 'Learn how to use Axiom querying API to create and get query objects.' +description: 'Learn how to use the Axiom API to query data.' sidebarTitle: Query data tags: ['axiom documentation', 'documentation', 'axiom', 'axiom api', 'rest api', 'rest', 'authorization', 'query', 'querying', 'node js', 'headers', 'send data', 'ingesting', 'json', 'arrays', 'nested arrays', 'objects', 'strings', 'csv', 'response'] --- -Use Axiom querying API to create and get query objects. +import Prerequisites from "/snippets/standard-prerequisites.mdx" +import ReplaceToken from "/snippets/replace-token.mdx" -## Authorization and headers +This page explains how to query data via the Axiom API using the following: -The only expected header is `Authorization: Bearer` which is your token to authenticate the request. For more information, see [Tokens](/reference/tokens). +- [cURL](#query-data-with-curl) +- [Axiom Node.js library](#query-data-with-axiom-nodejs) -## Using Axiom Node.js library to query data +For an introduction to the basics of the Axiom API and to the authentication options, see [Introduction to Axiom API](/restapi/introduction). -Axiom maintains the [axiom-js](https://github.com/axiomhq/axiom-js) to provide official Node.js bindings for the Axiom API. +The API requests on this page use the query data endpoint. For more information, see the [API reference](/restapi/endpoints/queryApl). -Install using `npm install`: + -```shell -npm install @axiomhq/js -``` - -If you use the [Axiom CLI](https://github.com/axiomhq/cli), run `eval $(axiom config export -f)` to configure your environment variables. - -Otherwise, create an [API token](/reference/tokens) and export it as `AXIOM_TOKEN`. - -Create and use a client like this: - -```ts -// The purpose of this example is to show how to query a dataset using the Axiom -// Processing Language (APL). -import { Axiom } from '@axiomhq/js'; - -const axiom = new Axiom({ - token: process.env.AXIOM_TOKEN -}); - -async function query() { - const aplQuery = "['flights'] | where altitude > 49000 and flight != '' "; - - const res = await axiom.query(aplQuery); - if (!res.matches || res.matches.length === 0) { - console.warn('no matches found'); - return; - } - - for (let matched of res.matches) { - console.log(matched.data); - } -} - -query(); -``` - -In the above example we’re querying a dataset containing contemporary flight data obtained from an ADSB antenna. Results may look similar to this: - -```json -{ - aircraft: null, - altitude: 123600, - category: null, - flight: 'BCI96D ', - hex: '407241', - lat: 50.951285, - lon: -1.347961, - messages: 13325, - mlat: [ 'lat', 'lon', 'track', 'speed', 'vert_rate' ], - now: null, - nucp: 0, - rssi: -13.3, - seen: 3.6, - seen_pos: 19.7, - speed: 260, - squawk: '6014', - tisb: [], - track: 197, - type: null, - vert_rate: 64 -} -{ - aircraft: null, - altitude: 123600, - category: null, - flight: 'BCI96D ', - hex: '407241', - lat: 50.951285, - lon: -1.347961, - messages: 13325, - mlat: [ 'lat', 'lon', 'track', 'speed', 'vert_rate' ], - now: null, - nucp: 0, - rssi: -13.3, - seen: 4.6, - seen_pos: 20.8, - speed: 260, - squawk: '6014', - tisb: [], - track: 197, - type: null, - vert_rate: 64 -} -``` - -Further [examples](https://github.com/axiomhq/axiom-js/tree/main/examples/js) can be found in the [axiom-js](https://github.com/axiomhq/axiom-js) repo. - -## Querying via Curl using APL - -This section provides a guide on how to leverage the power of APL through curl commands. By combining the flexibility of curl with the querying capabilities of APL, users can seamlessly fetch and analyze their data right from the terminal. - -Whether you’re looking to fetch specific data points, aggregate metrics over time, or filter datasets based on certain criteria, the examples provided here will serve as a foundation to build upon. As you become more familiar with APL’s syntax and curl’s options, you'll find that the possibilities are vast and the insights you can derive are profound. - -## Examples - -## Count of distinct routes - -```bash -curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \ --H 'Authorization: Bearer API_TOKEN' \ --H 'Accept: application/json' \ --H 'Accept-Encoding: gzip' \ --H 'Content-Type: application/json' \ --d '{ - "apl": "vercel | summarize Count = dcount(vercel.route)", - "startTime": "2023-08-15T00:00:00Z", - "endTime": "2023-08-22T00:00:00Z" - }' -``` +## Query data with cURL -## Top 5 routes by count +To query data with cURL: -```bash -curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \ --H 'Authorization: Bearer API_TOKEN' \ --H 'Accept: application/json' \ --H 'Accept-Encoding: gzip' \ --H 'Content-Type: application/json' \ --d '{ - "apl": "vercel | summarize Count = dcount(vercel.route)", - "startTime": "2023-08-15T00:00:00Z", - "endTime": "2023-08-22T00:00:00Z" - }' -``` - -## Average request duration - -```bash -curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \ --H 'Authorization: Bearer API_TOKEN' \ --H 'Accept: application/json' \ --H 'Accept-Encoding: gzip' \ --H 'Content-Type: application/json' \ --d '{ - "apl": "vercel | summarize AvgDuration = avg(vercel.duration)", - "startTime": "2023-08-15T00:00:00Z", - "endTime": "2023-08-22T00:00:00Z" - }' -``` - -## Requests with duration greater than 1 second - -```bash -curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \ --H 'Authorization: Bearer API_TOKEN' \ --H 'Accept: application/json' \ --H 'Accept-Encoding: gzip' \ --H 'Content-Type: application/json' \ --d '{ - "apl": "vercel | where vercel.duration > 1000", - "startTime": "2023-08-15T00:00:00Z", - "endTime": "2023-08-22T00:00:00Z" - }' -``` +1. Build the APL query. For more information, see [Introduction to APL](/apl/introduction). +1. Encode the APL query as a JSON object and enter it into the body of the API request. +1. Optional: In the body of the request, set optional parameters such as `startTime` and `endTime`. For more information, see the [query data API reference](/restapi/endpoints/queryApl). +1. Set the `Content-Type` header to `application/json`. +1. Set the `Authorization` header to `Bearer API_TOKEN`. Replace `API_TOKEN` with the Axiom API token you have generated. +1. Send the POST request to one of the following: + - For tabular output, use `https://api.axiom.co/v1/datasets/_apl?format=tabular`. + - For legacy output, use `https://api.axiom.co/v1/datasets/_apl?format=legacy`. -## Top 3 routes with the highest average duration +### Example ```bash -curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \ --H 'Authorization: Bearer API_TOKEN' \ --H 'Accept: application/json' \ --H 'Accept-Encoding: gzip' \ --H 'Content-Type: application/json' \ --d '{ - "apl": "vercel | summarize AvgDuration = avg(vercel.duration) by vercel.route | top 3 by AvgDuration desc", - "startTime": "2023-08-15T00:00:00Z", - "endTime": "2023-08-22T00:00:00Z" - }' -``` - -## Requests grouped by hour - -```bash -curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \ --H 'Authorization: Bearer API_TOKEN' \ --H 'Accept: application/json' \ --H 'Accept-Encoding: gzip' \ --H 'Content-Type: application/json' \ --d '{ - "apl": "vercel | summarize Count = count() by bin(_time, 1h)", - "startTime": "2023-08-15T00:00:00Z", - "endTime": "2023-08-22T00:00:00Z" - }' -``` - -## Requests with errors - -```bash -curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \ --H 'Authorization: Bearer API_TOKEN' \ --H 'Accept: application/json' \ --H 'Accept-Encoding: gzip' \ --H 'Content-Type: application/json' \ --d '{ - "apl": "vercel | where vercel.status >= 400", - "startTime": "2023-08-15T00:00:00Z", - "endTime": "2023-08-22T00:00:00Z" - }' -``` - -## Getting the most common user agents - -```bash -curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \ --H 'Authorization: Bearer API_TOKEN' \ --H 'Accept: application/json' \ --H 'Accept-Encoding: gzip' \ --H 'Content-Type: application/json' \ --d '{ - "apl": "[\"sample-http-logs\"] | summarize count() by user_agent | top 5 by count_", - "startTime": "2023-08-15T00:00:00Z", - "endTime": "2023-08-22T00:00:00Z" - }' -``` - -## Identifying the server data centers with the highest number of requests - -```bash -curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \ --H 'Authorization: Bearer API_TOKEN' \ --H 'Accept: application/json' \ --H 'Accept-Encoding: gzip' \ --H 'Content-Type: application/json' \ --d '{ - "apl": "[\"sample-http-logs\"] | summarize count() by server_datacenter | top 3 by count_", - "startTime": "2023-08-15T00:00:00Z", - "endTime": "2023-08-22T00:00:00Z" - }' -``` - -## Identifying the average, minimum, and maximum request duration for each method type - -```bash -curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \ --H 'Authorization: Bearer API_TOKEN' \ --H 'Accept: application/json' \ --H 'Accept-Encoding: gzip' \ --H 'Content-Type: application/json' \ --d '{ - "apl": "[\"sample-http-logs\"] | summarize avg(todouble(req_duration_ms)), min(todouble(req_duration_ms)), max(todouble(req_duration_ms)) by method", - "startTime": "2023-08-15T00:00:00Z", - "endTime": "2023-08-22T00:00:00Z" - }' -``` - -## Finding the top 3 URIs accessed via TLS connections with a response body size greater than a specified threshold - -```bash -curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \ --H 'Authorization: Bearer API_TOKEN' \ --H 'Accept: application/json' \ --H 'Accept-Encoding: gzip' \ --H 'Content-Type: application/json' \ --d '{ - "apl": "[\"sample-http-logs\"] | where is_tls == true and todouble(resp_body_size_bytes) > 5000 | summarize count() by uri | top 3 by count()", - "startTime": "2023-08-15T00:00:00Z", - "endTime": "2023-08-22T00:00:00Z" - }' -``` - -## Calculating the 95th percentile of the request duration for each server datacenter - -```bash -curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \ --H 'Authorization: Bearer API_TOKEN' \ --H 'Accept: application/json' \ --H 'Accept-Encoding: gzip' \ --H 'Content-Type: application/json' \ --d '{ - "apl": "[\"sample-http-logs\"] | summarize percentile(todouble(req_duration_ms), 95) by server_datacenter", - "startTime": "2023-08-15T00:00:00Z", - "endTime": "2023-08-22T00:00:00Z" - }' -``` - -## Active issue contributors - -```bash -curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \ --H 'Authorization: Bearer API_TOKEN' \ --H 'Accept: application/json' \ --H 'Accept-Encoding: gzip' \ --H 'Content-Type: application/json' \ --d '{ - "apl": "[\"github-issue-comment-event\"] | where repo startswith \"kubernetes/\" | where actor !endswith \"[bot]\" | summarize dcount(actor) by bin_auto(_time)", - "startTime": "2023-08-15T00:00:00Z", - "endTime": "2023-08-22T00:00:00Z" - }' -``` - -## Top Issue Wranglers - -```bash -curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \ --H 'Authorization: Bearer API_TOKEN' \ --H 'Accept: application/json' \ --H 'Accept-Encoding: gzip' \ --H 'Content-Type: application/json' \ --d '{ - "apl": "[\"github-issues-event\"] | where actor !endswith \"[bot]\" and repo startswith \"cockroachdb/\" and actor !~ \"cockroach-teamcity\" | summarize topk(actor, 5) by bin_auto(_time), action", - "startTime": "2023-08-15T00:00:00Z", - "endTime": "2023-08-22T00:00:00Z" - }' -``` - -## Using Curl to query the API - -`POST api.axiom.co/v1/datasets/\{id\}/query` - -```bash -curl -X 'POST' \ - 'https://api.axiom.co/v1/datasets//query?saveAsKind=&streaming-duration=&nocache=true' \ - -H 'Content-Type: application/json' \ - -H 'Authorization: Bearer API_TOKEN' \ - -d '{ - "aggregations": [ - { - "alias": "string", - "argument": {}, - "field": "string", - "op": "count" - } - ], - "continuationToken": "string", - "cursor": "string", - "endTime": "string", - "filter": { - "caseSensitive": true, - "children": [ - "string" - ], - "field": "string", - "op": "and", - "value": {} - }, - "groupBy": [ - "string" - ], - "includeCursor": true, - "limit": 0, - "order": [ - { - "desc": true, - "field": "string" - } - ], - "project": [ - { - "alias": "string", - "field": "string" - } - ], - "queryOptions": { - "against": "string", - "againstStart": "string", - "againstTimestamp": "string", - "caseSensitive": "string", - "containsTimeFilter": "string", - "datasets": "string", - "displayNull": "string", - "editorContent": "string", - "endColumn": "string", - "endLineNumber": "string", - "endTime": "string", - "integrationsFilter": "string", - "openIntervals": "string", - "quickRange": "string", - "resolution": "string", - "startColumn": "string", - "startLineNumber": "string", - "startTime": "string", - "timeSeriesView": "string" - }, - "resolution": "string", +curl --request POST \ + --url 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \ + --header 'Authorization: Bearer API_TOKEN' \ + --header 'Content-Type: application/json' \ + --data '{ + "apl": "http-logs | limit 10", "startTime": "string", - "virtualFields": [ - { - "alias": "string", - "expr": "string" - } - ] + "endTime": "string" }' ``` -## Response Example + -Response code **200** and the response body: +**Example response** -```json +```json [expandable] { - "buckets": { - "series": [ - { - "endTime": "2022-07-26T03:00:48.925Z", - "groups": [ - { - "aggregations": [ - { - "op": "string", - "value": {} - } - ], - "group": { - "additionalProp1": {}, - "additionalProp2": {}, - "additionalProp3": {} - }, - "id": 0 - } - ], - "startTime": "2022-07-26T03:00:48.925Z" - } - ], - "totals": [ - { - "aggregations": [ - { - "op": "string", - "value": {} - } - ], - "group": { - "additionalProp1": {}, - "additionalProp2": {}, - "additionalProp3": {} - }, - "id": 0 - } - ] + "format": "tabular", + "status": { + "elapsedTime": 260650, + "minCursor": "0d8q6stroluyo-07c3957e7400015c-0000c875", + "maxCursor": "0d8q6stroluyo-07c3957e7400015c-0000c877", + "blocksExamined": 4, + "blocksCached": 0, + "blocksMatched": 0, + "rowsExamined": 197604, + "rowsMatched": 197604, + "numGroups": 0, + "isPartial": false, + "cacheStatus": 1, + "minBlockTime": "2025-03-26T12:03:14Z", + "maxBlockTime": "2025-03-26T12:12:42Z" }, - "fieldsMeta": [ + "tables": [ { - "description": "string", - "hidden": true, - "name": "string", - "type": "string", - "unit": "string" + "name": "0", + "sources": [ + { + "name": "http-logs" + } + ], + "fields": [ + { + "name": "_sysTime", + "type": "datetime" + }, + { + "name": "_time", + "type": "datetime" + }, + { + "name": "content_type", + "type": "string" + }, + { + "name": "geo.city", + "type": "string" + }, + { + "name": "geo.country", + "type": "string" + }, + { + "name": "id", + "type": "string" + }, + { + "name": "is_tls", + "type": "boolean" + }, + { + "name": "message", + "type": "string" + }, + { + "name": "method", + "type": "string" + }, + { + "name": "req_duration_ms", + "type": "float" + }, + { + "name": "resp_body_size_bytes", + "type": "integer" + }, + { + "name": "resp_header_size_bytes", + "type": "integer" + }, + { + "name": "server_datacenter", + "type": "string" + }, + { + "name": "status", + "type": "string" + }, + { + "name": "uri", + "type": "string" + }, + { + "name": "user_agent", + "type": "string" + }, + { + "name": "is_ok_2 ", + "type": "boolean" + }, + { + "name": "city_str_len", + "type": "integer" + } + ], + "order": [ + { + "field": "_time", + "desc": true + } + ], + "groups": [], + "range": { + "field": "_time", + "start": "1970-01-01T00:00:00Z", + "end": "2025-03-26T12:12:43Z" + }, + "columns": [ + [ + "2025-03-26T12:12:42.68112905Z", + "2025-03-26T12:12:42.68112905Z", + "2025-03-26T12:12:42.68112905Z" + ], + [ + "2025-03-26T12:12:42Z", + "2025-03-26T12:12:42Z", + "2025-03-26T12:12:42Z" + ], + [ + "text/html", + "text/plain-charset=utf-8", + "image/jpeg" + ], + [ + "Ojinaga", + "Humboldt", + "Nevers" + ], + [ + "Mexico", + "United States", + "France" + ], + [ + "8af366cf-6f25-42e6-bbb4-d860ab535a60", + "032e7f68-b0ab-47c0-a24a-35af566359e5", + "4d2c7baa-ff28-4b1f-9db9-8e6c0ed5a9c9" + ], + [ + false, + false, + true + ], + [ + "QCD permutations were not solvable in linear time, expected compressed time", + "QCD permutations were not solvable in linear time, expected compressed time", + "Expected a new layer of particle physics but got a Higgs Boson" + ], + [ + "GET", + "GET", + "GET" + ], + [ + 1.396373193863436, + 0.16252390534308514, + 0.4093416175186162 + ], + [ + 3448, + 2533, + 1906 + ], + [ + 84, + 31, + 29 + ], + [ + "DCA", + "GRU", + "FRA" + ], + [ + "201", + "200", + "200" + ], + [ + "/api/v1/buy/commit/id/go", + "/api/v1/textdata/cnfigs", + "/api/v1/bank/warn" + ], + [ + "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; AS; rv:11.0) like Gecko", + "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/535.24 (KHTML, like Gecko) Chrome/19.0.1055.1 Safari/535.24", + "Mozilla/5.0 (Windows; U; MSIE 9.0; WIndows NT 9.0; en-US))" + ], + [ + true, + true, + true + ], + [ + 7, + 8, + 6 + ] + ] } ], - "matches": [ - { - "_rowId": "string", - "_sysTime": "2022-07-26T03:00:48.925Z", - "_time": "2022-07-26T03:00:48.925Z", - "data": { - "additionalProp1": {}, - "additionalProp2": {}, - "additionalProp3": {} - } - } + "datasetNames": [ + "http-logs" ], - "status": { - "blocksExamined": 0, - "cacheStatus": 0, - "continuationToken": "string", - "elapsedTime": 0, - "isEstimate": true, - "isPartial": true, - "maxBlockTime": "2022-07-26T03:00:48.925Z", - "messages": [ + "fieldsMetaMap": { + "http-logs": [ + { + "name": "status", + "type": "", + "unit": "", + "hidden": false, + "description": "HTTP status code" + }, + { + "name": "resp_header_size_bytes", + "type": "integer", + "unit": "none", + "hidden": false, + "description": "" + }, + { + "name": "geo.city", + "type": "string", + "unit": "", + "hidden": false, + "description": "the city" + }, + { + "name": "resp_body_size_bytes", + "type": "integer", + "unit": "decbytes", + "hidden": false, + "description": "" + }, + { + "name": "content_type", + "type": "string", + "unit": "", + "hidden": false, + "description": "" + }, { - "code": "string", - "count": 0, - "msg": "string", - "priority": "string" + "name": "geo.country", + "type": "string", + "unit": "", + "hidden": false, + "description": "" + }, + { + "name": "req_duration_ms", + "type": "float", + "unit": "ms", + "hidden": false, + "description": "Request duration" } - ], - "minBlockTime": "2022-07-26T03:00:48.925Z", - "numGroups": 0, - "rowsExamined": 0, - "rowsMatched": 0 + ] } } ``` + +## Query data with Axiom Node.js + +1. [Install and configure](/guides/javascript#use-axiomhq-js) the Axiom Node.js library. +1. Build the APL query. For more information, see [Introduction to APL](/apl/introduction). +1. Pass the APL query as a string to the `axiom.query` function. + + ```ts + const res = await axiom.query(`['DATASET_NAME'] | where foo == 'bar' | limit 100`); + console.log(res); + ``` + + + +For more examples, see the [examples in GitHub](https://github.com/axiomhq/axiom-js/tree/main/examples). + +For more information on other libraries you can use to query data, see [Send data](send-data/ingest). \ No newline at end of file