generated from mintlify/starter
-
Notifications
You must be signed in to change notification settings - Fork 7
Add column_ifexists #273
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
tothmano
wants to merge
2
commits into
main
Choose a base branch
from
mano/column-ifexists
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Add column_ifexists #273
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
150 changes: 150 additions & 0 deletions
150
apl/scalar-functions/conditional-functions/column-ifexists.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,150 @@ | ||
--- | ||
title: column_ifexists | ||
description: 'This page explains how to use the column_ifexists function in APL.' | ||
--- | ||
|
||
Use `column_ifexists()` to make your queries resilient to schema changes. The function checks if a field with a given name exists in the dataset. If it does, the function returns it. If not, it returns a fallback field or expression that you provide. | ||
|
||
This is especially useful when working with datasets that evolve over time or come from multiple sources with different schemas. Instead of failing when a field is missing, your query continues running by using a default. Use this function to safely handle queries where the presence of a field isn’t guaranteed. | ||
|
||
## For users of other query languages | ||
|
||
If you come from other query languages, this section explains how to adjust your existing queries to achieve the same results in APL. | ||
|
||
<AccordionGroup> | ||
<Accordion title="Splunk SPL users"> | ||
|
||
In Splunk, field selection is strict—missing fields typically return `null` in results, but conditional logic for fallback fields requires using `eval` or `coalesce`. In APL, `column_ifexists()` directly substitutes the fallback field at query-time based on schema. | ||
|
||
<CodeGroup> | ||
```sql Splunk example | ||
... | eval field=if(isnull(Capital), State, Capital) | ||
``` | ||
|
||
```kusto APL equivalent | ||
StormEvents | project column_ifexists('Capital', State) | ||
``` | ||
</CodeGroup> | ||
|
||
</Accordion> | ||
<Accordion title="ANSI SQL users"> | ||
|
||
In SQL, you need to check for the existence of a field using system views or error handling. `column_ifexists()` in APL simplifies this by allowing fallback behavior inline without needing procedural code. | ||
|
||
<CodeGroup> | ||
```sql SQL example | ||
SELECT CASE | ||
WHEN EXISTS(SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS | ||
WHERE TABLE_NAME = 'StormEvents' AND COLUMN_NAME = 'Capital') | ||
THEN Capital ELSE State END AS Result | ||
FROM StormEvents | ||
``` | ||
|
||
```kusto APL equivalent | ||
StormEvents | project column_ifexists('Capital', State) | ||
``` | ||
</CodeGroup> | ||
|
||
</Accordion> | ||
</AccordionGroup> | ||
|
||
## Usage | ||
|
||
### Syntax | ||
|
||
```kusto | ||
column_ifexists(FieldName, DefaultValue) | ||
``` | ||
|
||
### Parameters | ||
|
||
- `FieldName`: The name of the field to return as a string. | ||
- `DefaultValue`: The fallback value to return if `FieldName` doesn’t exist. This can be another field or a literal. | ||
|
||
### Returns | ||
|
||
Returns the field specified by `FieldName` if it exists in the table schema. Otherwise, returns the result of `DefaultValue`. | ||
|
||
## Use case examples | ||
|
||
<Tabs> | ||
<Tab title="Log analysis"> | ||
|
||
You want to examine HTTP logs, and your schema might have a `geo.region` field in some environments and not in others. You fall back to `geo.country` when `geo.region` is missing. | ||
|
||
**Query** | ||
|
||
```kusto | ||
['sample-http-logs'] | ||
| project _time, location = column_ifexists('geo.region', ['geo.country']) | ||
``` | ||
|
||
[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'sample-http-logs'%5D%20%7C%20project%20_time%2C%20location%20%3D%20column_ifexists('geo.region'%2C%20%5B'geo.country'%5D)%22%7D) | ||
|
||
**Output** | ||
|
||
| _time | location | | ||
|----------------------|----------------| | ||
| 2025-04-28T12:04:10Z | United States | | ||
| 2025-04-28T12:04:12Z | Canada | | ||
| 2025-04-28T12:04:15Z | United Kingdom | | ||
|
||
The query returns `geo.region` if it exists; otherwise, it falls back to `geo.country`. | ||
|
||
</Tab> | ||
<Tab title="OpenTelemetry traces"> | ||
|
||
You analyze OpenTelemetry traces and you’re not sure if your data contains a `status_code` field. You fall back to `OK` when it’s missing. | ||
|
||
**Query** | ||
|
||
```kusto | ||
['otel-demo-traces'] | ||
| extend status = column_ifexists('status_code', 'OK') | ||
| project _time, trace_id, span_id, status | ||
``` | ||
|
||
[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'otel-demo-traces'%5D%20%7C%20extend%20status%20%3D%20column_ifexists('status_code'%2C%20'OK')%20%7C%20project%20_time%2C%20trace_id%2C%20span_id%2C%20status%22%7D) | ||
|
||
**Output** | ||
|
||
| _time | trace_id | span_id | status | | ||
|---------------------|--------------|-------------|--------| | ||
| 2025-04-28T10:30:12Z | abc123 | span567 | nil | | ||
| 2025-04-28T10:30:15Z | def456 | span890 | 200 | | ||
|
||
The query returns `status_code` if it exists. Otherwise, it falls back to `OK`. | ||
|
||
</Tab> | ||
<Tab title="Security logs"> | ||
|
||
You inspect logs for suspicious activity. In some datasets, a `threat_level` field exists, but not in all. You use the `status` field as a fallback. | ||
|
||
**Query** | ||
|
||
```kusto | ||
['sample-http-logs'] | ||
| project _time, id, threat = column_ifexists('threat_level', status) | ||
``` | ||
|
||
[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'sample-http-logs'%5D%20%7C%20project%20_time%2C%20id%2C%20threat%20%3D%20column_ifexists('threat_level'%2C%20status)%22%7D) | ||
|
||
**Output** | ||
|
||
| _time | id | threat | | ||
|---------------------|--------|--------| | ||
| 2025-04-28T13:22:11Z | u123 | 200 | | ||
| 2025-04-28T13:22:13Z | u456 | 403 | | ||
|
||
The function avoids breaking the query if `threat_level` doesn’t exist by defaulting to `status`. | ||
|
||
</Tab> | ||
</Tabs> | ||
|
||
## List of related functions | ||
|
||
- [coalesce](/apl/scalar-functions/string-functions#coalesce): Returns the first non-null value from a list of expressions. Use when you want to handle null values, not missing fields. | ||
- [iff](/apl/scalar-functions/conditional-function#iff): Performs conditional logic based on a boolean expression. Use when you want explicit control over evaluation. | ||
- [isnull](/apl/scalar-functions/string-functions#isnull): Checks if a value is null. Useful when combined with other functions for fine-grained control. | ||
- [case](/apl/scalar-functions/conditional-function#case): Allows multiple conditional branches. Use when fallback logic depends on multiple conditions. | ||
- [project](/apl/tabular-operators/project-operator): Selects and transforms fields. Use with `column_ifexists()` to build resilient field projections. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This
nil
should be"OK"
. And probably, we shouldn't mix types here. That always gets confusing. Probably better to return 100 or something.