Skip to content

Commit da30a95

Browse files
DeviousCardiclaude
andcommitted
Document that a volatile coalesce or nvl argument is evaluated eagerly
`nvl`'s user documentation promised that the second argument "is not evaluated". That stops being true once the volatility guard skips the `CASE` rewrite, because `ScalarFunctionExpr::evaluate` evaluates every child before calling `invoke_with_args`, and nothing in the physical layer defers an argument. `coalesce` promised nothing either way, but the new behaviour is worth stating there too. `scalar_functions.md` is generated from these `#[user_doc]` attributes by `dev/update_function_docs.sh`, and CI fails on drift, so the regenerated file is included here. Also adds a 56.0.0 upgrade-guide entry, as `api-health.md` asks for user-visible SQL changes, modelled on 54.0.0's evaluation-order section and pointing at the same `CASE` workaround. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent cec2357 commit da30a95

4 files changed

Lines changed: 30 additions & 4 deletions

File tree

‎datafusion/functions/src/core/coalesce.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ use itertools::Itertools;
3232

3333
#[user_doc(
3434
doc_section(label = "Conditional Functions"),
35-
description = "Returns the first of its arguments that is not _null_. Returns _null_ if all arguments are _null_. This function is often used to substitute a default value for _null_ values.",
35+
description = "Returns the first of its arguments that is not _null_. Returns _null_ if all arguments are _null_. Arguments after the first non-_null_ one are normally not evaluated, but when any argument other than the last is volatile every argument is evaluated, so that each is evaluated exactly once. This function is often used to substitute a default value for _null_ values.",
3636
syntax_example = "coalesce(expression1[, ..., expression_n])",
3737
sql_example = r#"```sql
3838
> select coalesce(null, null, 'datafusion');

‎datafusion/functions/src/core/nvl.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ use datafusion_macros::user_doc;
2727

2828
#[user_doc(
2929
doc_section(label = "Conditional Functions"),
30-
description = "Returns _expression2_ if _expression1_ is NULL otherwise it returns _expression1_ and _expression2_ is not evaluated. This function can be used to substitute a default value for NULL values.",
30+
description = "Returns _expression2_ if _expression1_ is NULL otherwise it returns _expression1_. _expression2_ is normally not evaluated, but when _expression1_ is volatile both arguments are evaluated, so that _expression1_ is evaluated exactly once. This function can be used to substitute a default value for NULL values.",
3131
syntax_example = "nvl(expression1, expression2)",
3232
sql_example = r#"```sql
3333
> select nvl(null, 'a');

‎docs/source/library-user-guide/upgrading/56.0.0.md‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -577,3 +577,29 @@ Wire compatibility is directional:
577577
- A 55.0 reader must not consume an alias-preserving 56.0 MERGE payload. It
578578
ignores the unknown field but cannot preserve the qualifier required by the
579579
expressions, which can cause resolution failure or incorrect rebinding.
580+
### A volatile `coalesce` or `nvl` argument is now evaluated eagerly
581+
582+
`coalesce` and `nvl` are normally rewritten to `CASE WHEN a IS NOT NULL THEN a
583+
ELSE b END`, which names every argument but the last one twice. For a volatile
584+
argument those two mentions were two independent draws, so
585+
`coalesce(nullif(floor(random() * 2), 0), -1)` could return `NULL` — a value a
586+
single evaluation can never produce — or fail at run time with
587+
`Column 'c' is declared as non-nullable but contains null values`.
588+
589+
The rewrite is now skipped when any argument other than the last is volatile,
590+
and the call is evaluated by a kernel that reads each argument exactly once.
591+
That fixes the wrong results, but it also means the remaining arguments are
592+
evaluated rather than skipped:
593+
594+
```sql
595+
-- previously returned rows, because `y / x` was never evaluated;
596+
-- now raises `Divide by zero error`
597+
SELECT coalesce(random(), y / x) FROM t;
598+
```
599+
600+
Only calls that contain a volatile argument before the last one are affected.
601+
A volatile *last* argument still takes the lazy rewrite, and non-volatile
602+
`coalesce` and `nvl` are unchanged.
603+
604+
To force conditional evaluation, rewrite using `CASE`, which has standardized
605+
short-circuit semantics.

‎docs/source/user-guide/sql/scalar_functions.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -962,7 +962,7 @@ trunc(numeric_expression[, decimal_places])
962962

963963
### `coalesce`
964964

965-
Returns the first of its arguments that is not _null_. Returns _null_ if all arguments are _null_. This function is often used to substitute a default value for _null_ values.
965+
Returns the first of its arguments that is not _null_. Returns _null_ if all arguments are _null_. Arguments after the first non-_null_ one are normally not evaluated, but when any argument other than the last is volatile every argument is evaluated, so that each is evaluated exactly once. This function is often used to substitute a default value for _null_ values.
966966

967967
```sql
968968
coalesce(expression1[, ..., expression_n])
@@ -1066,7 +1066,7 @@ nullif(expression1, expression2)
10661066

10671067
### `nvl`
10681068

1069-
Returns _expression2_ if _expression1_ is NULL otherwise it returns _expression1_ and _expression2_ is not evaluated. This function can be used to substitute a default value for NULL values.
1069+
Returns _expression2_ if _expression1_ is NULL otherwise it returns _expression1_. _expression2_ is normally not evaluated, but when _expression1_ is volatile both arguments are evaluated, so that _expression1_ is evaluated exactly once. This function can be used to substitute a default value for NULL values.
10701070

10711071
```sql
10721072
nvl(expression1, expression2)

0 commit comments

Comments
 (0)