feat(spark): add trunc, date_trunc and time_trunc functions

[cht42]

Which issue does this PR close?

• Closes [datafusion-spark] Add date_trunc, time_trunc and trunc functions #19828.
• Part of [EPIC] Complete datafusion-spark Spark Compatible Functions #15914

Rationale for this change

implement spark:

• https://spark.apache.org/docs/latest/api/sql/index.html#trunc
• https://spark.apache.org/docs/latest/api/sql/index.html#date_trunc
• https://spark.apache.org/docs/latest/api/sql/index.html#time_trunc

What changes are included in this PR?

Add spark compatible wrappers around datafusion date_trunc function to handle spark specificities.

Are these changes tested?

Yes in SLT

Are there any user-facing changes?

Yes

[Jefffrey]

Just concerned about if matching return field nullability here is something we should watch for?

[cht42]

yep DF's date_trunc returm field will be nullable.. if #19511 goes through it should fix this issue

[Jefffrey]

I don't see #19511 landing anytime soon so we might need to fix this in the DF date_trunc to ensure consistency here

[cht42]

done

[martin-g]

Suggested change

	let fmt_expr = &args[0];
	let [fmt_expr, time_expr] = take_function_args(self.name(), args)?;

[martin-g]

fmt is normalized (lowercased) above and validated but here you pass the original args (non-normalized).
Maybe it will be better to pass the fmt:

let fmt_expr = Expr::Literal(ScalarValue::new_utf8(fmt.as_str()), None);
...
ScalarFunction::new_udf(
    datafusion_functions::datetime::date_trunc(),
    vec![fmt_expr, time_expr],
),

[cht42]

shouldn't matter, DF will handle the original argument as well and lowercase it

[andygrove]

@cht42 could you add tests for DST handling with different time zones? I am not convinced that the current PR handles this correctly.

Here is an AI-generated test that highlights the issue:

# Test with explicit timezone in timestamp
statement ok
SET datafusion.execution.time_zone = 'America/New_York';

# Cross-day boundary: 03:30 UTC on July 15 = 23:30 EDT on July 14
# TODO: Spark returns 2024-07-14 (converts to session tz first)
# DataFusion returns 2024-07-15 (truncates in UTC)
query P
SELECT date_trunc('DAY', '2024-07-15T03:30:00Z'::timestamp);
----
2024-07-15T00:00:00

# Reset timezone for other tests
statement ok
RESET datafusion.execution.time_zone

Spark repro:

$ /opt/spark-3.5.7-bin-hadoop3/bin/spark-sql --conf spark.sql.session.timeZone=America/New_York -e "SELECT date_trunc('DAY', timestamp'2024-07-15T03:30:00Z');"
...
Spark master: local[*], Application Id: local-1768499563499
2024-07-14 00:00:00

[cht42]

good catch @andygrove , I added the tests and some logic to handle those cases. let me know what you think

[Jefffrey]

This seems like a bug

[Jefffrey]

The above arm catches all timestamps though? Since it doesn't match on Some(tz); so when would this arm take effect?

[andygrove]

good catch @andygrove , I added the tests and some logic to handle those cases. let me know what you think

Looks good. Thanks for addressing that.