Skip to content

Commit a516c6b

Browse files
ZetaSQL TeamKimiWaRokkuWoKikanai
ZetaSQL Team
authored and
KimiWaRokkuWoKikanai
committed
Export of internal ZetaSQL changes.
-- Change by ZetaSQL Team <[email protected]>: Update the resolved ast doc for ZetaSQL. -- Change by Brandon Dolphin <[email protected]>: Refactor out shared AddMeasureColumnsToTable function for working with measure columns. -- Change by Christoph Dibak <[email protected]>: Implement per-aggregation budgeting in the reference impl -- Change by Nevena Kotlaja <[email protected]>: No public description -- Change by Nevena Kotlaja <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Implement method TimestampPicoValue::FromString(). -- Change by ZetaSQL Team <[email protected]>: Fix standalone expression evaluation for multi-level aggregates. -- Change by ZetaSQL Team <[email protected]>: Add documentation for graph query syntax -- Change by ZetaSQL Team <[email protected]>: Fix a bug where information from `ExprResolutionInfo` is not propagated correctly up to parent `ExprResolutionInfos` for multi-level aggregation. -- Change by ZetaSQL Team <[email protected]>: Add SQL rewriter and missing analyzer tests for map_contains_key. -- Change by Brandon Dolphin <[email protected]>: Add deserialization for MeasureType. -- Change by ZetaSQL Team <[email protected]>: Add the parser implementation for pipe recursive union. -- Change by ZetaSQL Team <[email protected]>: Refactor ResolveMatchRecognize() into smaller methods. -- Change by ZetaSQL Team <[email protected]>: Add a RET_CHECK to ensure that PIVOT expressions in the reference implementation are not multi-level aggregations. -- Change by ZetaSQL Team <[email protected]>: Ensure that volatile grouping expressions are not considered grouping consts. -- Change by ZetaSQL Team <[email protected]>: Rename ResolvedMatchRecognizeEmptyPattern to ResolvedMatchRecognizePatternEmpty to have a consistent prefix with other nodes. -- Change by Brandon Dolphin <[email protected]>: Remove documented requirement for FEATURE_V_1_4_MULTILEVEL_AGGREGATION when analyzing measure expressions. -- Change by ZetaSQL Team <[email protected]>: Add more compliance tests for multi-level aggregation. -- Change by ZetaSQL Team <[email protected]>: Move the OPTIONS() list to the end of the MATCH_RECOGNIZE() syntax. -- Change by ZetaSQL Team <[email protected]>: Add a new option to AnalyzerOptions and ErrorMessageOptions to enable an enhanced form of error redaction which preserves some information about the error, while keeping the output stable. -- Change by ZetaSQL Team <[email protected]>: Coerce the predicate expression in MATCH_RECOGNIZE's DEFINE clause to BOOL before giving an error. -- Change by ZetaSQL Team <[email protected]>: Update the resolved ast documentation for the ZetaSQL release in Q4 2024. -- Change by ZetaSQL Team <[email protected]>: Add set operators to pipe syntax: UNION, INTERSECT, EXCEPT, CORRESPONDING -- Change by Yassin Ezbakhe <[email protected]>: `MatcherCollection`: Add `AddMatcher` method. -- Change by Jeff Shute <[email protected]>: Implement experimental terminal pipe operator for EXPORT DATA. -- Change by Brandon Dolphin <[email protected]>: Fix casing in table name extraction for `TABLE x` and `FROM x` referring to TVF args. -- Change by ZetaSQL Team <[email protected]>: Support pattern variable references in MATCH_RECOGNIZE's measures clause. -- Change by ZetaSQL Team <[email protected]>: Add SQL rewriter for map_get function. -- Change by Brandon Dolphin <[email protected]>: Update error message in JSON literal test. -- Change by ZetaSQL Team <[email protected]>: small changes -- Change by ZetaSQL Team <[email protected]>: Update COUNT and COUNTIF documentation. -- Change by Brandon Dolphin <[email protected]>: Fix typo in measure_expression target names. -- Change by Brandon Dolphin <[email protected]>: Add Java implementation of MeasureType. -- Change by ZetaSQL Team <[email protected]>: Add support for graph queries -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: internal -- Change by ZetaSQL Team <[email protected]>: Add MATCH_RECOGNIZE compliance test covering the scenario where a DEFINE expression evaluates to NULL rather than TRUE or FALSE. -- Change by ZetaSQL Team <[email protected]>: Remove unimplemented optimization-related items from the pattern matching api: -- Change by Brandon Dolphin <[email protected]>: Implement MeasureType::TypeNameWithModifiers. -- Change by Brandon Dolphin <[email protected]>: Fix typo in sql_test_base.cc variable name. -- Change by ZetaSQL Team <[email protected]>: Add ZetaSketch to the list of external applications that can be used to build sketches. -- Change by ZetaSQL Team <[email protected]>: Add anonymization threshold to the AnonymizationInfo class -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Fix struct construction syntax in examples of sparse vectors. -- Change by Jeff Shute <[email protected]>: Initial resolved AST and analyzer for pipe FORK. -- Change by ZetaSQL Team <[email protected]>: Internal fix. -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Add INTERVAL to JSON encoding documentation. -- Change by ZetaSQL Team <[email protected]>: Internal changes. -- Change by ZetaSQL Team <[email protected]>: Validate composite measure. -- Change by Brandon Dolphin <[email protected]>: Add deprecation warning for `TABLE x` syntax cases which will be affected by resolution order change. -- Change by ZetaSQL Team <[email protected]>: Add SQL rewriter for MAP_CARDINALITY -- Change by Brandon Dolphin <[email protected]>: Add feature `FEATURE_TABLE_SYNTAX_RESOLVE_ARGUMENT_LAST` to preserve the old (incorrect) resolution order for `TABLE x` syntax in SQL TVF bodies. -- Change by ZetaSQL Team <[email protected]>: Analyzer support for ON CONFLICT clause in INSERT DML -- Change by ZetaSQL Team <[email protected]>: Fix a bug in the script executor where redeclaring a variable without type parameters in a session would cause an error -- Change by Lev Kandel <[email protected]>: Prepare code for breaking change in Protobuf C++ API. Protobuf 6.30.0 will change the return types of Descriptor::name() and other methods to absl::string_view. This makes the code work both before and after such a change. -- Change by ZetaSQL Team <[email protected]>: Fix an incorrect comment. -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Change complexity limit for state machine algo from 15,000 states to 100,000 edges, as it's the number of edges, not states, that really determines the time and space requirements of the algorithm. Also, moved the check from NFABuilder into NFA itself so that it gets enforced inside of EpsilonRemover, which, in the worst case, can drastically increase the number of edges, while simultaneously reducing the number of states. -- Change by Jeff Shute <[email protected]>: Add some support for referencing Constants with values. -- Change by ZetaSQL Team <[email protected]>: Unify the creation of multi_level_aggregate_info QRI for all the paths that create a default. -- Change by John Fremlin <[email protected]>: Add RowIdentityColumns function to catalog Table -- Change by ZetaSQL Team <[email protected]>: Remove references to Bison from comments in gen_parse_tree.py. -- Change by ZetaSQL Team <[email protected]>: Remove "Bison" from the names of some functions and variables in keywords.h. -- Change by ZetaSQL Team <[email protected]>: Enable SELECT .. FOR UPDATE for external use. -- Change by ZetaSQL Team <[email protected]>: Fix the order of parameters in the Regexp::Extract function. -- Change by ZetaSQL Team <[email protected]>: Add support for UINT64 arguments to `TIMESTAMP_FROM_UNIX_SECONDS`, `TIMESTAMP_FROM_UNIX_MILLIS`, and `TIMESTAMP_FROM_UNIX_MICROS`. -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Enable multi-level aggregation in RQG. -- Change by ZetaSQL Team <[email protected]>: Fix a bug with deferred resolution for multi-level aggregates. -- Change by ZetaSQL Team <[email protected]>: Fix some formatting issues with ST_EQUALS. -- Change by ZetaSQL Team <[email protected]>: Remove "pipe operator" from pipe operator table to economize. -- Change by Laramie Leavitt <[email protected]>: Avoid creating a new PRNG on each test loop. -- Change by Jeff Shute <[email protected]>: Parser implementation for experimental pipe FORK. -- Change by ZetaSQL Team <[email protected]>: Internal Change (And 482 more changes) GitOrigin-RevId: 158e00ddbc5bb8b8ef75e6b6473daf4fcef6174a Change-Id: I6520749d9676a3fce2e84a49f3e1dd6804390ab7
1 parent 194cd32 commit a516c6b

File tree

711 files changed

+196848
-17044
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

711 files changed

+196848
-17044
lines changed

‎.bazelrc

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222
# suppression you can invoke with --config=clang
2323
build:clang --action_env=BAZEL_CXXOPTS="-std=c++17"
2424
build:clang --cxxopt=-std=c++17
25+
build:clang --cxxopt=-Wno-module-import-in-extern-c
2526
build:clang --cxxopt=-Wno-deprecated-declarations
2627
build:clang --cxxopt=-Wno-range-loop-analysis
2728
build:clang --cxxopt=-Wno-inconsistent-missing-override
@@ -54,6 +55,8 @@ build:g++ --cxxopt=-Wno-stringop-truncation
5455

5556
# C++17 is required to build ZetaSQL, hence `-cxxopt=-std=c++17`. On MacOS
5657
# `--host_cxxopt=-std=c++17` is also needed.
57-
build --cxxopt=-std=c++17 --host_cxxopt=-std=c++17
58-
run --cxxopt=-std=c++17 --host_cxxopt=-std=c++17
59-
test --cxxopt=-std=c++17 --host_cxxopt=-std=c++17
58+
# "--enable_bzlmod=false" is required because grpc does not support bzlmod yet.
59+
# "--java_runtime_version=remotejdk_11": to make the build more hermetic.
60+
build --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 --config=clang --enable_bzlmod=false --java_runtime_version=remotejdk_11
61+
run --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 --config=clang --enable_bzlmod=false --java_runtime_version=remotejdk_11
62+
test --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 --config=clang --enable_bzlmod=false --java_runtime_version=remotejdk_11

‎.bazelversion

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
6.5.0
1+
7.2.1

‎Dockerfile

Lines changed: 12 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -8,27 +8,24 @@ FROM ubuntu:18.04 as build
88
RUN apt-get update && apt-get -qq install -y default-jre default-jdk
99

1010
# Install prerequisites for bazel
11-
RUN apt-get update && apt-get -qq install curl tar build-essential wget \
12-
python python3 zip unzip
13-
14-
ENV BAZEL_VERSION=6.5.0
15-
16-
# Install bazel from source
17-
RUN mkdir -p bazel && \
18-
cd bazel && \
19-
wget https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION}/bazel-${BAZEL_VERSION}-dist.zip &&\
20-
unzip bazel-${BAZEL_VERSION}-dist.zip && \
21-
rm -rf bazel-${BAZEL_VERSION}-dist.zip
22-
ENV PATH=$PATH:/usr/bin:/usr/local/bin
23-
ENV EXTRA_BAZEL_ARGS="--tool_java_runtime_version=local_jdk"
24-
RUN cd bazel && bash ./compile.sh
25-
RUN cp /bazel/output/bazel /usr/local/bin
11+
RUN apt-get -qq install curl tar build-essential wget python python3 zip unzip
12+
13+
ENV BAZEL_VERSION=7.2.1
14+
15+
RUN apt install apt-transport-https curl gnupg -y
16+
RUN curl -fsSL https://bazel.build/bazel-release.pub.gpg | gpg --dearmor >bazel-archive-keyring.gpg
17+
RUN mv bazel-archive-keyring.gpg /usr/share/keyrings
18+
RUN echo "deb [arch=amd64 signed-by=/usr/share/keyrings/bazel-archive-keyring.gpg] https://storage.googleapis.com/bazel-apt stable jdk1.8" | tee /etc/apt/sources.list.d/bazel.list
19+
20+
RUN apt update && apt -qq install -y bazel-${BAZEL_VERSION}
21+
RUN ln -s /usr/bin/bazel-${BAZEL_VERSION} /usr/bin/bazel
2622

2723
RUN apt-get update && DEBIAN_FRONTEND="noninteractive" \
2824
TZ="America/Los_Angeles" apt-get install -y tzdata
2925

3026
# Unfortunately ZetaSQL has issues with clang (default bazel compiler), so
3127
# we install GCC. Also install make for rules_foreign_cc bazel rules.
28+
RUN apt-get -qq install -y software-properties-common make rename git ca-certificates libgnutls30
3229
RUN apt-get -qq install -y software-properties-common
3330
RUN add-apt-repository ppa:ubuntu-toolchain-r/test && \
3431
apt-get -qq update && \
@@ -50,8 +47,6 @@ RUN useradd -ms /bin/bash zetasql
5047
RUN chown -R zetasql:zetasql /zetasql
5148
USER zetasql
5249

53-
ENV BAZEL_ARGS="--config=g++"
54-
5550
ENV HOME=/home/zetasql
5651
RUN mkdir -p $HOME/bin
5752

‎README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ ZetaSQL is not itself a database or query engine. Instead,
77
it's intended to be used by multiple engines, to provide consistent
88
language and behavior (name resolution, type checking, implicit
99
casting, etc.). Specific query engines may implement a subset of features,
10-
giving errors for unuspported features.
10+
giving errors for unsupported features.
1111
ZetaSQL's compliance test suite can be used to validate query engine
1212
implementations are correct and consistent.
1313

‎WORKSPACE

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,14 @@ workspace(name = "com_google_zetasql")
3737

3838
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
3939

40+
http_archive(
41+
name = "toolchains_llvm",
42+
canonical_id = "1.0.0",
43+
sha256 = "e91c4361f99011a54814e1afbe5c436e0d329871146a3cd58c23a2b4afb50737",
44+
strip_prefix = "toolchains_llvm-1.0.0",
45+
url = "https://github.com/bazel-contrib/toolchains_llvm/releases/download/1.0.0/toolchains_llvm-1.0.0.tar.gz",
46+
)
47+
4048
http_archive(
4149
name = "rules_jvm_external",
4250
sha256 = "b17d7388feb9bfa7f2fa09031b32707df529f26c91ab9e5d909eb1676badd9a6",
@@ -47,9 +55,9 @@ http_archive(
4755
# gRPC Java
4856
http_archive(
4957
name = "io_grpc_grpc_java",
50-
url = "https://github.com/grpc/grpc-java/archive/v1.56.0.tar.gz",
51-
strip_prefix = "grpc-java-1.56.0",
52-
sha256 = "4af5ecbaed16455fcda9fdab36e131696f5092858dd130f026069fcf11817a21",
58+
sha256 = "301e0de87c7659cc790bd2a7265970a71632d55773128c98768385091c0a1a97",
59+
strip_prefix = "grpc-java-1.61.0",
60+
url = "https://github.com/grpc/grpc-java/archive/v1.61.0.zip",
5361
)
5462

5563
load("@rules_jvm_external//:repositories.bzl", "rules_jvm_external_deps")

‎bazel/grpc_cf_engine.patch

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
--- src/core/BUILD
2+
+++ src/core/BUILD
3+
@@ -2563,6 +2563,7 @@
4+
external_deps = [
5+
"absl/container:flat_hash_map",
6+
"absl/log:check",
7+
+ "absl/status",
8+
"absl/strings",
9+
"absl/strings:str_format",
10+
],

‎bazel/grpc_extra_deps.patch

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ index 4d8afa3131..b42224501f 100644
77
api_dependencies()
88

99
- go_rules_dependencies()
10-
- go_register_toolchains(version = "1.18")
10+
- go_register_toolchains(version = "1.20")
1111
- gazelle_dependencies()
1212
-
1313
# Pull-in the go 3rd party dependencies for protoc_gen_validate, which is

‎bazel/icu.BUILD

Lines changed: 13 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -35,17 +35,20 @@ filegroup(
3535
configure_make(
3636
name = "icu",
3737
configure_command = "source/configure",
38-
env = select({
39-
"@platforms//os:macos": {
40-
"AR": "",
41-
"CXXFLAGS": "-fPIC", # For JNI
42-
"CFLAGS": "-fPIC", # For JNI
43-
},
44-
"//conditions:default": {
45-
"CXXFLAGS": "-fPIC", # For JNI
46-
"CFLAGS": "-fPIC", # For JNI
47-
},
38+
args = select({
39+
# AR is overridden to be libtool by rules_foreign_cc. It does not support ar style arguments
40+
# like "r". We need to prevent the icu make rules from adding unsupported parameters by
41+
# forcing ARFLAGS to keep the rules_foreign_cc value in this parameter
42+
"@platforms//os:macos": [
43+
"ARFLAGS=\"-static -o\"",
44+
"MAKE=gnumake",
45+
],
46+
"//conditions:default": [],
4847
}),
48+
env = {
49+
"CXXFLAGS": "-fPIC", # For JNI
50+
"CFLAGS": "-fPIC", # For JNI
51+
},
4952
configure_options = [
5053
"--enable-option-checking",
5154
"--enable-static",

‎bazel/icu4c-64_2.patch

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
CXX = @CXX@
66
AR = @AR@
77
-ARFLAGS = @ARFLAGS@ r
8-
+ARFLAGS = @ARFLAGS@
8+
+ARFLAGS = @ARFLAGS@ -crs
99
RANLIB = @RANLIB@
1010
COMPILE_LINK_ENVVAR = @COMPILE_LINK_ENVVAR@
1111
UCLN_NO_AUTO_CLEANUP = @UCLN_NO_AUTO_CLEANUP@

0 commit comments

Comments
 (0)