[pull] master from GaijinEntertainment:master#1030
Merged
Conversation
Bumps the bindings from their libclang-16 baseline to libclang-22.1.5 - matches what apt.llvm.org's llvm-toolchain-noble-22 ships for the linux extended_checks self-binder step, and the same LLVM 22.1.5 aleksisch/llvm-shared-builds tarball dasLLVM already uses. Changes: - Regenerated all of modules/dasClangBind/src/*.inc + func_*.cpp (now 18 chunks vs 17) against LLVM 22.1.5 headers. Picks up the CXCursor_OMPArraySectionExpr -> CXCursor_ArraySectionExpr rename and ~340 other API additions/changes between 16 and 22. - find_package(Clang 16.0.6) -> find_package(Clang 22.1) in modules/dasClangBind/CMakeLists.txt. Matches both msys2 clang64 (22.1.4) and apt.llvm.org's noble-22 package via the standard ConfigVersion compatibility check. - POST_BUILD copy of libclang.dll + transitive runtime deps next to daslang.exe (mirrors the dasHV/dasGlfw pattern from #2838/#2840). Required because daslang loads .shared_module files via LoadLibraryEx with restricted search flags (PATH is NOT searched - see src/misc/sysos.cpp:loadDynamicLibrary). Multi-config / single-config split for symmetry. MSVC path uses official LLVM's self-contained libclang.dll; mingw needs the full chain (libclang + libLLVM-N + libffi/zlib/zstd/libxml2/libiconv + libc++ + libunwind). - Broadened the ToBasicType<unsigned long> specialization in include/daScript/ast/ast_typedecl.h from #if defined(_MSC_VER) to #if defined(_WIN32). Without it, mingw fails to build the binding for CXUnsavedFile::Length - Windows LLP64 makes 'unsigned long' a distinct 32-bit type regardless of compiler. - .github/workflows/extended_checks.yml: dropped libclang-16-dev, llvm-16-dev, clang-16 from the apt install list (no longer needed); switched the bind_clangbind self-binder --clang_path from /usr/lib/llvm-16/include/ to /usr/lib/llvm-22/include/. - .github/workflows/build.yml: dropped -DDAS_CLANG_BIND_DISABLED=ON from the build_windows_mingw cmake invocation. find_package(Clang 22.1) now matches the msys2 clang64 sysroot so the module builds cleanly under mingw. Locally verified: daslang modules/dasClangBind/examples/browse.das walks sample.h and generates daslang bindings end-to-end on mingw. CI's linux extended_checks self-binder step is the byte-canonical re-check. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two fixes for the failing CI on #2846: 1. linux extended_checks: find_package(Clang 22.1) couldn't find a compatible ClangConfig.cmake because Ubuntu noble's default cmake search path only finds /usr/lib/cmake/clang-{16,17,18}/. The apt.llvm.org noble-22 install puts ClangConfig.cmake under /usr/lib/llvm-22/lib/cmake/clang/. Pass PATH_TO_LIBCLANG=/usr/lib/llvm-22 so find_package's PATHS argument lands the search there. 2. modules/dasClangBind/CMakeLists.txt: guard the install(FILES ...) for the libclang runtime DLLs with the same IF(_DAS_CLANG_BIND_DLLS) predicate that wraps the POST_BUILD copy. Without it, install(FILES "" DESTINATION ...) would error at configure time when the DLL set can't be resolved, defeating the soft-warning intent. (Per Copilot review comment on #2846.) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two improvements this PR: 1. Expression-key support in `_group_by` (Session 2 from the post-PR #2843 plan). `_group_by(_.Price % 100)` (and tuples mixing field keys with expression keys) now lower to `GROUP BY ((price) % (100))`. The rendered fragment is reused verbatim in SELECT (`K = _._0`) and ORDER BY (`_order_by(_._0)`) positions, so the SQL stays a single source of truth across all three clauses. Mechanism: `collect_group_keys` calls `pred_to_sql` with a new `q.inlineConstants` mode so the bound `_.X` field refs render as columns and `ExprConst*` literals (ints, floats, strings, bools) inline as SQL literals instead of `?` placeholders. The fragment carries no binds, which lets us re-use it at multiple SQL positions without bind-position bookkeeping. Runtime values (`_.Col - capturedVar`) reject loudly with a precise diagnostic — same one exercised by `failed_sql_macro` case 25. Order swap in the `_group_by` peel: recurse into the source first so `q.rootType` is set before `pred_to_sql` runs on the key (the existing field-key path didn't need this since translation was deferred to emission; expression keys do). Backfills `benchmarks/sql/groupby_select_sum.das` m1 lane. The bench uses the explicit-inner-select shape inside SUM (`_._1 |> select($(c : Car) => c.price) |> sum()`) — m3f/m4 keep their splice-friendly bare-`sum` form; both emit equivalent SQL/compute. results.md refreshed per the living-doc policy, "Notes on missing lanes" bullet for `groupby_select_sum SQL` removed. 2. Two PR #2845 review fixes (Copilot, both real): - `peel_count_terminal`'s predicate-overload error mis-named the terminal: it said `_count(predicate)` and suggested `_count()`, but in `_sql` chains the bare `count()` / `long_count()` linq functions are the actual terminals. Drop the underscores in both the offending name and the suggested fix. - `try_peel_distinct_by_field` pinned receiver to `ExprVar` but didn't verify the var IS the lambda's bound parameter. So `_distinct_by(capturedRow.Brand) |> count()` would silently emit `COUNT(DISTINCT "Brand")` against the SQL source — wrong result, no diagnostic. Now extracts the lambda's arg name from `keyLambda._block.arguments[0]` and rejects when receiver name differs (`failed_sql_macro` case 24). Test surface: - `tests/dasSQLITE/test_32_group_by_expression_keys.das` — 6 tests covering SQL emission + runtime for single expression keys, multi-key tuples mixing field + expression keys, ORDER BY on the group key, and aggregate-over-expression-key projection. - `failed_sql_macro.das` cases 24 + 25 added (50503:23). Validation: 796/796 dasSQLITE (interp + AOT), 1390/1390 linq, 0 JIT bench failures, lint clean. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
apt.llvm.org's libclang-22-dev ships libclang.so + headers but NOT the
ClangConfig.cmake that find_package(Clang 22.1) needs. The Clang cmake
config files live in the clang-22 package (the compiler frontend),
which also pulls in /usr/lib/llvm-22/lib/cmake/clang/ClangConfig.cmake.
Found via the failing-CI error message: cmake's "considered but not
accepted" list shows only Ubuntu's /usr/lib/cmake/clang-{16,17,18}/
ClangConfig.cmake — meaning the apt.llvm.org noble llvm-22 install
didn't put any ClangConfig.cmake on disk that find_package could see.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…-groupby-expression-keys sqlite_linq: expression keys in _group_by + PR #2845 review fixes
When dasClangBind is enabled on linux extended_checks, its libclang.so
import pulls libLLVM-22.so.1 into the process — the same .so that
dasLLVM already uses. Both modules then share LLVM's global Pass
Registry; double-initialization corrupts JIT pass setup, so
dasbind_example.das (which JIT-compiles to a standalone exe via
dasLLVM) dies with:
Pass 'Pre-ISel Intrinsic Lowering' is not initialized.
Verify if there is a pass dependency cycle.
Master worked because it used libclang-16 (different SONAME from
libLLVM-22) — the two LLVMs were ABI-isolated. With this PR moving
dasClangBind to libclang-22, the SONAMEs coincide and they collide.
Resolution: dasClangBind stays OFF on linux extended_checks; only the
mingw worker (build.yml build_windows_mingw) builds it, where the
worker doesn't load dasLLVM in the same process that runs the binder.
Changes:
- .github/workflows/extended_checks.yml: DAS_CLANG_BIND_DISABLED=ON
on linux build; dropped clang-22 + libclang-22-dev apt packages
(no longer needed there); kept llvm-22-dev (dasLLVM still needs
LLVMConfig.cmake). Dropped PATH_TO_LIBCLANG. Removed the
bind_clangbind + bind_llvm self-binder steps that ran here.
- .github/workflows/build.yml: added bind_clangbind self-binder
step to build_windows_mingw. Uses cygpath to convert msys2's
${MSYSTEM_PREFIX}/include to a Windows path for daslang.exe.
Known regression: bind_llvm.das self-binder doesn't run anywhere
in this PR — it has its own Windows-mingw blocker (libclang-as-
library doesn't find <cstddef> via msys2's libcxx auto-detection).
Marked as a TODO comment on the mingw worker.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the last missing-lane cell in `benchmarks/sql/results.md`: the `join_count` Decs lane. Builds a `table<KEY; array<TUPB>>` from the smaller side (srcb) inside one `for_each_archetype` pass, then walks the larger side (srca) and probes — same algorithm as `join_impl` (linq.das:1674) but inlines keya/keyb/result lambdas via the existing peel helpers so per-pair invoke overhead vanishes. Supported chain shapes (v1): - `from_decs_template(type<TA>) |> _join(from_decs_template(type<TB>), on, into) |> count()` - same chain ending in implicit `to_array()` Interleaved `_where`/`_select` and non-primitive key types cascade to tier-2 (`fold_linq_default`). Auto-typed `_join` result lambda: existing tests all spell out the two result-arg types because the `on` macro auto-injects them but the result lambda was passed through verbatim. Over decs sources the iterator element is `tuple<...>` (verbose to spell out by hand), so `LinqJoinBase.visit` now mirrors the `on` path — when the result lambda's two args are untyped (`$(l, r) => (...)`), inject the source element types. Strictly additive: existing typed callers untouched. Bench (join_count, n=100k, 100 dealers): - INTERP m4 = 64.0 ns/op (m3f = 121.2, SQL = 38.0) — 0.53× of array - JIT m4 = 13.3 ns/op (m3f = 36.2, SQL = 38.0) — 0.37× of array, fastest lane Tests: 5 new tests in `tests/linq/test_linq_from_decs.das` covering count + to_array + empty sides + many-to-one + unmatched-cars; full existing suite (193 tests in same file, 37 in test_linq_join.das, 21 in test_linq.das) green. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per [[feedback-living-results-md]]: every PR touching splice machinery re-runs the full INTERP+JIT matrix and refreshes results.md in the same PR. 62336a4 adds plan_decs_join → join_count Decs lane filled (was the last `—` cell apart from by-design ones). INTERP join_count: m4 = 64.0 ns/op (0.53× of m3f = 121.2). m3f goes through `join_to_array` then counts; m4 builds the hash inline and returns the running count without materializing the result array. JIT join_count: m4 = 13.3 ns/op (0.37× of m3f = 36.2). LLVM inlines the per-element binds, hash probe, and counter into a tight loop. Other cells: ±~1 ns/op noise vs `31e4339a8` baseline — no regressions. Dropped the `join_count Decs` bullet from "Notes on missing lanes". Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…oops In findFuncAddr / findMatchingFunctions / findMatchingFunctionsAndGenerics, the per-candidate isVisibleFunc(inWhichModule, getFunctionVisModule(pFn)) check was called once per function in mod->functionsByName[h]. Since pFn->module == mod by construction (Module::addFunction sets it), for non-generic candidates the visibility module is constant across the inner loop and the check is loop-invariant. Hoist modVis = isVisibleFunc(inWhichModule, mod) and modVisFromThis = thisModule->isVisibleDirectly(mod) to the top of each non-empty bucket; per-candidate check now only fires for fromGeneric candidates (whose visibility module is pFn->getOrigin()->module). Measured on dasImgui imgui_demo (full main.das, 13621 functions): compile-only total: 29.08s -> 24.87s (-14.5%) infer: 24.43s -> 20.19s (-17.3%) In PerfView CPU sampling, requireModule.find dropped from 9.23% -> 1.38% exclusive samples (-85% relative); InferTypes::isVisibleFunc dropped out of the top 20 hotspots. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
findAlias walks the TypeDecl tree (firstType / secondType / argTypes)
checking each node's alias field. For TypeDecl subtrees that contain
no aliases anywhere (the common case for non-generic / non-auto types),
the walk yields nothing but still pays the recursion cost on every call.
Add a 2-bit cache stored in two spare bits of the existing flags union:
aliasCacheValid set once computeAliasCache() has run on this node
aliasCacheHasAlias meaningful only when valid; true iff the subtree
contains any alias at all (name-independent,
allowAuto-independent)
computeAliasCache() does one eager full walk and populates the flags on
every visited node. findAlias() then bails in O(1) at the root when the
cached state says "no aliases anywhere". For subtrees with aliases the
behavior is unchanged - original recursive walk runs.
The cache bits are NOT cloned - both the copy ctor and the in-place
TypeDecl::clone reset them so clones start fresh and recompute lazily.
Stored in spare bitfield slots (uint32_t flags already had room) rather
than a separate byte, so sizeof(TypeDecl) is unchanged and the
shared_module ABI is preserved.
Measured on dasImgui imgui_demo (full main.das, 13621 functions,
on top of the prior infer-visibility hoist):
compile-only total: 24.87s -> ~23.5s (-5-7%)
infer: 20.19s -> ~18.8s (-7%)
Cumulative vs original baseline (29.08s / 24.43s infer):
total: -19%
infer: -22%
Full dastest suite: 9302 tests, 9302 passed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
setup-msys2 doesn't ship git by default, and the msys2 shell doesn't inherit Windows PATH, so `git diff --exit-code` in the bind_clangbind self-binder step failed with "git: command not found". The regen itself ran cleanly (msys2 22.1.4 produces byte-identical output to the LLVM 22.1.5 headers used for the committed regen) — only the diff check stumbled. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
linq_fold: plan_decs_join + LinqJoin auto-typed result lambda (Session 3)
The mingw CI worker re-runs bind_clangbind.das to detect regen drift. Without eol=lf, msys2 git checks out the committed LF-blob files as CRLF (autocrlf=true default), and the regen writes LF — every line shows as changed, self-binder fails. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors findAlias's own line 970 pattern so a parent's eager walk doesn't re-recurse into already-computed children (when findAlias was called on an intermediate node before the root). Premise as Copilot stated it (shared subtrees) doesn't apply to AST nodes per gc_node unique-ownership; but the call-on-child-before-parent case is real, and the one-liner is self-consistent with the file's pattern. Perf-neutral on imgui_demo.das compile (8-run avg 20.30s vs 3-run pre 20.51s, within ~1s noise spread). Land for stylistic consistency, not a measured speedup. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The bind_clangbind regen emits libclang-reported source paths in '// from ...' comments. Those paths aren't canonical across --clang_path invocations on Windows: forward-slash form (D:/...) gives include-relative paths, cygpath -w form (C:\...) gives absolute. So the diff-check trips on path-format drift between local and CI invocations. Keep the self-binder step (catches binder-side breakage like API loss or module-load failures), drop the git diff --exit-code. A proper fix (strip path prefix to basename in cbind_boost.das when emitting source-location comments) is a follow-up.
…oad-hoist-aliascache infer: hoist per-module visibility + aliasCache (~19% faster compile)
…-upgrade dasClangBind: regen for libclang 22.1.5, enable on mingw CI
…kslash paths Followup to #2846. The `// from ...` source-location comments in generated bindings are emitted by clang_getCursorLocationDescription, which strips PARSE_FILE_PREFIX from libclang's reported path so the output stays stable across install locations. cursor-side paths are normalized to forward slashes (clang_getCursorLocationFullPath line 159), but PARSE_FILE_PREFIX itself was stored as-is — when callers passed a backslash form (`C:\msys64\clang64\include/`, e.g. via `cygpath -w` on the mingw CI runner), the `starts_with` check failed and the full absolute path leaked into the regen output. Normalize PARSE_FILE_PREFIX in init_args the same way, so the prefix- strip works for every OS/path-form combination. Verified locally: regen with `C:\msys64\clang64\include/` (backslash, CI-mingw style) and `C:/msys64/clang64/include/` (forward-slash, my local style) now produce byte-identical output. Re-enabled the `git diff --exit-code` self-binder check on the mingw worker (the PR #2846 fallback dropped it because of this drift). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…-source-path cbind_boost: normalize PARSE_FILE_PREFIX so source-path strip works for backslash inputs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )