[pull] master from GaijinEntertainment:master#1020
Merged
Conversation
New `plan_decs_distinct` planner mirrors `plan_distinct`, inserted in the cascade BEFORE `plan_distinct` (same rule as 5d/5e — decs bridge match is stricter). Streaming dedup via `var inscope seen : table<typedecl(_::unique_key (...))>` hoisted above `for_each_archetype` so the seen-table persists across archetypes. Two emit shapes: - **Buffer-required** (`to_array` default, optionally `take(N)`): per-element splice computes the key, `key_exists` check, on miss does `taken++ → seen|>insert → buf|>push_clone`. When `take(N)` is present, outer iteration switches to `for_each_archetype_find` and the take-cap `return true` propagates across archetypes — true streaming early-exit. Take-limit bound to a `let` at outer scope so a side-effecting `take(arg)` evaluates exactly once. - **Buffer-elided** (`count`/`long_count` → `length(seen)` / `int64(length(seen))`; `sum` folds inline at fresh-key site via `acc += <projection>`): no buffer allocation, only the seen-table is materialized. count + take both present cascades to tier 2 (matches array- side `plan_distinct`). Side-effecting `_select(proj)` upstream binds once per element to a fresh `decs_pv` local — key + buf push (or sum fold) share the bind, matching array- side single-eval per source element. `distinct_by(key)` wraps the key block in `invoke(<keyBlock>, <projection-or-tup>)` and the seen-table's value type is derived via `_::unique_key(invoke(<keyBlock>, default<elemT>))`. Copilot review fixes: - **plan_distinct predicate-drop bug (pre-existing).** Adds the symmetric arg-count guard on the terminator pop. `plan_distinct` was popping `count`/`long_count`/`sum` unconditionally — so `each(arr).distinct().count(predicate)` silently dropped the predicate and emitted `length(seen)` instead of filtering. Repro returned 5 instead of 3 for `[1,1,2,2,3,3,4,4,5,5].distinct().count(_>2)`. Fix mirrors the guard already in `plan_decs_distinct`: pop only when `length(args)==1`; otherwise cascade to tier-2 which honors the predicate via `count(seq, pred)`. +4 parity tests (2 array, 2 decs) that fail pre-fix and pass post-fix. - Comment wording in `plan_decs_distinct` no longer references the non-existent `sum(src, sel)` overload. Benchmarks (100K rows, INTERP): | benchmark | m1 sql | m3 | m3f | m4 (was) | m4 (now) | win | |---|---:|---:|---:|---:|---:|---:| | distinct_count | 41 | 44 | 15 | 97 | **28** | 3.5× | | distinct_take | 0 | 30 | 0 | 34 | **0** | matches m3f | Coverage: `_select(_.brand).distinct()` + to_array / count / long_count / sum / take(N) / take(0) / take(N>num_distinct), `_where(_.id<8)._select(_.brand). distinct()`, `_distinct_by(_.brand)` + to_array / count / take(N), empty-decs distinct yields empty, side-effecting take(N) arg evaluates exactly once. `count(pred)`/`long_count(pred)` after distinct (both array + decs sources) must honor the predicate. AST shape gates for: distinct+count (no to_sequence, no decs_buf, single key_exists, plain for_each_archetype), distinct+take (for_each_archetype_find + decs_buf + decs_seen + decs_taken counters), distinct_by+to_array (unique_key wrapping the key invocation), distinct+sum (no decs_buf, decs_acc declared+folded). +19 tests (122 → 141 in test_linq_from_decs; +2 in test_linq_fold). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ecs-unroll-slice5c linq_fold: plan_decs_unroll Slice 5c — distinct/distinct_by on decs
Adds unambiguous bulk-append names mirroring push/push_clone/emplace,
backfills the existing bulk forms with reserve() so they stop paying
N reallocations, and ships a lint rule (PERF022) that flags the
hand-rolled for-loop element-at-a-time shape and recommends the
new _from name.
The single names push/push_clone are overloaded between single-
element and bulk forms - B |> push(arr) silently flips meaning
depending on whether B is array of T or array of T-array. The _from
suffix names the bulk intent unambiguously.
PERF022 details:
- Fires on for (s in A) { B |> push(s) } and push_clone, all
three pipe/dot/plain-call shapes (compiler folds to the same
ExprCall).
- Body must be exactly one statement; if-guarded, multi-stmt,
transform-then-push, multi-source, and nested for-loop shapes
do NOT match (no clean bulk migration).
- Source must be array of T or C-array - range/string/iterator/
generator sources have no bulk overload.
- Destination must be array of T - generic auto destinations
(e.g. [expect_any_vector] clone(args, ...) in builtin.das)
are skipped because push_from doesn't exist for dasvector.
- Warns at the for-loop's .at (not the inner push's) so the
same-location dedup in perf_warning doesn't collide with
PERF006 which fires at the push.
- emplace is out of scope: for-iter-var is const-ref, but
emplace requires var-ref, so the hand-rolled shape doesn't
compile in the first place.
builtin.das additions (11 new overloads): push_from x4,
push_clone_from x4 (fills the array of numT source gap that
push_clone didn't have), emplace_from x3 (gap-fill too). Each calls
reserve(long_length(Arr) + long_length(varr)) before iterating.
Existing bulk push/push_clone/emplace overloads also get the
reserve() backfill so the old name stays efficient for callers
that don't migrate. array-of-arrays single-element overloads
(push(array of T-array, T-array)) are out of scope - those push a
sub-array as ONE element, not bulk.
Codebase sweep - daslib + utils:
daslib/{algorithm,daspkg,fio,linq,linq_fold,lint,perf_lint,regex,
style_lint}, utils/{daspkg/index,detect-dupe/main,detect-dupe/
test_detect_dupe,mcp/tools/common,mcp/tools/cpp_common,mcp/
tools/cpp_grep_usage} all migrated. tests/, modules/, examples/,
tutorials/ left as-is - CI lint gate catches new code organically.
Tests:
- utils/lint/tests/perf022_for_push_bulk.das - 6 positives
(push/push_clone x pipe/dot/plain), 7 negatives (if-guard,
multi-stmt, transform, multi-source, nested-for, push-other,
baseline push_from).
- tests/language/array.das gains testMultiPushFrom covering each
new overload (array of T + C-array sources, "Arr already has
elements" reserve case).
- utils/lint/tests/perf006_push_without_reserve.das: nolint
PERF022 on two cases where the new rule co-fires.
Docs:
- doc/source/reference/language/lint.rst - PERF022 section.
- tutorials/language/06_arrays.das + .rst - "Bulk append" example.
- doc/reflections/das2rst.das Containers regex picks up the
three new public names.
- doc/source/stdlib/handmade/function-builtin-{push,push_clone,
emplace}_from-*.rst - 1-line descriptions.
- CLAUDE.md table + skills/perf_lint.md table - PERF022 row.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reuse them only on pull request. Rebuild clean repo on merge to master.
Fix linter errors
Now we can return bool from main in jit exe.
Now WASM generation supported for daScript. It uses LLVM infrastructure again.
…pilation-llvm cross compilation llvm
…h-from PERF022 + push_from / push_clone_from / emplace_from bulk overloads
…builds ci: cache build results on pull request
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )