Skip to content

Commit f77a072

Browse files
borisbatclaude
andcommitted
mouse-data/docs: 16 new + 1 updated card from linq_fold + Phase 2A session
Cards added in the course of the linq_fold splice rewrite + PR GaijinEntertainment#2691 (has_sideeffects + counter-lane elision). Topics: linq_fold / macro-emission patterns: - daslang-generic-instance-detect-via-fromgeneric — func.fromGeneric is the canonical "which generic was this instantiated from?" link; func.name on typed instances is mangled. - daslib-macro-boost-has-sideeffects-predicate — new public predicate, full classification table, known limitations, test plumbing. - qmacro-invoke-source-bind-typedecl-modifier-iter-vs-array — typedecl block-param const/ref handling differs between iterator and array sources; the two diagnostic error messages tell you which branch you picked wrong. - qmacro-gensym-per-callsite-via-lineinfo — backtick-prefixed names + line+column suffix, force_at / force_generated / can_shadow. - my-fold-macro-emits-a-loop-with-for-it-in-source-... (UPDATED) — peel_each pattern corrected for generic-instance detection + positive array gate + block-param typedecl handling. LINQ semantics: - are-there-parity-tests-in-tests-linq-that-compare-fold-output-to-... - which-typedecl-predicates-identify-types-where-length-expr-is-... - why-does-each-arr-fail-with-unsafe-when-not-source-of-for-loop-... - what-s-the-right-sqlite-linq-chain-form-for-aggregates-sum-min-max-... - my-macro-substitutes-it-for-a-projection-expression-via-template-... - when-a-call-macro-needs-to-pick-copy-vs-move-init-for-a-projection-... - where-does-nolint-rule-go-when-a-lint-warning-is-emitted-from-inside-... Tooling / ops: - how-do-i-run-dastest-in-benchmark-only-mode-and-what-s-the-command-... - cpp-profiler-macos-samply-instruments.md - what-s-the-end-to-end-checklist-for-adding-a-new-daslib-das-module-... - how-do-i-call-a-dasimgui-or-any-managed-c-method-on-a-struct-field-... Updated: - why-does-my-dastest-integration-test-hang-at-readiness-gate-failed-... — original card pointed at a require-order red herring; real cause was ref_time_ticks() returning ns on POSIX while wait_until_ready's deadline math assumed μs. Fix landed in PR GaijinEntertainment#2685. No code changes — docs only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent a39e155 commit f77a072

17 files changed

Lines changed: 769 additions & 36 deletions

File tree

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
---
2+
slug: are-there-parity-tests-in-tests-linq-that-compare-fold-output-to-the-underlying-linq-operators
3+
title: Are there parity tests in tests/linq/ that compare `_fold` output to the underlying linq operators?
4+
created: 2026-05-16
5+
last_verified: 2026-05-16
6+
links: []
7+
---
8+
9+
There's no file named "parity" or similar. The parity-test surface IS the broader [tests/linq/](tests/linq/) directory:
10+
11+
- `test_linq.das` — comprehension basics
12+
- `test_linq_aggregation.das` — count/sum/min/max/avg
13+
- `test_linq_querying.das` — any/all/contains
14+
- `test_linq_transform.das` — select / select_many / zip
15+
- `test_linq_sorting.das` — order / reverse
16+
- `test_linq_group_by.das` — group_by / having
17+
- `test_linq_join.das` — joins
18+
- `test_linq_partition.das` — take / skip / chunk / take_while / skip_while
19+
- `test_linq_set.das` — distinct / union / except / intersect / unique
20+
- `test_linq_element.das` — first / last / single / element_at
21+
- `test_linq_concat.das` — concat / prepend / append
22+
- `test_linq_generation.das` — range / repeat
23+
- `test_linq_bugs.das` — regressions
24+
25+
Each file uses `[test]` functions with `t |> run("name") @(t) { ... }` blocks asserting `t |> equal(actual, expected)`. These exercise the regular linq operators (`where_`, `select`, `count`, ...) directly — they're not split into "fold-on" vs "fold-off" variants.
26+
27+
Dedicated `_fold` tests live in `test_linq_fold.das` (functional output) and `test_linq_fold_ast.das` (AST-shape verification — pattern-matches the macro expansion). These DO compare `_fold(chain)` output against the plain `chain` output for the shapes the macro recognizes.
28+
29+
When the user says "parity tests" in linq context, treat the full `test_linq_*.das` suite as the operator-coverage map. Phase-2+ benchmark/splice PRs should add a `benchmarks/sql/` entry for each shape exercised here that isn't already covered (tracked as a checklist in `benchmarks/sql/LINQ.md`).
30+
31+
## Questions
32+
- Are there parity tests in tests/linq/ that compare `_fold` output to the underlying linq operators?
33+
- What's the "parity test" coverage surface for linq?
34+
- Where are tests for linq operators?
35+
36+
## Questions
37+
- Are there parity tests in tests/linq/ that compare `_fold` output to the underlying linq operators?
Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
---
2+
slug: cpp-profiler-macos-samply-instruments
3+
title: What C++ sampling profiler should I use on macOS for daslang (and how do I run it)?
4+
created: 2026-05-16
5+
last_verified: 2026-05-16
6+
links: []
7+
---
8+
9+
# C++ sampling profiler on macOS (Apple Silicon)
10+
11+
VS Code has **no first-class C++ profiler integration on macOS** — the "Performance Profiler" / similar extensions wrap Linux `perf` and don't help here. Skip them. Run a sampler from the integrated terminal and view results in browser/Instruments.
12+
13+
## samply (default choice)
14+
15+
Rust-built, Firefox-Profiler frontend, zero config.
16+
17+
```bash
18+
cargo install samply
19+
samply record ./build/daslang script.das
20+
```
21+
22+
- Opens flamegraph in browser automatically.
23+
- Symbolicates Mach-O cleanly if you build `-DCMAKE_BUILD_TYPE=RelWithDebInfo` (do NOT use plain `Release` — symbols are stripped).
24+
- Works without sudo on Apple Silicon.
25+
- Good for "where does the CPU go" questions.
26+
27+
## Xcode Instruments — Time Profiler (second opinion)
28+
29+
Native macOS sampler, kernel-assisted, best symbolication on Apple Silicon. Use when samply's view is ambiguous or you want call-tree + timeline together.
30+
31+
```bash
32+
xcrun xctrace record --template 'Time Profiler' --launch -- ./build/daslang script.das
33+
```
34+
35+
Then open the resulting `.trace` bundle (Instruments launches). UI is outside VS Code.
36+
37+
## daslang-specific recipe
38+
39+
Pair the sampler with the per-module compile-time logging (`-log-compile-time` CLI flag, added on branch `bbatkin/log-compile-time-cli`):
40+
41+
```bash
42+
cmake --build build --config RelWithDebInfo -j 64
43+
samply record ./build/daslang -log-compile-time path/to/script.das
44+
```
45+
46+
- `-log-compile-time` tells you which module is slow.
47+
- Sampling tells you which function inside that module is hot.
48+
- Together they narrow "compile is slow" to a specific phase + symbol.
49+
50+
## What NOT to use
51+
52+
- `perf` — Linux only, doesn't exist on Darwin.
53+
- Intel VTune — x86-mostly, ignore on Apple Silicon.
54+
- `gprof` — instrumenting, not sampling; ancient.
55+
- VS Code C++ profiler extensions — see above, all are Linux/perf wrappers or toys.
56+
- `hyperfine` / `poop` — benchmarking (whole-program timing), not profiling (per-function hotspots). Different question.
57+
58+
## Build flag reminder
59+
60+
Both samply and Instruments need symbols. The two viable build types on this repo:
61+
62+
- `RelWithDebInfo` — fast code + symbols. Use this for profiling.
63+
- `Debug` — slow code; profile reflects debug overhead, not real hotspots. Avoid.
64+
65+
Plain `Release` strips symbols and you'll get `???` everywhere in the flamegraph.
66+
67+
## Questions
68+
- What C++ sampling profiler should I use on macOS for daslang (and how do I run it)?
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
---
2+
slug: daslang-generic-instance-detect-via-fromgeneric
3+
title: How do I detect that an ExprCall is to a daslang generic (e.g. each, length, find) when func.name is the mangled instance name and not the original generic's name?
4+
created: 2026-05-16
5+
last_verified: 2026-05-16
6+
links: []
7+
---
8+
9+
When a daslang generic function (`def each(a : array<auto(TT)>) : iterator<TT&>`, `def length(a : auto | #) : int`, etc.) is resolved against a concrete type at infer time, the resolved `Function?` instance gets a **mangled name** like `` `builtin`each`30908`12345 ``. Macro code that compares `call.func.name == "each"` will never match a typed instance.
10+
11+
The original generic's identity lives in `call.func.fromGeneric`:
12+
13+
```das
14+
[macro_function]
15+
def private is_each_call(call : ExprCall?) : bool {
16+
if (call == null || call.func == null) return false
17+
return (call.func.name == "each"
18+
|| (call.func.fromGeneric != null && call.func.fromGeneric.name == "each"))
19+
}
20+
```
21+
22+
The `name == "each"` branch covers the unusual case where you see the call before the typer has specialized it (e.g. inside a custom call_macro that runs early). The `fromGeneric.name` branch is the normal case for any post-infer chain.
23+
24+
**When this bites:** writing a `[macro_function]` that pattern-matches on a stdlib helper by name — `each`, `length`, `key_exists`, `find`, `set_insert`, all the generic `to_array`/`to_table` variants. Without the `fromGeneric` check, every typed chain silently falls through your match and your macro behaves as if the helper wasn't there.
25+
26+
**Generalizes beyond function calls:** same applies to method overload resolution. `call.func.fromGeneric` is the canonical "which generic was this instantiated from?" link. There's no `originalName` field — the chain is `func → func.fromGeneric → fromGeneric.name`.
27+
28+
**Doesn't apply to:** C++ builtins from `addExtern<>` (no fromGeneric, the `func.name` is the bound name directly). Builtins also have `func.flags.builtIn = true` if you need to distinguish.
29+
30+
See [[my-fold-macro-emits-a-loop-with-for-it-in-source-acc-reserve-length-source-but-the-reserve-doesn-t-fire-when-the-chain-starts-wi]] for the concrete case where this broke `peel_each` in `daslib/linq_fold.das`.
31+
32+
## Questions
33+
- How do I detect that an ExprCall is to a daslang generic (e.g. each, length, find) when func.name is the mangled instance name and not the original generic's name?
Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
---
2+
slug: daslib-macro-boost-has-sideeffects-predicate
3+
title: Is there a conservative side-effect detector for Expression nodes in daslib macro_boost — something I can call from a call_macro to know if it's safe to elide an evaluation at macro time?
4+
created: 2026-05-16
5+
last_verified: 2026-05-16
6+
links: []
7+
---
8+
9+
Yes — `has_sideeffects(expr : Expression?) : bool` in `daslib/macro_boost` (added in PR #2691, follow-up to Phase 2A loop planner). Returns `true` if the expression has or **might have** side effects; `false` ONLY when provably pure.
10+
11+
```das
12+
require daslib/macro_boost public
13+
14+
if (has_sideeffects(projection)) {
15+
// Emit the bind — projection must run for its side effects.
16+
sideEffectStmts |> push <| qmacro_expr() {
17+
var $i(finalBindName) = $e(projection)
18+
}
19+
} else {
20+
// Skip the bind — pure projection, no observable effect.
21+
}
22+
```
23+
24+
**Conservative — false is a promise:**
25+
26+
- SAFE leaves: `ExprVar`, all `ExprConst*`, `ExprAddr`, `ExprTypeInfo/Decl/Tag`.
27+
- SAFE via recursion: `ExprField`, `ExprSafeField`, `ExprSwizzle`, `ExprRef2Value/Ptr`, `ExprPtr2Ref`, `ExprIs`, `ExprAsVariant`, `ExprIsVariant`, `ExprSafeAsVariant`, `ExprCast`, `ExprNullCoalescing`, `ExprStringBuilder` (string heap is no-op per compiler), `ExprKeyExists` (pure container read).
28+
- `ExprAt`: safe when `subexpr._type` is NOT `isGoodTableType` (tables auto-insert default on missing key — a write). `ExprSafeAt` (`?[...]`) always safe.
29+
- `ExprOp1/Op2/Op3`: op-name allowlist for pure ops on workhorse types — `+ - * == != < <= > >= & | ^ << >> && || ?:` (Op2), `- ! ~ +` (Op1). Falls back to `func.flags.builtIn && !knownSideEffects && !unsafeOperation`. `/` and `%` BLACKLISTED (div-by-zero panic).
30+
- `ExprCall`/`ExprCallFunc`: allowed when `func.flags.builtIn && !knownSideEffects && !unsafeOperation`, then recurse args.
31+
- Everything else (including `ExprNew`, all `ExprMake*`, user-defined calls, `ExprInvoke`, `ExprYield`, statement-context exprs): UNSAFE.
32+
33+
**Known limitations / when it returns conservative-unsafe:**
34+
35+
- daslang-generic helpers like `length(arr)` and `key_exists(tab, k)` — the resolved `func.name` is the mangled instance, and the typer hasn't always reached the `flags.builtIn=true` C++ overload before the call_macro fires. They show up as user-call shapes and get rejected. Workaround: don't rely on this for length/key_exists in projections (they appear in `has_sideeffects` tests as `target_generic_length_unresolved` / `target_key_exists_unresolved` returning `true`).
36+
- User-defined pure helpers — there's no `[no_side_effects]` annotation yet. The compiler's `expr.flags.noSideEffects` fast path catches some cases (set during infer), but anything the typer didn't tag falls through to UNSAFE.
37+
38+
**Tests:** `tests/macro_boost/test_has_sideeffects.das` has 24 cases (17 safe + 5 unsafe + 2 conservative-unsafe) wired via a `_test_has_sideeffects` probe `call_macro` ([`tests/macro_boost/_has_sideeffects_probe.das`](../../tests/macro_boost/_has_sideeffects_probe.das)) that runs the predicate at macro time and emits `ExprConstBool` of the result. Use the same probe pattern when testing any new predicate that needs to run at macro time but be exercised via runtime tests.
39+
40+
**Real use:** `daslib/linq_fold.das` `plan_loop_or_count` uses it for three optimizations: discardable `var vfinal =` bind elision, count→length shortcut gate (whole loop elided when no filter + all projections pure + source has length), and tracking `allProjectionsPure` across chained selects. select_count benchmark went from 2 → 0 ns/op.
41+
42+
## Questions
43+
- Is there a conservative side-effect detector for Expression nodes in daslib macro_boost — something I can call from a call_macro to know if it's safe to elide an evaluation at macro time?
Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
---
2+
slug: how-do-i-call-a-dasimgui-or-any-managed-c-method-on-a-struct-field-that-s-bound-as-a-raw-pointer-e-g-addfontfromfilettf-on-getio
3+
title: How do I call a dasImgui (or any managed C++) method on a struct field that's bound as a raw pointer — e.g. AddFontFromFileTTF on GetIO().Fonts?
4+
created: 2026-05-16
5+
last_verified: 2026-05-16
6+
links: []
7+
---
8+
9+
## TL;DR
10+
11+
When a managed struct's field is bound as a pointer (`T?`) and the method on that pointed-to struct expects the value by-ref (`T implicit`), you must explicitly **dereference**. Plain `field |> method(...)` errors with mismatched types.
12+
13+
## The error you'll hit
14+
15+
```
16+
error[30341]: no matching functions or generics: AddFontFromFileTTF(imgui::ImFontAtlas?&, string const&, ...)
17+
candidate function: ImFontAtlas implicit ...
18+
invalid argument 'self' (0). expecting 'imgui::ImFontAtlas implicit', passing 'imgui::ImFontAtlas?&'
19+
```
20+
21+
The `?` is the giveaway — `GetIO().Fonts` is `ImFontAtlas?` (raw pointer; field bound via `addField<DAS_BIND_MANAGED_FIELD(Fonts)>` against C++ `ImFontAtlas* Fonts`), but the method binding `das_call_member< ImFont * (ImFontAtlas::*)(...) >` takes the receiver by-value/ref.
22+
23+
## The fix
24+
25+
Bind a local ref through `unsafe(*ptr)`, then call as usual:
26+
27+
```daslang
28+
var atlas & = unsafe(*GetIO().Fonts)
29+
let f = atlas |> AddFontFromFileTTF(ttf, 14.0f, null, null)
30+
```
31+
32+
Equivalent inline form:
33+
34+
```daslang
35+
unsafe(*GetIO().Fonts) |> AddFontFromFileTTF(ttf, 14.0f, null, null)
36+
```
37+
38+
## Why each part
39+
40+
- **`*ptr`** is daslang's pointer-deref syntax (see `daslib/if_not_null.rst`: *"a dereferenced call: ``if (ptr != null) { call(*ptr, args) }``"*). The alternative `deref(ptr)` exists too but is rarer in modules; `*` is the idiom.
41+
- **`unsafe(...)`** is required because dereferencing a raw `T?` is unsafe (no null check, no lifetime guarantee).
42+
- **`var atlas &`** binds a local *reference* — without `&` you'd be copying the whole `ImFontAtlas` struct into a stack temporary, which (a) wastes memory and (b) means any mutation the method does (font atlas builds, glyph rasterization) hits the copy and is lost.
43+
- **The pipe `|>` works fine on the local ref**`atlas |> method(x, y)` desugars to `method(atlas, x, y)` and the `implicit` first-param accepts the ref directly.
44+
45+
## Why NOT the other shapes
46+
47+
- `GetIO().Fonts.AddFontFromFileTTF(...)``.method()` sugar is sugar for `method(self, ...)` only when `self` is a struct value. CLAUDE.md explicitly: *"Does NOT work on: primitives, tuples/arrays, and lambda typedefs"* — and (this case) raw pointers. Field *access* on a pointer auto-derefs (`GetIO().Fonts.TexID` works); method dispatch does not.
48+
- `GetIO().Fonts->AddFontFromFileTTF(...)``->` is for class instances (smart_ptr / class types), not raw C-struct pointers from `ManagedStructureAnnotation`.
49+
- `deref(GetIO().Fonts) |> AddFontFromFileTTF(...)` — works but the pipe gets a temporary value not a ref; mutations on the receiver disappear. Use `var x & = unsafe(*p)` instead.
50+
51+
## When this comes up
52+
53+
Anywhere a C++ binding exposes a struct field as `T*` (typical for "owns-an-atlas" or "owns-a-context" patterns):
54+
- `ImGuiIO::Fonts``ImFontAtlas?`
55+
- `ImDrawData::CmdLists` → indirection on lists
56+
- anything bound via raw `addField<DAS_BIND_MANAGED_FIELD(X)>` where the C++ type is `Foo*`
57+
58+
If the C++ field were a value (`ImFontAtlas Fonts;` instead of `ImFontAtlas* Fonts;`), it'd bind as the struct directly and the pipe would just work.
59+
60+
## Related
61+
62+
- [[dasimgui-new-state-struct-widget-auto-emit-just-works]] — different topic (state-struct registration) but same module family.
63+
- [[how-do-i-pack-an-im-col32-color-from-dasimgui-v2-code-without-depending-on-the-v1-daslib-imgui-boost-path]] — sibling dasImgui idiom.
64+
65+
## Questions
66+
- How do I call a dasImgui (or any managed C++) method on a struct field that's bound as a raw pointer — e.g. AddFontFromFileTTF on GetIO().Fonts?
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
---
2+
slug: how-do-i-run-dastest-in-benchmark-only-mode-and-what-s-the-command-line-syntax
3+
title: How do I run dastest in benchmark-only mode and what's the command-line syntax?
4+
created: 2026-05-16
5+
last_verified: 2026-05-16
6+
links: []
7+
---
8+
9+
Benchmarks are functions annotated with `[benchmark]` from `dastest/testing_boost.das`. Run them via the dastest harness with `--bench`:
10+
11+
```bash
12+
# All benchmarks in a directory (skip the regular tests)
13+
./bin/daslang dastest/dastest.das -- --bench --test benchmarks/sql --test-names none
14+
15+
# Just one file
16+
./bin/daslang dastest/dastest.das -- --bench --test benchmarks/sql/count_aggregate.das --test-names none
17+
18+
# Filter by [benchmark] function-name prefix (substring match on the function name)
19+
./bin/daslang dastest/dastest.das -- --bench --bench-names sum_ --test benchmarks/sql --test-names none
20+
21+
# Collect N samples for variance / averaging
22+
./bin/daslang dastest/dastest.das -- --bench --test benchmarks/sql/count_aggregate.das --test-names none --count 5
23+
```
24+
25+
Key flags:
26+
- `--bench` — enable benchmark execution
27+
- `--test <path>` — folder or single file (NOT positional)
28+
- `--test-names none` — skip regular `[test]` discovery (benchmarks only)
29+
- `--bench-names <prefix>` — filter benchmarks by function-name prefix
30+
- `--bench-format <native|go|json>` — output format
31+
- `--count <n>` — repeat all benchmarks N times
32+
33+
Benchmarks only run after all module **tests** have passed; that's why `--test-names none` is the canonical "skip tests, run benchmarks" combo.
34+
35+
Output is `<sub_name> N ns/op <bytes>/op <allocs>/op <SB>/op <strings>/op`. If the benchmark `b |> run(name, chunk_size, op)` form passes a chunk_size (typically the dataset size), the displayed ns/op is **divided by that chunk_size** — i.e. per-element time, not per-op-call time. Sub-nanosecond results (`0 ns/op`) usually mean early-exit hit the answer in O(1) regardless of dataset size.
36+
37+
Reference: `dastest/README.md` and `dastest/dastest_clargs.das`.
38+
39+
## Questions
40+
- How do I run dastest in benchmark-only mode and what's the command-line syntax?
41+
- What's the dastest --bench command line?
42+
- How do I filter dastest benchmarks by name?
43+
44+
## Questions
45+
- How do I run dastest in benchmark-only mode and what's the command-line syntax?

0 commit comments

Comments
 (0)