Merge pull request GaijinEntertainment#2128 from GaijinEntertainment/language-testing

daScriptTests migrated to dastest framework

Author: borisbat

.github/copilot-instructions.md

@@ -36,6 +36,10 @@ Task-specific instructions are split into skill files under `skills/`. You MUST
 
 Multiple skill files may apply to a single task. For example, creating a new daslib module requires reading `skills/das_formatting.md`, `skills/daslib_modules.md`, and possibly `skills/documentation_rst.md`.
 
+### Updating Instructions with New Knowledge
+
+When you discover something new about daslang syntax, semantics, or conventions — whether through compiler errors, user corrections, or experimentation — **update this file** (and its `.github/copilot-instructions.md` mirror) with the new knowledge. Add it to the appropriate section (gen2 syntax, common gotchas, etc.) so future sessions benefit. If it relates to a specific skill area, update the relevant `skills/*.md` file too.
+
 ## daslang Language — Gen2 Syntax (REQUIRED)
 
 All code examples and documentation MUST use gen2 syntax (add `options gen2` at the top of every file). Key rules:
@@ -57,6 +61,7 @@ All code examples and documentation MUST use gen2 syntax (add `options gen2` at
 - **Bitfield dot access:** read with `f.flag` (returns bool), write with `f.flag = true/false`
 - **`typeinfo`** gen2 syntax: trait name goes **outside** parentheses — `typeinfo trait_name(type<T>)`, NOT `typeinfo(trait_name type<T>)`. With subtrait: `typeinfo has_method<name>(type<T>)`. With two traits: `typeinfo trait<sub;extra>(type<T>)`
 - **`static_if`:** `static_if (condition) { ... }` — parentheses required in gen2
+- **Type function call:** `take(type<int>, 1, 2)` — NOT `take < int > (1, 2)`. The `type<T>` is passed as a regular argument
 
 ### Important defaults
 
@@ -186,6 +191,7 @@ All code examples and documentation MUST use gen2 syntax (add `options gen2` at
 - Blocks CANNOT be stored in containers, returned from functions, or captured — use lambdas or function pointers for those use cases
 - `match`, `multi_match`, `static_match` macros (from `daslib/match.das`) handle side effects automatically — do NOT add `[sideeffects]` annotations to functions that only use match
 - `[export] def main()` returns `void` — do NOT `return true` or return other values from main
+- **`feint` vs `print` in tests**: `feint` is a no-op with the same signature and `SideEffects::modifyExternal` as `print` — it won't be optimized out, but produces no output. Use `feint` in tests instead of `print` unless testing actual print/logging behavior
 - **`push` vs `emplace` vs `push_clone`** for arrays: `push(arr, val)` copies `val` into the array (fails for non-copyable types like `array<int>`); `emplace(arr, val)` **moves** `val` into the array (source is zeroed/destroyed after); `push_clone(arr, val)` **clones** `val` into the array (works for any type, preserves source). When generating code that operates on user structs with potentially non-copyable fields, prefer `push_clone` (preserves source) or `emplace` (when source consumption is intentional).
 - **Non-copyable types**: `array<T>`, `table<K;V>`, lambdas, and any struct containing them cannot be copied with `=` or `push`. Use `:=` (clone-assign), `push_clone`, or `<-` (move) instead. The compiler error is clear: "can't copy non-copyable type"
 
@@ -205,20 +211,14 @@ All code examples and documentation MUST use gen2 syntax (add `options gen2` at
 - `include/daScript/` — C++ headers
 - `daslib/` — Standard library modules (86 .das files)
 - `dastest/` — Test framework
-- `tests/` — Test suite (with per-module subfolders: `tests/decs/`, `tests/match/`, `tests/json/`, etc.)
+- `tests/` — Test suite (35 subdirectories, ~226 `.das` files). See `tests/README.md` for a full index of every test file, what it tests, and whether it expects compile errors.
 - `doc/source/reference/language/` — RST language documentation (36 files)
 - `doc/source/stdlib/` — RST standard library documentation (auto-generated + handmade)
 - `doc/reflections/` — Documentation generation tools (das2rst.das, rst.das, gen_module_examples.py)
 - `tutorials/language/` — Language tutorial `.das` files (42 progressive tutorials)
 - `tutorials/integration/cpp/` — C++ integration tutorials (embedding daslang in C++ host applications)
 - `tutorials/macros/` — Macro tutorials (call macros, reader macros, etc.)
 - `doc/source/reference/tutorials/` — RST companion pages for each tutorial
-- `tests/linq/` — LINQ module tests (15 test files, ~500 tests)
-- `tests/functional/` — Functional module tests
-- `tests/json/` — JSON module tests (4 test files, ~148 tests)
-- `tests/regex/` — Regex module tests (8 test files, 278 tests)
-- `tests/interfaces/` — Interface module tests (4 test files, 67 tests)
-- `tests/soa/` — SOA module tests (4 test files, 126 tests: basic, iteration, container ops, non-copyable fields)
 - `modules/` — External plugin modules
 
 ## Keywords Reference

.github/workflows/build.yml

@@ -309,29 +309,25 @@ jobs:
             # when exceptions disabled, we still throw in tests and there'll be memory leaks
             echo -e "leak:format_error\n" > suppressions.txt
             export LSAN_OPTIONS="suppressions='suppressions.txt'"
-            [ "${{ env.das_llvm_disabled }}" = "ON" ] || ./daScriptTest -jit
-            ./daScriptTest
             # Use more time to pass UBSAN and disable leaks in JIT mode because we exit using exit(), and do not call destructors.
-            [ "${{ env.das_llvm_disabled }}" = "ON" ] || ASAN_OPTIONS=detect_leaks=0 ./daslang _dasroot_/dastest/dastest.das -jit -- --color --test ../tests || ASAN_OPTIONS=detect_leaks=0 ./daslang _dasroot_/dastest/dastest.das -jit -- --color --isolated-mode --timeout 4800 --test ../tests
+            # TEMPORARILY DISABLED: -jit dastest
+            # [ "${{ env.das_llvm_disabled }}" = "ON" ] || ASAN_OPTIONS=detect_leaks=0 ./daslang _dasroot_/dastest/dastest.das -jit -- --color --test ../tests || ASAN_OPTIONS=detect_leaks=0 ./daslang _dasroot_/dastest/dastest.das -jit -- --color --isolated-mode --timeout 4800 --test ../tests
             ./daslang _dasroot_/dastest/dastest.das -- --color --test ../tests || ./daslang _dasroot_/dastest/dastest.das -- --color --isolated-mode --timeout 3600 --test ../tests
             ;;
            windows32)
             cd bin/"${{ matrix.cmake_preset }}"
-            ./daScriptTest.exe
             ./daslang _dasroot_/dastest/dastest.das -- --color --test ../../tests || ./daslang _dasroot_/dastest/dastest.das -- --color --isolated-mode --timeout 3600 --test ../../tests
             ;;
            windows*)
             cd bin/"${{ matrix.cmake_preset }}"
-            [ "${{ env.das_llvm_disabled }}" = "ON" ] || ./daScriptTest.exe -jit
-            ./daScriptTest.exe
-            [ "${{ env.das_llvm_disabled }}" = "ON" ] || ./daslang _dasroot_/dastest/dastest.das -jit -- --color --test ../../tests || ./daslang _dasroot_/dastest/dastest.das -jit -- --color --isolated-mode --timeout 3600 --test ../../tests
+            # TEMPORARILY DISABLED: -jit dastest
+            # [ "${{ env.das_llvm_disabled }}" = "ON" ] || ./daslang _dasroot_/dastest/dastest.das -jit -- --color --test ../../tests || ./daslang _dasroot_/dastest/dastest.das -jit -- --color --isolated-mode --timeout 3600 --test ../../tests
             ./daslang _dasroot_/dastest/dastest.das -- --color --test ../../tests || ./daslang _dasroot_/dastest/dastest.das -- --color --isolated-mode --timeout 3600 --test ../../tests
             ;;
            *)
             cd bin
-            [ "${{ env.das_llvm_disabled }}" = "ON" ] || ./daScriptTest -jit
-            ./daScriptTest
-            [ "${{ env.das_llvm_disabled }}" = "ON" ] || ./daslang _dasroot_/dastest/dastest.das -jit -- --color --test ../tests || ./daslang _dasroot_/dastest/dastest.das -jit -- --color --isolated-mode --timeout 3600 --test ../tests
+            # TEMPORARILY DISABLED: -jit dastest
+            # [ "${{ env.das_llvm_disabled }}" = "ON" ] || ./daslang _dasroot_/dastest/dastest.das -jit -- --color --test ../tests || ./daslang _dasroot_/dastest/dastest.das -jit -- --color --isolated-mode --timeout 3600 --test ../tests
             ./daslang _dasroot_/dastest/dastest.das -- --color --test ../tests || ./daslang _dasroot_/dastest/dastest.das -- --color --isolated-mode --timeout 3600 --test ../tests
             ;;
         esac

CLAUDE.md

@@ -36,6 +36,10 @@ Task-specific instructions are split into skill files under `skills/`. You MUST
 
 Multiple skill files may apply to a single task. For example, creating a new daslib module requires reading `skills/das_formatting.md`, `skills/daslib_modules.md`, and possibly `skills/documentation_rst.md`.
 
+### Updating Instructions with New Knowledge
+
+When you discover something new about daslang syntax, semantics, or conventions — whether through compiler errors, user corrections, or experimentation — **update this file** (and its `.github/copilot-instructions.md` mirror) with the new knowledge. Add it to the appropriate section (gen2 syntax, common gotchas, etc.) so future sessions benefit. If it relates to a specific skill area, update the relevant `skills/*.md` file too.
+
 ## daslang Language — Gen2 Syntax (REQUIRED)
 
 All code examples and documentation MUST use gen2 syntax (add `options gen2` at the top of every file). Key rules:
@@ -57,6 +61,7 @@ All code examples and documentation MUST use gen2 syntax (add `options gen2` at
 - **Bitfield dot access:** read with `f.flag` (returns bool), write with `f.flag = true/false`
 - **`typeinfo`** gen2 syntax: trait name goes **outside** parentheses — `typeinfo trait_name(type<T>)`, NOT `typeinfo(trait_name type<T>)`. With subtrait: `typeinfo has_method<name>(type<T>)`. With two traits: `typeinfo trait<sub;extra>(type<T>)`
 - **`static_if`:** `static_if (condition) { ... }` — parentheses required in gen2
+- **Type function call:** `take(type<int>, 1, 2)` — NOT `take < int > (1, 2)`. The `type<T>` is passed as a regular argument
 
 ### Important defaults
 
@@ -186,6 +191,7 @@ All code examples and documentation MUST use gen2 syntax (add `options gen2` at
 - Blocks CANNOT be stored in containers, returned from functions, or captured — use lambdas or function pointers for those use cases
 - `match`, `multi_match`, `static_match` macros (from `daslib/match.das`) handle side effects automatically — do NOT add `[sideeffects]` annotations to functions that only use match
 - `[export] def main()` returns `void` — do NOT `return true` or return other values from main
+- **`feint` vs `print` in tests**: `feint` is a no-op with the same signature and `SideEffects::modifyExternal` as `print` — it won't be optimized out, but produces no output. Use `feint` in tests instead of `print` unless testing actual print/logging behavior
 - **`push` vs `emplace` vs `push_clone`** for arrays: `push(arr, val)` copies `val` into the array (fails for non-copyable types like `array<int>`); `emplace(arr, val)` **moves** `val` into the array (source is zeroed/destroyed after); `push_clone(arr, val)` **clones** `val` into the array (works for any type, preserves source). When generating code that operates on user structs with potentially non-copyable fields, prefer `push_clone` (preserves source) or `emplace` (when source consumption is intentional).
 - **Non-copyable types**: `array<T>`, `table<K;V>`, lambdas, and any struct containing them cannot be copied with `=` or `push`. Use `:=` (clone-assign), `push_clone`, or `<-` (move) instead. The compiler error is clear: "can't copy non-copyable type"
 
@@ -205,20 +211,14 @@ All code examples and documentation MUST use gen2 syntax (add `options gen2` at
 - `include/daScript/` — C++ headers
 - `daslib/` — Standard library modules (86 .das files)
 - `dastest/` — Test framework
-- `tests/` — Test suite (with per-module subfolders: `tests/decs/`, `tests/match/`, `tests/json/`, etc.)
+- `tests/` — Test suite (35 subdirectories, ~226 `.das` files). See `tests/README.md` for a full index of every test file, what it tests, and whether it expects compile errors.
 - `doc/source/reference/language/` — RST language documentation (36 files)
 - `doc/source/stdlib/` — RST standard library documentation (auto-generated + handmade)
 - `doc/reflections/` — Documentation generation tools (das2rst.das, rst.das, gen_module_examples.py)
 - `tutorials/language/` — Language tutorial `.das` files (42 progressive tutorials)
 - `tutorials/integration/cpp/` — C++ integration tutorials (embedding daslang in C++ host applications)
 - `tutorials/macros/` — Macro tutorials (call macros, reader macros, etc.)
 - `doc/source/reference/tutorials/` — RST companion pages for each tutorial
-- `tests/linq/` — LINQ module tests (15 test files, ~500 tests)
-- `tests/functional/` — Functional module tests
-- `tests/json/` — JSON module tests (4 test files, ~148 tests)
-- `tests/regex/` — Regex module tests (8 test files, 278 tests)
-- `tests/interfaces/` — Interface module tests (4 test files, 67 tests)
-- `tests/soa/` — SOA module tests (4 test files, 126 tests: basic, iteration, container ops, non-copyable fields)
 - `modules/` — External plugin modules
 
 ## Keywords Reference

CMakeLists.txt

@@ -909,17 +909,9 @@ SET(DAS_DASCRIPT_MAIN_SRC
     CACHE INTERNAL "DAS_DASCRIPT_MAIN_SRC"
 )
 
-# Test module
-
-
 # Tests
 if (NOT ${DAS_TESTS_DISABLED})
-    include(examples/test/CMakeLists.txt)
     include(examples/pathTracer/CMakeLists.txt)
-
-    FOREACH(_example ${DAS_ALL_EXAMPLES})
-        TARGET_LINK_LIBRARIES(${_example} libDaScriptTest)
-    ENDFOREACH()
 endif()
 
 # Profile module

dastest/dastest.das

@@ -17,6 +17,14 @@ options multiple_contexts
 
 let private jsonResultPrefix = "##dastest##\n"
 
+struct FileTiming {
+    uri : string
+    dt : int     // microseconds
+}
+
+var private fileTimings : array<FileTiming>
+var private timingOutliers : int = 0
+
 
 def private collect_input_paths(args : array<string>; var paths : array<string>) {
     for (i in range(length(args) - 1)) {
@@ -109,6 +117,8 @@ def main() {
         return
     }
 
+    timingOutliers = args |> get_int_arg("--timing-outliers", 0)
+
     let isolatedMode = args |> has_value("--isolated-mode")
     let ctx <- SuiteCtx(args)
     if (!isolatedMode) {
@@ -124,6 +134,9 @@ def main() {
             } else {
                 log::red("FAIL {uri} {time_dt_hr(fileDt)}")
             }
+            if (timingOutliers > 0) {
+                fileTimings |> push(FileTiming(uri = clone_string(uri), dt = fileDt))
+            }
             res += status
         }
     } else {
@@ -202,6 +215,9 @@ def main() {
                         for (l in isoRes.log) {
                             log::info_raw(l)
                         }
+                        if (timingOutliers > 0) {
+                            fileTimings |> push(FileTiming(uri = clone_string(isoRes.uri), dt = isoRes.fileDt))
+                        }
                         if (isoRes.exitCode == 0 && isoRes.parsedResult && isoRes.status.failed + isoRes.status.errors == 0) {
                             log::green("PASS {isoRes.uri} {time_dt_hr(isoRes.fileDt)}")
                         } else {
@@ -238,10 +254,54 @@ def timeout_tests(var res : SuiteResult; start_time : int64; timeout_sec : float
 }
 
 
+def private print_timing_outliers() {
+    let n = length(fileTimings)
+    // compute mean
+    var sum = 0.0lf
+    for (ft in fileTimings) {
+        sum += double(ft.dt)
+    }
+    let mean = sum / double(n)
+    // compute stddev
+    var variance = 0.0lf
+    for (ft in fileTimings) {
+        let diff = double(ft.dt) - mean
+        variance += diff * diff
+    }
+    let stddev = sqrt(variance / double(n))
+    // sort descending by time
+    sort(fileTimings) <| $(a, b) {
+        return a.dt > b.dt
+    }
+    let count = min(timingOutliers, n)
+    let threshold = mean + 2.0lf * stddev
+    // only print if there are actual outliers (top entry above threshold) or if few files
+    var meanStr = ""
+    meanStr = fmt(":.3f", mean / 1000000.0lf)
+    var stddevStr = ""
+    stddevStr = fmt(":.3f", stddev / 1000000.0lf)
+    var thresholdStr = ""
+    thresholdStr = fmt(":.3f", threshold / 1000000.0lf)
+    log::info("Timing: mean {meanStr}s, stddev {stddevStr}s, threshold {thresholdStr}s")
+    log::info("Top {count} slowest files:")
+    for (i in range(count)) {
+        let dtSec = double(fileTimings[i].dt) / 1000000.0lf
+        let marker = double(fileTimings[i].dt) > threshold ? " [outlier]" : ""
+        var dtStr = ""
+        dtStr = fmt(":.3f", dtSec)
+        var line = "  {dtStr}s  {fileTimings[i].uri}{marker}"
+        log::info(line)
+    }
+}
+
+
 def finish_tests(res : SuiteResult; start_time : int64) {
     let resCode = res.failed + res.errors
     let totalDt = get_time_usec(start_time)
     log::info("\n{to_string(res)}")
+    if (resCode == 0 && timingOutliers > 0 && length(fileTimings) > 1) {
+        print_timing_outliers()
+    }
     if (resCode == 0) {
         log::green("SUCCESS! {time_dt_hr(totalDt)}")
     } else {

dastest/fs.das

@@ -84,6 +84,9 @@ def scan_dir(path : string; var cache : table<string; void?>; var res : array<st
             return
         }
         if (fStat.is_dir) {
+            if (n |> starts_with("_")) {
+                return
+            }
             f |> scan_dir(cache, res, suffix)
         } elif (fStat.is_reg && f |> ends_with(suffix) && !cache |> key_exists(f)) {
             cache |> insert(f, null)

doc/reflections/das2rst.das

@@ -136,7 +136,7 @@ get_value|insert_clone|emplace_new|insert_default|emplace_default|get_with_defau
         group_by_regex("Smart ptr infrastructure", mod, %regex~(add_ptr_ref|smart_ptr|get_const_ptr|move|move_new|get_ptr$)%%),
         group_by_regex("Macro infrastructure", mod, %regex~(is_folding|is_compiling|is_in_completion|is_reporting_compilation_errors)%%),
         group_by_regex("Profiler", mod, %regex~(profile|reset_profiler|dump_profile_info|collect_profile_info)$%%),
-        group_by_regex("System infrastructure", mod, %regex~(panic|print|sprint|sprint_json|to_log|to_compiler_log|error|terminate|breakpoint|stackwalk|get_das_root|is_in_aot|aot_enabled|is_intern_strings|eval_main_loop)$%%),
+        group_by_regex("System infrastructure", mod, %regex~(panic|print|feint|sprint|sprint_json|to_log|to_compiler_log|error|terminate|breakpoint|stackwalk|get_das_root|is_in_aot|aot_enabled|is_intern_strings|eval_main_loop)$%%),
         group_by_regex("Memory manipulation", mod, %regex~(intptr|memcmp|variant_index|set_variant_index|hash|memcpy|lock_data|map_to_array|map_to_ro_array)$%%),
         group_by_regex("Binary serializer", mod, %regex~(binary_load|binary_save)$%%),
         group_by_regex("Path and command line", mod, %regex~(get_command_line_arguments)$%%),

doc/source/stdlib/builtin.rst

@@ -2310,6 +2310,7 @@ System infrastructure
   *  :ref:`breakpoint () <function-builtin_breakpoint>`
   *  :ref:`error (text: string) <function-builtin_error_string>`
   *  :ref:`eval_main_loop (block: block\<():void\>) <function-builtin_eval_main_loop_block_ls__c_void_gr_>`
+  *  :ref:`feint (text: string) <function-builtin_feint_string>`
   *  :ref:`get_das_root () : string <function-builtin_get_das_root>`
   *  :ref:`is_in_aot () : bool <function-builtin_is_in_aot>`
   *  :ref:`is_intern_strings () : bool <function-builtin_is_intern_strings>`
@@ -2354,6 +2355,15 @@ Executes the application main loop by repeatedly invoking `block` until it retur
 
 :Arguments: * **block** : block<void> implicit
 
+.. _function-builtin_feint_string:
+
+.. das:function:: feint(text: string)
+
+No-op replacement for `print`. Has the same signature and side-effect annotations as `print`, but intentionally does nothing. Use `feint` in tests where print-like behavior is needed to prevent the call from being optimized out, but no actual output is desired.
+
+
+:Arguments: * **text** : string implicit
+
 .. _function-builtin_get_das_root:
 
 .. das:function:: get_das_root() : string

doc/source/stdlib/detail/builtin.rst

@@ -34,6 +34,8 @@
 
 .. |handmade/function-builtin-print| replace:: to be documented in |handmade/function-builtin-print|.rst
 
+.. |handmade/function-builtin-feint| replace:: to be documented in |handmade/function-builtin-feint|.rst
+
 .. |handmade/function-builtin-error| replace:: to be documented in |handmade/function-builtin-error|.rst
 
 .. |handmade/function-builtin-sprint| replace:: to be documented in |handmade/function-builtin-sprint|.rst