`intrinsic-test`: Adding x86 behavioural testing. #1894

madhav-madhusoodanan · 2025-08-03T13:31:06Z

Context

This is a redo of PR #1814, since a lot of details have changed with PRs #1863, #1862, #1861, #1852.

crates/intrinsic-test/src/x86/intrinsic.rs

crates/intrinsic-test/src/x86/mod.rs

folkertdev · 2025-08-05T10:36:48Z

crates/intrinsic-test/src/x86/types.rs

+                match str::parse::<u32>(etype_processed.as_str()) {
+                    Ok(value) => data.bit_len = Some(value),
+                    Err(_) => {
+                        data.bit_len = match data.kind() {
+                            TypeKind::Char(_) => Some(8),
+                            TypeKind::BFloat => Some(16),
+                            TypeKind::Int(_) => Some(32),
+                            TypeKind::Float => Some(32),
+                            _ => None,
+                        };
+                    }
+                }


why are only some type kinds covered here? Maybe this could be a method on TypeKind?

crates/intrinsic-test/src/x86/compile.rs

folkertdev

you should rebase on top of the upstream master branch instead of merging it in. That keeps the git history clean.

folkertdev · 2025-08-05T22:48:30Z

ci/run.sh

+    x86_64-unknown-linux-gnu*)
+        CPPFLAGS="${TEST_CPPFLAGS}" RUSTFLAGS="${HOST_RUSTFLAGS}" RUST_LOG=warn \
+            cargo run "${INTRINSIC_TEST}" "${PROFILE}"  \
+            --bin intrinsic-test -- intrinsics_data/x86-intel.xml \
+            --runner "${TEST_RUNNER}" \
+            --cppcompiler "${TEST_CXX_COMPILER}" \
+            --target "${TARGET}"
+        ;;


we'll have to see how to do this exactly, but we want to split these out of the main CI job to speed it up

madhav-madhusoodanan · 2025-09-05T08:19:24Z

Seems like the CI run at this point failed due to this error. I'll retry shortly:

#6 39.45   Could not connect to archive.ubuntu.com:80 (185.125.190.82), connection timed out Could not connect to archive.ubuntu.com:80 (185.125.190.83), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.91.83), connection timed out Could not connect to archive.ubuntu.com:80 (185.125.190.81), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.91.81), connection timed out Could not connect to archive.ubuntu.com:80 (185.125.190.36), connection timed out Could not connect to archive.ubuntu.com:80 (185.125.190.39), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.91.82), connection timed out
#6 39.45   Unable to connect to archive.ubuntu.com:80:
#6 39.45 Fetched 126 kB in 39s (3208 B/s)
#6 39.45 Reading package lists...
#6 39.46 W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/questing/InRelease  Unable to connect to archive.ubuntu.com:80:
#6 39.46 W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/questing-updates/InRelease  Unable to connect to archive.ubuntu.com:80:
#6 39.46 W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/questing-backports/InRelease  Unable to connect to archive.ubuntu.com:80:
#6 39.46 W: Some index files failed to download. They have been ignored, or old ones used instead.

folkertdev · 2025-09-10T16:31:59Z

Can you rebase this (and remove the merge commits) sometime? That would make it a lot easier to see what has actually changed.

Also, why do you push to then have CI fail? Running this locally is a lot faster than having CI do it, because you can skip the earlier steps and run just the intrinsic tests.

folkertdev

oh, wait, it probably is up to date? it's just that xml file is enormous. that's crazy. Should we like, try to trim that down?

crates/intrinsic-test/src/x86/mod.rs

madhav-madhusoodanan · 2025-09-10T19:15:00Z

@folkertdev I've been syncing using the below command whenever I'm updating from master:

git rebase master

Am I doing it correctly?

madhav-madhusoodanan · 2025-09-10T19:17:13Z

About the XML file specification, may I ask what "trim down" means?

folkertdev · 2025-09-10T19:17:24Z

depends a bit on your git setup, but it seems to work allright. I was just confused by the enormous number of lines changed, but that's all due to that XML file.

folkertdev · 2025-09-10T19:20:07Z

What i mean is, could we process that XML file into a smaller XML file that only stores what we need? That would reduce the size of the repo (not sure how big that file is, maybe it compresses well?) and the speed of the intrinsic tests.

There are risks too, e.g. what we generate could go out of date with the official XML file. So maybe it's fine this way.

madhav-madhusoodanan · 2025-09-10T19:26:57Z

Ahh, makes sense.

I think it might be best to keep the source of truth as unchanged as possible, since there is no direct way to obtain the sources sometimes.

For example, the XML file originally existed in the stdarch-verify crate, and I've had to check it by manually downloading the x86 reference (which comes in a folder containing HTML, CSS and JS files) at the link called Download: Offline Intel® Intrinsics Guide in the Intel intrinsics reference site

The entire XML data can be found stored as a hardcoded string that is assigned to a variable in a JS file.

array. Extra: 1. Added better load fuctions 2. Added an update_simd_len() function to support cases where the bit_len of the element need to be inferred from its partner arguments before calculating the simd_len

extras: 1. call update_simd_len() after inferring bit_len for arguments of certain intrinsics 2. handle the effective bit_len for _mm_mpsadbw_epu8 intrinsic's `imm8` argument which has only 3 bits that are used

arguments with UI16 etype and __m128d type)

intrinsic

definition available for armv7 also

currently

arguments in Rust) to C++ testfiles. Also added extra compilation flags

being killed. Also separated declarations and definitions for C++ testfiles.

processing

to C++ version of the same.

base type

instead of letting C++ do it (and potentially change the bits)

same (during typecasting)

like integer to float or vice versa)

to be cast to float16_t because the bits are stored in the lower half of the type)

variants for compatibility with the Rust version

immediately

rustbot assigned folkertdev Aug 3, 2025

madhav-madhusoodanan commented Aug 3, 2025

View reviewed changes

crates/intrinsic-test/src/x86/intrinsic.rs Outdated Show resolved Hide resolved

madhav-madhusoodanan force-pushed the intrinsic-test-x86-addition branch 2 times, most recently from 41db5a8 to 5fc0f3b Compare August 5, 2025 10:16

folkertdev reviewed Aug 5, 2025

View reviewed changes

madhav-madhusoodanan force-pushed the intrinsic-test-x86-addition branch from 9e28106 to 111cd5d Compare August 5, 2025 16:22

folkertdev reviewed Aug 5, 2025

View reviewed changes

madhav-madhusoodanan force-pushed the intrinsic-test-x86-addition branch from f3f87f2 to 2ec747c Compare August 9, 2025 12:20

madhav-madhusoodanan force-pushed the intrinsic-test-x86-addition branch from 2ec747c to a8313d0 Compare September 5, 2025 08:16

madhav-madhusoodanan force-pushed the intrinsic-test-x86-addition branch 13 times, most recently from 126a3ce to e79129a Compare September 10, 2025 05:58

folkertdev reviewed Sep 10, 2025

View reviewed changes

crates/intrinsic-test/src/x86/mod.rs Outdated Show resolved Hide resolved

madhav-madhusoodanan added 28 commits October 10, 2025 17:35

chore: vector types cannot be the type of an individual element in an

289efd1

array. Extra: 1. Added better load fuctions 2. Added an update_simd_len() function to support cases where the bit_len of the element need to be inferred from its partner arguments before calculating the simd_len

chore: accomodate for immwidth field for constraints

341efd6

extras: 1. call update_simd_len() after inferring bit_len for arguments of certain intrinsics 2. handle the effective bit_len for _mm_mpsadbw_epu8 intrinsic's `imm8` argument which has only 3 bits that are used

feat: defined more load functions that are natively not defined (such as

c1053df

arguments with UI16 etype and __m128d type)

chore: corrected the imm-width correction location for _mm_mpsadbw_epu8

33f80b7

intrinsic

feat: added exclusion list to intrinsic-test CI pipeline

058cd3e

chore: clean up unused variables

8f1a811

feat: moved cast<T1, T2> to architecture-specific definitions

db04c94

fix: remove extra brackets for cast definition in arm/config.rs

483bc55

make std::ostream& operator<<(std::ostream& os, float16_t value);

83d8559

definition available for armv7 also

feat: add missing_x86.txt to filter out intrinsics that cannot be tested

cd18cdc

currently

feat: added custom helper functions (that helped load intrinsic

bc9b405

arguments in Rust) to C++ testfiles. Also added extra compilation flags

chore: add more compiler flags for compiling x86 intrinsics in C++

3e7ad48

chore: add verbose cli option to C++ compiler

4c1e29a

feat: add clang to dockerfile and change clang++-19 to clang++

7f75d28

fix: add libstdc++-dev to fix iostream not found error

0bc0ba3

fix: making compilation step run one by one to prevent the process from

1c918b6

being killed. Also separated declarations and definitions for C++ testfiles.

feat: attempting compilation of smaller chunks for faster parallel

c3adbbf

processing

feat: add c_programs to PATH and increase chunk size to 400

2be5995

feat: display __mmask8 values so that non-utf8 values are not displayed

c6abf5b

feat: add formatting for __m128i, __m256i, __m512i types that is similar

35ae79b

to C++ version of the same.

feat: make the debug_i16 into a generic debug_as function that adapts to

95d5937

base type

feat: casting the results of the lane function by preserving the bits

bf86ce2

instead of letting C++ do it (and potentially change the bits)

fix: update the display of uint8_t type in C++

f77ba2c

Explicitly cast bits instead of allowing C++ to automatically cast the

5f52e3a

same (during typecasting)

feat: update cast<> function to reduce spurious cast functions (cases

d9a2de4

like integer to float or vice versa)

Feat: Compile C++ testfiles using C++23 standard

0182991

Feat: allow downcasting (useful for certain cases where uint32_t needs

8e39edc

to be cast to float16_t because the bits are stored in the lower half of the type)

feat: explicitly casting the result of the lane function to unsigned

503f995

variants for compatibility with the Rust version

madhav-madhusoodanan force-pushed the intrinsic-test-x86-addition branch from 03c5a8e to 503f995 Compare October 10, 2025 12:08

feat: updated exclusion list with more intrinsics, that can be fixed

c259207

immediately

intrinsic-test: Adding x86 behavioural testing. #1894

Are you sure you want to change the base?

intrinsic-test: Adding x86 behavioural testing. #1894

Uh oh!

Conversation

madhav-madhusoodanan commented Aug 3, 2025

Context

Uh oh!

Uh oh!

Uh oh!

folkertdev Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

folkertdev left a comment

Choose a reason for hiding this comment

Uh oh!

folkertdev Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

madhav-madhusoodanan commented Sep 5, 2025

Uh oh!

folkertdev commented Sep 10, 2025

Uh oh!

folkertdev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

madhav-madhusoodanan commented Sep 10, 2025

Uh oh!

madhav-madhusoodanan commented Sep 10, 2025

Uh oh!

folkertdev commented Sep 10, 2025

Uh oh!

folkertdev commented Sep 10, 2025

Uh oh!

madhav-madhusoodanan commented Sep 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

`intrinsic-test`: Adding x86 behavioural testing. #1894

`intrinsic-test`: Adding x86 behavioural testing. #1894

madhav-madhusoodanan commented Sep 10, 2025 •

edited

Loading