This page describes how to run tests for the Slang compiler and the format of test files.
Run the standard Rust test suite:
# Run all tests (unit + CLI)
cargo test
# Run only unit tests
cargo test --lib
# Run only CLI/integration tests (the test target is named `mod`)
cargo test --test mod
# Run a specific test
cargo test --test mod -- cli::bin::defaultEvery ```console block in the
CLI user guide is an executable
trycmd case, run by the docs_examples CLI test
(solx/tests/cli/docs_examples.rs) with the rest of the suite. The commands
run in a temporary sandbox seeded from 02-command-line-interface.in/, and
the output lines in the guide must match the compiler's actual output.
When the test fails after an intentional output change, regenerate the stale blocks in place and review the documentation diff:
TRYCMD=overwrite cargo test -p slang --test mod docs_examplesOverwriting preserves ... line elisions, but inline [..] wildcards on
lines that no longer match are expanded to the literal output and must be
restored by hand — watch for benchmark timings and for hex-encoded DWARF
output.
For version bumps, do not use TRYCMD=overwrite: the CBOR trailer lines that
carry the version also carry a [..] metadata-digest wildcard, and a blessed
literal digest only breaks on the next Slang re-pin. Update the version in the
trailer lines directly instead: the three bytes after 65736c616e6743 (the
slang key and its byte-string header) are the major, minor and patch numbers.
The solx-tester tool runs integration tests by compiling contracts and executing them with revm.
# Build the compiler and tester
cargo build --release
# Run all integration tests
./target/release/solx-tester --solidity-compiler ./target/release/slang
# Run tests for a specific file
./target/release/solx-tester --solidity-compiler ./target/release/slang --path tests/solidity/simple/default.sol
# Run tests with specific optimizer settings
./target/release/solx-tester --solidity-compiler ./target/release/slang --optimizer M3B3--optimizer <PATTERN>— Filter by optimizer settings. Examples:M3B3— Match exact optimizer levelM^B3— Match M3 or Mz with B3M*B*— Match any M and B levels
--path <PATTERN>— Run only tests whose path contains the pattern.
The solx-dev tool can run tests against real-world Foundry and Hardhat projects:
# Build solx-dev
cargo build --release --bin solx-dev
# Run Foundry project tests
./target/release/solx-dev test foundry --test-config-path solx-dev/foundry-tests.toml
# Run Hardhat project tests
./target/release/solx-dev test hardhat --test-config-path solx-dev/hardhat-tests.tomlThe test configurations list projects that are cloned and tested automatically. See foundry-tests.toml and hardhat-tests.toml for the full list of tested projects.
The Sourcify sweep compiles each contract of a pinned Sourcify corpus with slang --standard-json.
When slang fails without timing out, the latest solx release compiles the same contract. This tells a slang gap apart from input neither compiler accepts.
cargo build --release --bin slang --bin solx-dev
# The committed fixtures
./target/release/solx-dev test sourcify --local-corpus-dir tests/sourcify-sweep/fixtures
./target/release/solx-dev test sourcify-report --results-path temp-sourcify-reports/results.jsonl
# The first 100 contracts of the pinned corpus
./target/release/solx-dev test sourcify --limit 100
# One shard of the pinned corpus, as a CI job runs it
./target/release/solx-dev test sourcify --shard-count 24 --shard-index 0 --memory-limit-mib 6144sourcify-tests.toml pins the corpus release and names the two compilers.
Each contract becomes one standard JSON input:
- its sources inline under their original paths
evmVersion,libraries,remappingsandviaIRpassed through verbatim- bytecode as the only output
- every version pragma rewritten to
pragma solidity >=<--solidity-version>;, so both compilers accept the same input
| Outcome | Meaning |
|---|---|
ok |
No error diagnostics and bytecode for a contract of the target source |
candidate-fail |
slang fails and the latest solx release compiles |
both-fail |
Both fail |
timeout |
slang exceeded --timeout |
The report ranks failures by normalized diagnostic, with example contracts. CI runs the sweep on every push to main and on pull requests labelled ci:sourcify-sweep, where the report becomes a pull request comment.
This section describes the format of test files used by solx-tester.
The repository contains three types of tests:
- Upstream — Tests following the Solidity semantic test format.
- Simple — Single-contract tests.
- Complex — Multi-contract tests and vendored DeFi projects.
Test data is located in:
tests/solidity/— Solidity test contractstests/llvm-ir/— LLVM IR test contracts
Each test comprises source code files and metadata.
Simple tests have only one source file, and their metadata is written in comments that start with !, for example, //! for Solidity.
Complex tests use a test.json file to describe their metadata and refer to source code files.
Metadata is a JSON object that contains the following fields:
cases— An array of test cases (described below).contracts— Used for complex tests to describe the contract instances to deploy. In simple tests, only oneTestcontract instance is deployed.
"contracts": {
"Main": "main.sol:Main",
"Callable": "callable.sol:Callable"
}libraries— An optional field that specifies library addresses for linker:
"libraries": {
"libraries/UQ112x112.sol": { "UQ112x112": "UQ112x112" },
"libraries/Math.sol": { "Math": "Math" }
}ignore— An optional flag that disables a test.modes— An optional field that specifies mode filters.Ystands for Yul pipeline,Efor EVM assembly pipeline. Compiler versions can be specified as SemVer ranges:
"modes": [
"Y",
"E",
"E >=0.8.30"
]group— An optional string field that specifies a test group for benchmarking.
All test cases are executed in a clean context, making them independent of each other.
Each test case contains the following fields:
name— A string name.comment— An optional string comment.inputs— An array of inputs (described below).expected— The expected return data for the last input.ignore,modes— Same as in test metadata.
Inputs specify the contract calls in the test case:
comment— An optional string comment.instance— The contract instance to call. Default:Test.caller— The caller address. Default:0xdeadbeef01000000000000000000000000000000.method— The method to call:#deployerfor the deployer call.#fallbackto perform a call with raw calldata.- Any other string is recognized as a function name. The function selector will be prepended to the calldata.
calldata— The input calldata:- A hexadecimal string:
"calldata": "0x00" - A numbers array (hex, decimal, or instance addresses). Each number is padded to 32 bytes:
"calldata": ["1", "2"]
- A hexadecimal string:
value— An optionalmsg.value, a decimal number withweiorETHsuffix.storage— Storage values to set before the call:
"storage": {
"Test.address": ["1", "2", "3", "4"]
}expected— The expected return data:- An array of numbers:
"expected": ["1", "2"] - Extended format with
return_data,exception, andevents:
- An array of numbers:
"expected": {
"return_data": ["0x01"],
"events": [
{
"topics": [
"0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"
],
"values": ["0xff"]
}
],
"exception": false
}The expected field can be an array of objects if different expected data is needed for different compiler versions. Use compiler_version as a SemVer range in extended expected format.
Notes:
InstanceName.addresscan be used in expected, calldata, and storage fields to insert a contract instance address.- If a deployer call is not specified for an instance, it will be generated automatically with empty calldata.
These tests follow the Solidity semantic test format.
Test descriptions and expected results are embedded as comments in the test file. Lines begin with // for Solidity files. The beginning of the test description is indicated by a comment line containing ----.