⚠️ IMPORTANT: Repository Migration in Progress - "The Weld"

This repository is being merged into ethereum/execution-specs during the week of October 20-24, 2025.

What's Happening?

All functionality from ethereum/execution-spec-tests (spec tests, test frameworks, and tooling) is moving to ethereum/execution-specs to streamline spec and test development. For background, see our blog post.

Timeline

Week of October 13-17, 2025:

• execution-spec-tests (EEST): Closing remaining PRs and porting issues to EELS. New PRs will not be accepted after this week.
• execution-specs (EELS): Finalizing BPO t8n changes to ensure test filling works for upcoming hard forks.

Week of October 20-24, 2025:

• October 20: Kickstart with a dry-run migration.
• October 20-24: Fix CI and fixture building/release issues that arise.
• October 24 (ETA): Weld finalized. All development moves to ethereum/execution-specs.

What This Means for You

• Spec Contributors: Continue working in ethereum/execution-specs as usual.
• Test Contributors:
  • Until October 20: Open draft PRs in ethereum/execution-spec-tests as you have been.
  • After October 24 (Weld Complete): Close any open draft PRs in EEST and reopen them in ethereum/execution-specs. All future test development happens in EELS.
• Client Teams: Test fixture releases will continue from ethereum/execution-spec-tests as usual. No changes to your consumption workflow.

Questions?

We will reach out with specific guidance as needed. For urgent questions, please contact the STEEL team.

---

Execution Spec Tests

About

The full execution-spec-tests documentation can be found here.

Note: All of ethereum/execution-spec-tests functionality (i.e., the spec tests, test frameworks and tooling) will move to ethereum/execution-specs in Q4 2025 to streamline spec and test development. For more information please see our blog post. For now, please continue contributing to both ethereum/execution-spec-tests and ethereum/execution-specs as you previously have - we will reach out to you if and when need be!

ethereum/execution-spec-tests is both a collection of test cases and a framework implemented in Python to generate tests for Ethereum execution clients.

The framework collects and executes the test cases in order to generate test fixtures (JSON) which can be consumed by any execution client to verify their implementation of ethereum/execution-specs. The fixtures, which define state transition and block tests, are generated by the framework using one of the t8n command-line tools that are provided by most execution clients, see below for an overview of the supported t8n tools.

---
title: Test Fixture Generation with execution-spec-tests
---
flowchart LR
  style C stroke:#333,stroke-width:2px
  style D stroke:#333,stroke-width:2px
  style G stroke:#F9A825,stroke-width:2px
  style H stroke:#F9A825,stroke-width:2px

  subgraph "ethereum/go-ethereum, ..."
    C[<code>evm t8n</code><br/>external executable]
  end

  subgraph ethereum/solidity
    D[<code>solc</code><br/>external executable]
  end

  subgraph ethereum/EIPs
    E(<code>EIPS/EIP-*.md</code><br/>SHA digest via Github API)
  end

  subgraph "ethereum/execution-spec-tests"
    A(<code>./tests/**/*.py</code><br/>Python Test Cases)
    B([<code>$ fill ./tests/</code><br/>Python Framework])
  end

  subgraph Test Fixture Consumers
    subgraph ethereum/hive
      G([<code>$ hive ...</code><br/>Go Test Framework])
    end
    H([Client executables])
  end

  C <-.-> B
  D <-.-> B
  A --> B
  E <-.-> |retrieve latest spec version\ncheck tested spec version| B
  B -->|output| F(<code>./fixtures/**/*.json</code>\nJSON Test Fixtures)
  F -->|input| G
  F -->|input| H

Loading

The generated test fixtures can be used:

1. Directly by client teams' test frameworks, and,
2. In the integration tests executed in the ethereum/hive framework.

Transition Tool Support

The following transition tools are supported by the framework:

Client	"t8n" Tool	Tracing Support
ethereum/evmone	evmone-t8n	Yes
ethereum/execution-specs	ethereum-spec-evm	Yes
ethereum/go-ethereum	evm t8n	Yes
ethereumjs	ethereumjs-t8ntool.sh	No
hyperledger/besu	evm t8n-server	Yes
status-im/nimbus-eth1	t8n	Yes

Upcoming EIP Development

Generally, specific t8n implementations and branches must be used when developing tests for upcoming EIPs.

We use named reference tags to point to the specific version of the t8n implementation that needs to be used fill the tests.

All current tags, their t8n implementation and branch they point to, are listed in .github/configs/evm.yaml.

Getting Started

Prerequisites

The tools provided by ethereum/execution-spec-tests use uv (docs.astral.sh/uv) to manage their dependencies and virtual environment. uv downloads Python for your target platform if one of the required versions (Python 3.11 or 3.12) is not available natively.

uv can be installed via curl (recommended; can self-update):

curl -LsSf https://astral.sh/uv/install.sh | sh

or pip (requires Python, can't self-update):

pip install uv

Installation

Clone execution-spec-tests and install its dependencies:

git clone --depth 1 https://github.com/ethereum/execution-spec-tests
cd execution-spec-tests
uv python install 3.12
uv python pin 3.12
uv sync --all-extras

See Installation Troubleshooting in the online docs if you encounter issues.

Exploring and Filling Test Cases

By default, JSON test fixtures are generated from this repository's Python test cases using the Ethereum Execution Layer Specification (EELS) reference implementation. The resulting JSON fixtures can be executed against execution clients to verify consensus. The process of generating fixtures is often referred to as "filling".

1. Explore test cases via --collect-only and search for test cases that combine PUSH0 and DELEGATECALL in the EVM functionality introduced in the Shanghai hard fork:

   uv run fill --collect-only -k "push0 and delegatecall" tests/shanghai/

   The fill command is based on pytest. The above command uses the optional pytest arguments:

   • --collect-only only collect test cases; don't execute them.
   • -k <expression> filter test cases by their test case ID based on the given expression.
   • tests/shanghai the directory containing the test cases (tells fill to only discover test cases in this directory; default: tests/).

   Expected console output:

2. Fill state_test fixtures for these test cases:

   uv run fill -k "push0 and delegatecall" tests/shanghai/ -m state_test -v

   where:

   • -m state_test only fills test cases marked as a state_test (see all available markers via uv run fill --markers).
   • -v enables verbose output.

   Expected console output:

3. Verify the generated fixtures:

   a. Check the corresponding fixture file has been generated:

   head fixtures/state_tests/shanghai/eip3855_push0/push0/push0_contract_during_call_contexts.json

   b. Open the generated HTML test using the link provided at the bottom of the console output. This is written to the output directory at:

   ./fixtures/.meta/report_fill.html

Usage

More information on how to obtain and consume the released test fixtures can be found in the documentation.

For further help with working with this codebase, see the online documentation:

1. Learn useful command-line flags.
2. Execute tests for features under development via the --from=FORK1 and --until=FORK2 flags.
3. Optional: Configure VS Code to auto-format Python code and execute tests within VS Code.
4. Implement a new test case, see Writing Tests.

Coverage

The available test cases can be browsed in the Test Case Reference doc.

Installation Troubleshooting

If you encounter issues during the installation process, please refer to the Installation Troubleshooting page.

Contributing

Contributions and feedback are welcome. Please see our Contributing Guidelines for detailed information on how to contribute, and the online documentation for this repository's coding standards and help on implementing new tests.

We welcome earnest newcomers, no matter how small the contribution! However, we do not accept:

• Contributions that only fix spelling or grammatical errors in documentation, code or elsewhere
• Pull requests from obvious airdrop farmers
• Drive-by or vibe code contributions without proper engagement or context

Pull requests should have reasonable substance or resolve an existing repository open issue.

Care is required when adding PRs or issues for functionality that is live on Ethereum mainnet, please refer to the Security Policy for more information about reporting vulnerabilities and eligibility for the bug bounty program.

License

This project is licensed under the MIT License - see the LICENSE file for details.