fix(fill): insist fill's output directory is empty or doesn't exist

docs/CHANGELOG.md

@@ -6,6 +6,13 @@ Test fixtures for use by clients are available for each release on the [Github r
 
 ## 🔜 [Unreleased]
 
+### 💥 Important Change for `fill` Users
+
+The output behavior of `fill` has changed:
+
+- Before: `fill` wrote fixtures into the directory specified by the `--output` flag (default: `fixtures`). This could have many unintended consequences, including unexpected errors if old or invalid fixtures existed in the directory (for details see [#1030](https://github.com/ethereum/execution-spec-tests/issues/1030)).
+- Now: `fill` will exit without filling any tests if the specified directory exists and is not-empty. This may be overridden by adding the `--clean` flag, which will first remove the specified directory.
+
 ### 💥 Breaking Change
 
 ### 🛠️ Framework
@@ -15,6 +22,8 @@ Test fixtures for use by clients are available for each release on the [Github r
 - ✨ The `static_filler` plug-in now has support for static state tests (from [GeneralStateTests](https://github.com/ethereum/tests/tree/develop/src/GeneralStateTestsFiller)) ([#1362](https://github.com/ethereum/execution-spec-tests/pull/1362)).
 - ✨ Introduce `pytest.mark.exception_test` to mark tests that contain an invalid transaction or block ([#1436](https://github.com/ethereum/execution-spec-tests/pull/1436)).
 - 🐞 Fix `DeprecationWarning: Pickle, copy, and deepcopy support will be removed from itertools in Python 3.14.` by avoiding use `itertools` object in the spec `BaseTest` pydantic model ([#1414](https://github.com/ethereum/execution-spec-tests/pull/1414)).
+- 🔀 Refactor: Encapsulate `fill`'s fixture output options (`--output`, `--flat-output`, `--single-fixture-per-file`) into a `FixtureOutput` class ([#1471](https://github.com/ethereum/execution-spec-tests/pull/1471)).
+- 🐞 Do not write generated fixtures into an existing, non-empty output directory; it must now be empty or `--clean` must be used to delete it first ([#1473](https://github.com/ethereum/execution-spec-tests/pull/1473)).
 
 #### `consume`
 

src/cli/gentest/tests/test_cli.py

@@ -1,5 +1,7 @@
 """Tests for the gentest CLI command."""
 
+from tempfile import TemporaryDirectory
+
 import pytest
 from click.testing import CliRunner
 
@@ -111,5 +113,13 @@ def get_mock_context(self: StateTestProvider) -> dict:
     assert gentest_result.exit_code == 0
 
     ## Fill ##
-    fill_result = runner.invoke(fill, ["-c", "pytest.ini", "--skip-evm-dump", output_file])
+    args = [
+        "-c",
+        "pytest.ini",
+        "--skip-evm-dump",
+        "--output",
+        TemporaryDirectory().name,
+        output_file,
+    ]
+    fill_result = runner.invoke(fill, args)
     assert fill_result.exit_code == 0, f"Fill command failed:\n{fill_result.output}"