chores(consume): rename `--bin` flag of consume direct to `--evm-bin` to be consistent with fill's `--evm-bin` by felix314159 · Pull Request #2548 · ethereum/execution-specs

felix314159 · 2026-03-24T11:13:22Z

🗒️ Description

Renames consume direct's --bin flag to --evm-bin so the binary flag is consistent with fill --evm-bin, and it removes stale references to ethereum-spec-evm-resolver now that the weld is completed.

🔗 Related Issues or PRs

N/A.

✅ Checklist

All: Ran fast tox checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:
```
uvx tox -e static
```
All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
All: Considered updating the online docs in the ./docs/ directory.
All: Set appropriate labels for the changes (only maintainers can apply labels).
Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.
Tests: For PRs implementing a missed test case, update the post-mortem document to add an entry the list.
Ported Tests: All converted JSON/YML tests from ethereum/tests or tests/static have been assigned @ported_from marker.

Cute Animal Picture

codecov · 2026-03-24T11:22:48Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.35%. Comparing base (65a3009) to head (1c20ec3).
⚠️ Report is 1 commits behind head on forks/amsterdam.

Additional details and impacted files

@@               Coverage Diff                @@
##           forks/amsterdam    #2548   +/-   ##
================================================
  Coverage            86.35%   86.35%           
================================================
  Files                  599      599           
  Lines                36904    36904           
  Branches              3771     3771           
================================================
  Hits                 31868    31868           
  Misses                4485     4485           
  Partials               551      551

Flag	Coverage Δ
unittests	`86.35% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

danceratopz

@felix314159 could you pull the cli flag change out into a dedicated PR (but see comment below), these changes are too orthogonal respectively contentious for a single PR imo.

About the flag change, I know the difference is mildly annoying, but this was a deliberate choice in consume as it can run tests that are don't strictly require an EVM binary. I must admit that the only example I can think of right now is eoftest which validated EOF bytecode without EVM execution. But I think I'd still rather keep it more general at --bin. If anything, fill's --evm-bin flag is slightly too verbose (it was too tightly coupled to passing geth's evm binary). I think would prefer to leave consume as-is, but you could add an additional a --bin alias for --evm-bin to fill if helps things feel more consistent?

danceratopz

Here's a review on the ethereum-spec-evm-resolver to ethereum-spec-evm fix-ups! Good catch!

danceratopz · 2026-03-27T09:40:27Z

packages/testing/src/execution_testing/client_clis/tests/test_execution_specs.py


 CURRENT_FOLDER = Path(realpath(__file__)).parent
 FIXTURES_ROOT = CURRENT_FOLDER / "fixtures"
-DEFAULT_EVM_T8N_BINARY_NAME = "ethereum-spec-evm-resolver"


Def worth cleaning up here, but I think we could completely remove this global and the test that uses it. The test is marked as skip, so essentially deadcode atm. The other option is to parametrize the test with other detected t8n tools? But I consider that overkill as we're not missing this test right now and it's implicitly system-tested via the benchchmark tests which use evmone-t8n/geth.

danceratopz · 2026-03-27T09:44:50Z

packages/testing/src/execution_testing/client_clis/tests/test_execution_specs.py

-    This is required in order for fill to locate the ethereum-spec-evm-resolver
-    "binary" (entrypoint) when being executed using pytester.
+    This is required so pytester-based invocations can locate the
+    execution-specs transition tool entrypoint while running in tests.


turbo nit sorry

Suggested change

execution-specs transition tool entrypoint while running in tests.

`ethereum-spec-evm t8n` entrypoint while running in tests.

danceratopz · 2026-03-27T09:46:32Z

packages/testing/src/execution_testing/client_clis/tests/test_execution_specs.py

This is the skip I mean. I think we can entirely remove this test.

danceratopz · 2026-03-27T10:48:16Z

Just to add that I didn't manage to finish reviewing the main part, but it looks to make makes sense. I'd like to check whether geth's behavior changed here (intentionally)? Will come back to this. Regarding this, assuming this is the main feature of this PR, wouldn't the title be: fix(test-fill): fix filling pre-Byzantium forks with geth?

felix314159 · 2026-03-27T11:17:52Z

the PR started with me wanting to change the consume direct flag and then evolved to a ethereum-spec-evm-resolver cleanup and then evolved to a fix filling pre-Byzantium forks with geth PR haha. i can split it

danceratopz · 2026-03-27T13:05:19Z

I'd like to check whether geth's behavior changed here (intentionally)?

Nothing changed on the geth side. I'm just surprised that we didn't spot this earlier. Perhaps we had a regression on the testing framework side. Thanks for finding and fixing it!

…direct), remove stale references to ethereum-spec-evm-resolver

Co-authored-by: danceratopz <danceratopz@gmail.com>

felix314159 · 2026-03-27T13:55:23Z

I moved the geth fix to a different PR, so this review should be straight-forward (just ignore my branch name)

spencer-tb

I like the change. LGTM! Approved the geth tx reciepts PR too.

felix314159 · 2026-03-30T10:41:26Z

@danceratopz can you 'resolve the threads' and make the 'requested change' thing disappear if you agree with how i implemented them?

danceratopz · 2026-03-30T10:54:00Z

@danceratopz can you 'resolve the threads' and make the 'requested change' thing disappear if you agree with how i implemented them?

As per #2548 (review) above, I would still prefer to:

add --bin as an alias to fill (in addition to --evm-bin) and additionally change all doc refs for fill from --evm-bin to --bin.
leave consume as-is.

@felix314159 wdyt?

spencer-tb · 2026-03-30T10:59:29Z

@danceratopz can you 'resolve the threads' and make the 'requested change' thing disappear if you agree with how i implemented them?

As per #2548 (review) above, I would still prefer to:
* add `--bin` as an alias to `fill` (in addition to `--evm-bin`) and additionally change all doc refs for `fill` from `--evm-bin` to `--bin`.

* leave `consume` as-is.
@felix314159 wdyt?

In hindsight, I think this is the way. --evm-bin I feel we got from geth terminology when EEST started, and is maybe too verbose. Having --bin feels more general, and better suited to client statetest/blocktest runner names

felix314159 · 2026-03-30T12:28:33Z

I would like there to be one flag, and I don't care whether it is --bin for both or --evm-bin for both, but I don't want to different flags that are the same (no alias)

spencer-tb · 2026-03-30T12:33:19Z

Happy to use --bin if we add a deprecation message for people when they use --evm-bin with fill

danceratopz · 2026-03-30T13:21:52Z

Happy to use --bin if we add a deprecation message for people when they use --evm-bin with fill

Yup, I echo this. I don't see the need to make a sudden breaking change for everyone. So alias, with deprecation message if you consider important (for me leaving an alias indefinitely is also ok).

Alongside the doc changes, we should also fix-up CI.

danceratopz self-assigned this Mar 25, 2026

danceratopz self-requested a review March 25, 2026 16:36

danceratopz requested changes Mar 27, 2026

View reviewed changes

danceratopz reviewed Mar 27, 2026

View reviewed changes

felix314159 and others added 2 commits March 27, 2026 14:46

chores: rename --bin to --evm-bin (now the same for fill and consume …

1e1e9ce

…direct), remove stale references to ethereum-spec-evm-resolver

fix: address feedback

9343545

Co-authored-by: danceratopz <danceratopz@gmail.com>

felix314159 force-pushed the fix-pre-byz-consume-direct-geth branch from 89330b0 to 9343545 Compare March 27, 2026 13:51

felix314159 changed the title ~~fix(src): Fix pre-byz consume direct for geth + chores~~ chores(consume): rename --bin flag of consume direct to --evm-bin to be consistent with fill's --evm-bin Mar 27, 2026

fix: ruff

1c20ec3

felix314159 requested a review from danceratopz March 27, 2026 13:55

felix314159 mentioned this pull request Mar 27, 2026

fix(test-fill): fix filling pre-Byzantium forks with geth #2576

Merged

7 tasks

spencer-tb approved these changes Mar 30, 2026

View reviewed changes

	execution-specs transition tool entrypoint while running in tests.
	`ethereum-spec-evm t8n` entrypoint while running in tests.

Conversation

felix314159 commented Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🗒️ Description

🔗 Related Issues or PRs

✅ Checklist

Cute Animal Picture

Uh oh!

codecov bot commented Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

danceratopz left a comment

Choose a reason for hiding this comment

Uh oh!

danceratopz left a comment

Choose a reason for hiding this comment

Uh oh!

danceratopz Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

danceratopz Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danceratopz Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

danceratopz commented Mar 27, 2026

Uh oh!

felix314159 commented Mar 27, 2026

Uh oh!

danceratopz commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

felix314159 commented Mar 27, 2026

Uh oh!

spencer-tb left a comment

Choose a reason for hiding this comment

Uh oh!

felix314159 commented Mar 30, 2026

Uh oh!

danceratopz commented Mar 30, 2026

Uh oh!

spencer-tb commented Mar 30, 2026

Uh oh!

felix314159 commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

spencer-tb commented Mar 30, 2026

Uh oh!

danceratopz commented Mar 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

felix314159 commented Mar 24, 2026 •

edited

Loading

codecov bot commented Mar 24, 2026 •

edited

Loading

danceratopz Mar 27, 2026 •

edited

Loading

danceratopz commented Mar 27, 2026 •

edited

Loading

felix314159 commented Mar 30, 2026 •

edited

Loading