Standardize opcode tracer behavior for debug_traceBlockByNumber and debug_traceTransaction

[MysticRyuujin]

Summary

This PR adds OpenRPC method and schema definitions for:

• debug_traceTransaction
• debug_traceBlockByNumber
• debug_traceBlockByHash

Scope is intentionally limited to the default opcode (struct) logger output, while still allowing named tracers through TraceConfig.tracer.

It is also based on the investigation done by ethPandaOps on trace-comparisons

Goal

The goal of this PR is to standardize the existing live debug_trace* opcode tracer contract as one canonical cross-client output:

• preserve the widely used geth-shaped RPC surface
• define clear field presence/absence rules
• define clear encoding rules for stack, memory, storage, and return data
• define exact semantics for when fields appear
• add conformance tests that drive clients toward the same default opcode tracer response

Non-goals

This PR does not:

• attempt literal EIP-3155 conformance
• redesign the trace format from scratch
• standardize named tracer result schemas

If the ecosystem wants a cleaner or more opinionated trace format than the current live RPC methods provide, that should be a separate effort.

Preserve vs normalize

Preserved from current live RPC practice

• top-level opcode trace shape: gas, failed, returnValue, structLogs
• op as a human-readable opcode name string
• gas and gasCost as JSON integers
• memory as chunked 32-byte words
• named tracers remain supported but out of scope here

Standardized by this PR

• stack values use canonical 0x-prefixed uint256 quantities
• memory words use 0x-prefixed bytes32
• storage keys and values use 0x-prefixed bytes32
• error must be absent when no error occurred
• returnData must be absent when disabled and valid hex when enabled
• EOA-to-EOA transfers must return structLogs: []
• block traces must return ordered { txHash, result } entries
• storage snapshots must be emitted only at SLOAD and SSTORE

Relationship to EIP-3155 / EIP-7756

This PR borrows the useful parts of the EIP work, but it does not adopt those specs literally.

The main intentional differences are:

• keep geth-style top-level field names instead of output / gasUsed / pass
• keep integer gas / gasCost
• keep op as a string name
• keep memory as chunked words instead of a single blob
• leave stateRoot, time, fork, memSize, and named tracer output schemas out of scope

Key field-level differences

Area	EIP-3155 / EIP-7756	This PR
Top-level summary fields	output, gasUsed, pass	returnValue, gas, failed
op	opcode byte, optionally with opName	opcode name string only
gas / gasCost	hex string or number, depending on EIP/version	JSON integers
memory	contiguous blob	chunked bytes32[]
Additional fields	may include stateRoot, time, fork, memSize	out of scope here

Storage semantics

This PR standardizes storage behavior as part of the canonical output:

• storage is a cumulative snapshot
• storage keys and values are 0x-prefixed bytes32
• storage is emitted only at SLOAD and SSTORE
• storage must be absent, not null or {}, at all other opcodes

Likely client changes

Based on the current tests, the existing PR investigation, and the ethPandaOps comparison work, the most likely alignment work is:

Client	Likely changes to align
Geth	Add 0x prefix to memory chunks and storage keys/values in opcode traces
Erigon	Verify memory/storage encoding and block-trace txHash pairing match the spec
Besu	Align stack/memory/storage encoding and field presence behavior with the canonical trace shape
Nethermind	Omit empty error / storage fields where required and align storage timing/output
Reth	Align returnData gating and verify storage timing/encoding behavior

This table is a concrete starting point for client review, not a claim of fully validated implementation diffs across every client version.

Tests

This PR adds focused tests for:

• debug_traceTransaction
  • EOA transfer returns structLogs: []
  • contract call trace shape is spec-compliant
  • unknown transaction returns an error
• debug_traceBlockByNumber
  • ordered per-transaction { txHash, result } responses
  • memory encoding
  • storage encoding
  • returnData gating behavior
  • genesis trace error
  • invalid block number error
• debug_traceBlockByHash
  • basic block with transactions
  • genesis trace error
  • invalid block hash error

Most large trace fixtures are SpecOnly, meaning they validate the standardized wire contract rather than replaying a recorded byte-for-byte dump from one client.

That is intentional: the purpose of this PR is to standardize one canonical response shape and semantics for the default opcode tracer, while avoiding overfitting the tests to incidental formatting outside the specified contract.

Recorded fixtures are still used where the expected behavior is small and unambiguous, such as:

• unknown transaction errors
• genesis trace errors
• invalid block number errors

Coverage still incomplete

Current coverage still does not fully prove:

• refund presence when non-zero
• broader storage snapshot timing patterns beyond the focused cases in this PR
• some revert/failure-path variants where clients have historically diverged

The proposal here is still to define one canonical target; this section simply identifies the areas where additional fixtures or client feedback may still be useful.

Current validation status

• make build: pass
• go test ./... in tools/: pass
• make fill: fail (need go-ethereum to conform to the spec as written here)- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

[chfast]

This is not well specified. Does it mean the amount of gas to be burned by successful execution of the instruction?

[MysticRyuujin]

I updated the description to reflect the intent, though I'm not 100% sure if this wording is correct? I think it is.

[MysticRyuujin]

In geth, OnOpcode is called before the opcode executes, receiving the pre-computed cost as a parameter. gasCost is that value — the amount that will be deducted from the remaining gas assuming successful execution, including dynamic costs such as memory expansion and cold/warm storage access charges (EIP-2929). So yeah, I think the updated description is accurate.

[chfast]

I think the main issue is in CALL instructions. I think gasCost should exclude the amount of gas passed down to the callee and only report the amount being burned as the instruction cost. This way the sum of all gasCost entries should give you the final gas used amount. cc @MariusVanDerWijden.

[alexb5dh]

Idea about sum being equal to the final used amount looks interesting, but want to highlight a few points:

• For formula to hold CALL gasCost won't include any "execution" gas, as it will already be accounted for in "inner" opcodes. Only base call gas, memory expansion, value transfer, and account creation fees. This may be a bit counterintuitive.
• From a brief check none of the top clients have such exact implementation, so this will require a change from all of them. Also most clients take full forwarded gas as gasCost (without deducting amount returned by the callee). May be wrong here so.

[MysticRyuujin]

This could be updated to support EIP-7708 if it ever actually makes it into a fork 😅

[alexb5dh]

No corcerns from Nethermind side.
Just not sure about reexec being in the standard, as doesn't seem like it's needed or widely used.

[canepat]

Just not sure about reexec being in the standard, as doesn't seem like it's needed or widely used.

+1 from Erigon

[0000c-0c]

run

all ok

[mattsse]

no concerns from reth

[macfarla]

the execution timeout is non-trivial to implement in Besu. would prefer this to be a nice to have rather than a must-have. everything else is fine.

[macfarla]

medium effort to implement this in Besu since we don't have this currently

[s1na]

probably also an error for "block not found" and parent not found

[MysticRyuujin]

Same comment as here

[MysticRyuujin]

go-ethereum PR was merged but there is no release, go.mod needs to be updated

[daniellehrner]

When stack, storage and memory is enabled, we've seen block traces on mainnet which were bigger than 47GB and that was even without padding. I think we should try to reduce the trace size as much as possible and not do any padding

[s1na]

Isn't the padding in practice only for the last word, which may be partial? The overhead seems low to me.

[daniellehrner]

Similar to storage we should only populate this for op codes that touch memory to reduce the trace size

[daniellehrner]

Looking through the Besu codebase the following op codes touch memory:

MLOAD, KECCAK256, LOG0–LOG4, RETURN, REVERT, MSIZE, MSTORE, MSTORE8, CALLDATACOPY, CODECOPY, EXTCODECOPY, RETURNDATACOPY, CALL, CALLCODE, DELEGATECALL, STATICCALL, CREATE, CREATE2, MCOPY

[s1na]

We discussed a similar idea which didn't end up happening because streaming traces alleviated some of the issue.

It is the same in spirit as to what you're suggesting with the difference that we focus on opcodes that change the memory (and anytime you enter a new scope) and at that point output the full memory.

[ahamlat]

I just run tests on debug_traceBlockByNumber with memory tracing enabled, on mainnet, and saw a block that generated 643 GiB of json output :

0x17AEF7D        658854.28MB

this is just to show case that we need to optimize memory tracing, and to support @daniellehrner's idea.

[chfast]

Maybe it's not worth to standardize this part then.

[ahamlat]

I think it makes even more sense to standardize it to be sure that debug_trace* calls generate reasonable amount of data. So basically, we need to reduce the impact of memory tracing on the json output size.

[s1na]

I'd like to point out that: 1) it is disabled by default and won't bloat the trace, 2) there are few use-cases that require memory.

That said, I fully support optimizing the impact of memory, either in this or a future PR. I'd propose a more granular flag than enableMemory for it. An enum which can support a range of values like none, full, onChange. Because there are multiple ways to optimise it, you can even go as far as only outputting the diff compared to last change.

There is a way to do this in a backwards compatible way too. If we pick full as the default value, then when enableMemory is toggled it behaves the same way it does today.

[jochem-brouwer]

One could optimize the onChange even further by only doing this on operations which cannot be deduced from the already known output of an evm trace with only the stack enabled. By re-execution one can now deduce how the memory looks like. The only exception here is if EXTCODECOPY/CODECOPY is being used, which requires access to the code.

Maybe I missed an edge case but I think for all other opcodes this can directly be deduced by re-execution using the known stack values. Also RETURNDATACOPY for precompiles would be able to be recalculated because we can infer the input, and therefore also the output.

I think that in current EVM the memory trace only has to consist of the code bytes accessed during the trace. This only has to be stored once. We only need the copied bytes and not the remainder of the code, so we do not have to include the full code for every EXTCODECOPY/CODECOPY hit (how to format these partial codes is a different question, if one copies all odd bytes of a contract this would likely lead to a lot of overhead bytes for the encoding of this).

This would obviously need tooling in order to help reconstruct the actual memory layouts (I think reusing the own EVM implementation works best here), but it would massively decrease the memory trace output. If the trace provides every state necessary for the trace to be re-run we are done. For txs as these do not have more than 16.7 / 2 ** 24 gas limit this is also a manageable size for each transaction (and one can calculate the upper limit of certain entries to the trace, like the amount of addresses with code, or the amount of storage slots)

[s1na]

Memory bloating the trace is a huge issue which we've been facing and we can and should improve on it. That said, I think this can be done in a backwards-compatible way, by the way of adding a memoryMode: enum config. It would configure how exactly memory will be emitted if it is enabled via enableMemory. Whether on every opcode, or on some opcodes, or a diff of memory, etc.

I'd like to propose that we still move forward with this PR in the spirit of making progress and not holding out on everything else which has consensus. Then directly make a future PR which standardises and adds support for lightweight memory traces.