oot support

[briaguya0]

this doesn't include all of the test scaffolding i've been using. that still exists in https://github.com/briaguya0/Torch/tree/oot-simplify-addasset

i'm very happy with almost every factory in here. the one big chunk left to figure out is DisplayListFactory. i originally had a bunch of oot specific changes littered throughout that file, now i just have OoTDListHelpers that basically just reimplements it with oot specific changes.

the other thing i'd like to do is verify this against other roms, but once the dlist stuff is sorted i see no reason to wait for every rom to work before moving this out of draft

[inspectredc]

First of all this is really cool to see! I haven't gone through and tested the factories yet but left a few questions about the PR overall in the major sections (the main major blocker being the issue in the DisplayListFactory)

[inspectredc]

Would this be beneficial for us to have in the main Torch factories folder?

[briaguya0]

i'm not sure, right now it feel very tied to oot, it's just replicating

https://github.com/HarbourMasters/ZAPDTR/blob/74a8e9d92214ce9e028aa1c51c1be7622e98d200/ZAPD/ZDisplayList.cpp#L2134-L2167

i'm not sure what it would take to make this not oot-specific, and i'm not sure what other games would benefit from it

[inspectredc]

Out of curiousity, is there any benefit to having this #ifdef since my understanding is that the cmakelists already handles the exclusion of this file

[briaguya0]

definitely not needed, i was just following the LUS pattern from things like

https://github.com/Kenix3/libultraship/blob/ddac21e09ee449d8086392ad2ec7493512c0ac6a/src/fast/backends/gfx_dxgi.cpp#L1

#if defined(ENABLE_DX11) || defined(ENABLE_DX12)

https://github.com/Kenix3/libultraship/blob/ddac21e09ee449d8086392ad2ec7493512c0ac6a/src/fast/CMakeLists.txt#L28-L32

if (NOT CMAKE_SYSTEM_NAME STREQUAL "Windows")
    list(FILTER Source_Files__Graphic EXCLUDE REGEX "graphic/Fast3D/backends/gfx_dxgi*")
    list(FILTER Source_Files__Graphic EXCLUDE REGEX "graphic/Fast3D/backends/gfx_direct3d*")
    list(FILTER Source_Files__Graphic EXCLUDE REGEX "graphic/Fast3D/backends/dxsdk/*")
endif()

but since that isn't standard in Torch i'm happy to remove the file-level guards

[inspectredc]

I'm not too sure this is really necessary since the segment range doesn't go this high but it can't hurt to have

[briaguya0]

i think this is mostly a ZAPD-ism. "segment 0x80" means "this is something we read from VRAM, but it exists in a different file"

we could use a different number for "segment" in these scenarios and update

    // If addr is below vramBase, it's not pointing into this file's VRAM range.
    // It's a segmented address using segment 0x80+.
    if (addr < vramBase) {
        return { addr, ResolvedAddr::Segmented };
    }

to patch addr to replace the 0x80 at the top with some number we use to mean "virtual segment" - but using 0x80 is what ZAPD was doing so sticking to that seemed reasonable

i worked with claude to write up a summary of how this logic came to be

claude dive

Understanding ResolveVirtualAddr and IS_VIRTUAL_SEGMENT

Context

inspectredc commented on Companion.cpp:1634 saying the distinction "can't hurt" but
questioning whether it's necessary since "the segment range doesn't go this high."
We need to fully understand why this logic exists.

Background: Why segment 128 and why it's not arbitrary

Normal N64 objects use segments 2-6. The ROM data contains pointers like 0x06001234
where the top byte (0x06) is the segment number. Torch's YAML declares
segments: [6, <rom_base>], and PopulateAddrMap stores assets with keys like
(6 << 24) | offset. When a DList has w1 = 0x06001234, Torch extracts segment 6
from the top byte, looks up the ROM base, and the gAddrMap key matches. The round-trip
works because the ROM pointer's top byte matches the YAML segment number.

Overlays and code sections are different. They're loaded into the N64's VRAM at
0x80XXXXXX addresses (KSEG0). Pointers in the ROM between overlay assets use these
VRAM addresses — e.g. 0x808C7378. The top byte is 0x80 = 128. For the same
round-trip to work, the YAML must use segment 128. This isn't a ZAPD convention that
we copied blindly — it's forced by the ROM data. ZAPD happens to use the same value
for the same reason (ZFile.h:44).

The problem: VRAM pointers don't directly match file offsets

For normal objects (segment 6), the ROM pointer IS the segment+offset: 0x06001234
means segment 6, offset 0x1234. The offset directly matches what the YAML declares.

For overlays, the ROM pointer is a VRAM address: 0x808C7378. The bottom 24 bits
(0x8C7378) are NOT a file offset — they're the bottom 24 bits of a VRAM address.
The actual file offset is 0x808C7378 - vramBase(0x808C1190) = 0x61E8. Without this
subtraction, the gAddrMap key (128 << 24) | 0x61E8 = 0x800061E8 won't match the
raw pointer's bottom 24 bits (128 << 24) | 0x8C7378 = 0x808C7378.

This is what ResolveVirtualAddr solves: it subtracts the VRAM base to recover the
file offset, then reconstructs the gAddrMap key.

What main's PatchVirtualAddr did (and why it broke)

Main's version (from inspectredc's commit 5e334dc):

if (addr & 0x80000000) {
    addr -= vramBase;
    addr += physStart;
}

Two problems:

1. Wrong result format: Converts VRAM → absolute ROM address. For ovl_Boss_Dodongo:

   • Input: 0x808C7378 (VRAM pointer to texture)
   • Result: 0x808C7378 - 0x808C1190 + 0x00B69530 = 0x00B6F718 (ROM absolute)
   • But gAddrMap key for this texture is (128 << 24) | 0x61E8 = 0x800061E8
   • Lookup fails: 0x00B6F718 ≠ 0x800061E8

2. Assumes all 0x80 addresses are into the current file: An overlay DList can
   reference data in a completely different file (e.g. gMtxClear in the code segment).
   The pointer 0x800FBC20 is a VRAM address into sys_matrix's range, not this
   overlay's. Subtracting this overlay's vramBase gives garbage.

What ResolveVirtualAddr does

Fixes both problems:

1. Correct format: Subtracts vramBase to get file offset, then reconstructs
   (128 << 24) | offset — matching gAddrMap's key format.

2. Range check: Only subtracts vramBase if addr >= vramBase (meaning it actually
   points into this file). Cross-file references (addr < vramBase) are passed through
   for external file lookup.

0x80XXXXXX addresses are VRAM pointers from ROM data

There is no "segment 0x80" on the N64. All 0x80XXXXXX addresses that flow through
ResolveVirtualAddr are real VRAM pointers read from ROM data (DList commands,
skeleton pointers, etc.). The N64's KSEG0 starts at 0x80000000, so all runtime
memory addresses start with 0x80.

The addr < vramBase check distinguishes whether a VRAM pointer is into the
current file or into a different file with a lower VRAM range. Cross-file
pointers are passed through for external file lookup.

Note: this would break if a file referenced something with a higher VRAM address
(the pointer would look like it's in the current file and get the wrong subtraction).
In practice this doesn't happen — shared code/data is loaded at low VRAM addresses
before overlays, so cross-file references always point downward. Overlays don't
reference other overlays since they're loaded/unloaded dynamically.

The four cases in ResolveVirtualAddr

Case 1: addr < 0x80000000 (normal segmented or file-relative)

Example: 0x06001234 (object_dodongo VTX reference with segment 6)
Result: returned as-is, classified as Segmented
When: processing any file with normal segments (objects, scenes, rooms)

Case 2: addr >= 0x80000000, no virtual mapping for current file

Example: 0x800EA0C8 encountered in a file without :config: virtual:
Result: returned as Unknown
When: defensive — e.g. a code file like z_fbdemo_circle that has segment 128
in its YAML but no virtual: mapping. The address is already a synthetic key.

Case 3: addr >= 0x80000000, below vramBase (cross-file VRAM pointer)

Example: processing ovl_Boss_Dodongo (vramBase=0x808C1190), DList has G_MTX
pointing to gMtxClear at VRAM 0x800FBC20 (in sys_matrix's VRAM range, not this
overlay's).
0x800FBC20 < 0x808C1190 → this VRAM pointer is NOT into our overlay's data.
Returned as { 0x800FBC20, Segmented }. GetNodeByAddr then searches external files
and resolves it using sys_matrix's virtual mapping:
0x800FBC20 - 0x80010F00 = 0xEAD20 → key (128 << 24) | 0xEAD20 = 0x800EAD20
→ finds gMtxClear.

Case 4: addr >= vramBase (VRAM pointer into current file)

Example: processing ovl_Boss_Dodongo, DList references texture at 0x808C7378

• 0x808C7378 >= 0x808C1190 → this IS a VRAM pointer into our overlay
• relOffset = 0x808C7378 - 0x808C1190 = 0x61E8
• Finds segment 128 in config → returns (128 << 24) | 0x61E8 = 0x800061E8
• Matches gAddrMap key → lookup succeeds

Concrete files

File	Segment	BaseAddress/virtual	VRAM pointers in ROM?
object_dodongo	6	none	No — uses 0x06XXXXXX
ovl_Boss_Dodongo	128 (default)	0x808C1190	Yes — 0x808CXXXX
z_fbdemo_circle	128 (default)	none (= 0)	Yes — 0x800XXXXX
sys_matrix	128 (default)	0x80010F00	Yes — 0x8001XXXX

Why IS_VIRTUAL_SEGMENT exists

IS_VIRTUAL_SEGMENT identifies addresses with the top byte >= 0x80 — i.e. N64 VRAM
addresses. It's used as a fallback in DList export: when a 0x80XXXXXX pointer
couldn't be resolved by any lookup, IS_VIRTUAL_SEGMENT catches it and writes null
vtxDecl (matching OTRExporter's behavior). Without it, the cross-segment fallback
would write (w1 & 0x0FFFFFFF) + 1 instead of 0, producing wrong bytes.

Concrete example: sCircleDList in z_fbdemo_circle references VTX via VRAM
address 0x800FB168. This file has no virtual mapping, so the address can't be
resolved. IS_VIRTUAL_SEGMENT catches it → null. Without the check, it would fall
through to the cross-segment fallback and write 0x000FB169 instead of 0x00000000.

Also used in:

• ASSET_PTR macro: extract offset from both regular segments and VRAM addresses
• Decompressor::TranslateAddr: route VRAM addresses to the segment table

Where 0x80XXXXXX addresses appear in ROM data

1. Overlay → same overlay (resolved by ResolveVirtualAddr)
ovl_Boss_Dodongo DList → sLavaFloorLavaTex: w1 = 0x808C7378
ResolveVirtualAddr: 0x808C7378 - vramBase(0x808C1190) = 0x61E8 → lookup succeeds

2. Overlay → different file (resolved by cross-file lookup)
ovl_Boss_Dodongo DList → gMtxClear: w1 = 0x800FBC20
Below this overlay's vramBase → passed to external file lookup →
sys_matrix resolves: 0x800FBC20 - 0x80010F00 = 0xEAD20 → lookup succeeds

3. Code section without virtual mapping (caught by IS_VIRTUAL_SEGMENT)
z_fbdemo_circle DList → sTransCircleVtx: w1 = 0x800FB168
No virtual mapping → PatchVirtualAddr returns unchanged → all lookups fail →
IS_VIRTUAL_SEGMENT(0x800FB168) = true → write null (matches OTRExporter)

Why 128 for the YAML segment number

N64 VRAM (KSEG0) starts at 0x80000000 — top byte is 0x80 = 128. Torch stores
overlay assets in gAddrMap with keys (128 << 24) | file_offset. This choice
doesn't add complexity — even with a different number, we'd still need:

• ResolveVirtualAddr to subtract vramBase from ROM VRAM pointers
• IS_VIRTUAL_SEGMENT to catch unresolvable VRAM addresses
• Cross-file VRAM resolution

128 follows ZAPD's convention (ZFile.h:44 segment = 0x80) and avoids colliding
with real N64 segments (0-15).

[inspectredc]

Perhaps we should just make this a general setting we add to the config as I think this would make sense based on project's own preferences

[briaguya0]

yeah, my original plan was to just remove this before PRing but i forgot to (i was using it when trying to improve o2r generation speed because dynamically adding things slowed it down a ton)

the config setting seems like a great idea!

[inspectredc]

May have to require some decomps/ports to check this doesn't break any checks they make

[briaguya0]

did some investigation into this here 8f06c36

as far as ports go, it seems ghostship and spaghettikart are fine, but starship will need to update the SF64_VER constants.

i didn't check decomps, good call

[inspectredc]

Does this break without OOT_SUPPORT? And would using Torch's own VTX factory remove the need for this?

[briaguya0]

Does this break without OOT_SUPPORT?

yes. good catch!

would using Torch's own VTX factory remove the need for this?

it wouldn't be a drop-in replacement. SearchVtx handles both VTX and OOT:ARRAY (of type VTX), and handles oot alias segments

i think some of the logic is inherently oot-specific, but some could probably be generalized. i'm not sure what the most logical way to split it up would be

[inspectredc]

Just a quick question about this system, is this a way for one file's data to be written to multiple locations in the output binary, and if so what is the use case for it?

[briaguya0]

is this a way for one file's data to be written to multiple locations in the output binary

yes

what is the use case for it?

documented here briaguya0@86964d9 in the "Why aliases exist" section

DLists embed a CRC64 of their own output path ("bhash"). Set_ DLists need to be byte-identical copies of the base DList (same bhash). Re-parsing the DList under a Set_ path produces a different bhash → binary mismatch. Aliases are copies,
not independent assets.

zapd handles this by reading the same data but exporting it to different paths (happens in SetMesh)

we can't do that because DListBinaryExporter::Export adds a bhash based on the path provided as replacement, so dlists exported because they're referenced by bdan_room_0Set_0000E0 would end up with different bhash values than the ones referenced by bdan_room_0, even though they are supposed to be referencing the exact same file (just at different paths).

it's possible we can get away with them having different bhash values and this is only needed to byte-match zapd generated o2rs, but i'd need to do some digging to be sure