Skip to content

Keep the newer-mapper probe alive when the fetched mapper has no fp8 tables - #7497

Merged
danielhanchen merged 2 commits into
mainfrom
fix-newer-mapper-probe-fp8-keyerror
Jul 27, 2026
Merged

Keep the newer-mapper probe alive when the fetched mapper has no fp8 tables#7497
danielhanchen merged 2 commits into
mainfrom
fix-newer-mapper-probe-fp8-keyerror

Conversation

@danielhanchen

Copy link
Copy Markdown
Member

Follow-up to #7478.

_get_new_mapper reads the two fp8 tables out of the fetched mapper.py under that file's own names, unlike the three NEW_ names it renames itself. A mapper.py that does not define them raises KeyError, the bare except swallows it, and the function returns five empty dicts, so the 4bit and 16bit upgrade check stops firing too. That check is the reason the probe exists, and it is the half that has nothing to do with fp8.

The probe fetches from main at runtime, so the file it gets is not under the installed version's control. Every mapper.py older than the fp8 tables is one that does not define them. Serving each version that has actually existed in this repo, and asking whether the 4bit upgrade check still works:

fetched mapper.py date before #7478 after #7478 this PR
d6bb89a (predates the fp8 tables) 2025-11-07 alive dead [0, 0, 0, 0, 0] alive [400, 997, 591, 0, 0]
86f7080 2025-11-25 alive alive alive
f840119 2026-03-03 alive alive alive
800ddc9 2026-04-15 alive alive alive
current main 2026-07-27 alive alive alive

Before #7478 this degraded gracefully, because a missing name was only ever a NameError on a name the code did not need. Reading the two names with .get restores that: the 4bit half keeps working and only the fp8 half goes empty, which costs nothing real, since the probe runs only after the installed tables have already missed.

Test

tests/test_new_mapper_fetched_fp8.py, two cases:

  1. a fetched mapper.py with no fp8 tables must not take the 4bit half down. Fails on current main with assert ({}), passes here.
  2. an fp8 repo that only the fetched mapper knows must still raise the upgrade NotImplementedError, and answering it must not adopt the fetched tables.

The second case covers a gap in test_new_mapper_no_global_leak.py: that file serves the repo's own mapper.py as both the installed and the fetched source, so fp8_block is not block is satisfied by any fresh dict, since exec always allocates new ones. It pins allocation, not provenance. A _get_new_mapper that returns the fetched fp8 tables filtered down to keys the installed tables already have, which is functionally a revert of 881180f, passes that file 2/2. This test splices an extra "8" entry into the fetched source only, and stubs importlib so vllm looks absent, since from 0.12.0 on __get_model_name returns the original name on the first pass and the fp8 path is never reached.

Against the three relevant revisions:

revision result
before #7478 fails, installed table was rebound
797f5e2 (isolate only) fails, DID NOT RAISE
current main fails case 1
this PR passes
tests/test_new_mapper_fetched_fp8.py::test_probe_answers_for_an_fp8_repo_only_the_fetched_mapper_knows PASSED
tests/test_new_mapper_fetched_fp8.py::test_probe_survives_a_fetched_mapper_without_the_fp8_tables PASSED
tests/test_new_mapper_no_global_leak.py .. PASSED
tests/test_bad_mappings_redirect.py . PASSED
tests/test_gemma_2b_mapper_key.py . PASSED
tests/test_get_model_name.py .. PASSED (40 subtests)

Full CPU suite: 2940 passed, 56 skipped. The test is CPU-only and makes no network calls: it ast-extracts the resolvers and stubs requests with the repo's own mapper.py, matching tests/test_bad_mappings_redirect.py.

…tables

_get_new_mapper reads the two fp8 tables out of the fetched mapper.py under
that file's own names, unlike the three NEW_ names it renames itself. A
mapper.py that does not define them raises KeyError, the bare except swallows
it, and the function returns five empty dicts, so the 4bit and 16bit upgrade
check stops firing as well. That check is the reason the probe exists.

Every mapper.py older than the fp8 tables is such a file: fetching the
2025-11-07 one leaves the probe with [0, 0, 0, 0, 0] instead of
[400, 997, 591]. Reading the two names with .get keeps the 4bit half working
and empties only the fp8 half, which costs nothing, since the probe runs only
after the installed tables have already missed.

Add a regression test that also pins the fetched-only fp8 upgrade error, which
the existing test cannot catch: it serves the repo's own mapper.py as both the
installed and the fetched source, so any fresh dict satisfies its identity
assertions.
@danielhanchen
danielhanchen requested a review from Datta0 as a code owner July 27, 2026 11:42
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@danielhanchen
danielhanchen merged commit 032550d into main Jul 27, 2026
51 checks passed
@danielhanchen
danielhanchen deleted the fix-newer-mapper-probe-fp8-keyerror branch July 27, 2026 12:01
danielhanchen added a commit to vineethsaivs/unsloth that referenced this pull request Jul 28, 2026
The two tests this PR originally added were already covered by
tests/test_new_mapper_fetched_fp8.py from unslothai#7497. An 8-mutant matrix over
loader_utils.py found nothing they caught that the existing file did not, so
they are dropped and test_new_mapper_no_global_leak.py goes back to main.

Two real gaps were open, both on the row branch that load_in_fp8 = True plus
UNSLOTH_HAS_FBGEMM selects ahead of block:

- the FBGEMM row branch in __get_model_name could be deleted outright with
  every test still green
- _resolve_with_mappers could ignore its fp8_row argument and silently fall
  back to the installed row table

Adds two tests to the existing file, reusing its _load_resolver rather than a
second harness. The row-only fixture splices into the fetched row table alone,
since an entry the block table also knows lets the block branch answer and
masks the regression.
danielhanchen added a commit that referenced this pull request Jul 28, 2026
* Pin the newer-mapper FP8 probe with tests that can fail

The two identity assertions added in #7478 compare the returned FP8 tables
against the installed ones, but the fixture serves the same mapper.py as both
the installed and the fetched source and exec always allocates fresh dicts, so
they pin allocation rather than provenance and hold for any new dict.

Replace them with two tests that drive get_model_name end to end: one splices an
FP8 entry into the fetched source only and asserts the upgrade error still fires,
the other serves a mapper.py with no FP8 tables and asserts the 4bit half of the
probe survives, which is the regression #7497 fixed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten the resolver stub for PR #7516

- Restore the fp8_block/fp8_row identity assert alongside the new provenance
  test. It is weak, not vacuous: it still catches a probe that hands back the
  installed table objects, and it costs nothing to keep.
- Bind Version and transformers_version in the stub namespace. Both are
  unreached under the current gates, so a change to either would fail with a
  bare NameError instead of the assertion.

Merged main, which clears the unrelated test_runtime_text_encoding failure the
branch inherited from its base.

* Cover the FP8 row-scaling path instead of duplicating the block one

The two tests this PR originally added were already covered by
tests/test_new_mapper_fetched_fp8.py from #7497. An 8-mutant matrix over
loader_utils.py found nothing they caught that the existing file did not, so
they are dropped and test_new_mapper_no_global_leak.py goes back to main.

Two real gaps were open, both on the row branch that load_in_fp8 = True plus
UNSLOTH_HAS_FBGEMM selects ahead of block:

- the FBGEMM row branch in __get_model_name could be deleted outright with
  every test still green
- _resolve_with_mappers could ignore its fp8_row argument and silently fall
  back to the installed row table

Adds two tests to the existing file, reusing its _load_resolver rather than a
second harness. The row-only fixture splices into the fetched row table alone,
since an entry the block table also knows lets the block branch answer and
masks the regression.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant