Import bitsandbytes before the hardware spoof rewrites torch

[danielhanchen]

Repo tests (CPU) is red on main for two independent reasons. #7468 covers one. This is the other: all 12 cases in tests/studio/install/test_rocm_rdna_routing.py error out, so the job exits non-zero even with zero test failures.

OSError: libhipblas.so.2: cannot open shared object file: No such file or directory
AttributeError: module 'torch._C' has no attribute '_cuda_getCurrentRawStream'

Why

tests/_zoo_rocm_spoof.py presents torch as a Radeon card so the hip routing paths are testable without AMD hardware. That flips torch.cuda.is_available() to True and sets torch.version.hip. bitsandbytes gates its backend on exactly that, in bitsandbytes/__init__.py:

if torch.cuda.is_available():
    from .backends.cuda import ops as cuda_ops

and backends/cuda/ops.py reads torch._C._cuda_getCurrentRawStream at module scope. CI installs a CPU-only torch (--index-url https://download.pytorch.org/whl/cpu), where that attribute does not exist. So a bitsandbytes imported after the spoof walks into a backend that cannot work.

It gets imported after the spoof because unsloth_zoo pulls it in eagerly (temporary_patches/__init__ -> gpt_oss -> moe_utils), and the test imports unsloth_zoo on the next line. That import is guarded, but only by except ImportError, and what comes back here is OSError and AttributeError.

Change

Import bitsandbytes inside the spoof's apply(), while is_available() is still False, so the CPU path is resolved and cached in sys.modules before torch is rewritten. It sits in the shared _zoo_aggressive_cuda_spoof.apply() ahead of the first mutation and inside the existing idempotence guard, so the ROCm spoof that layers on top is covered by the same three lines. Wrapped in except Exception so a host with no bitsandbytes at all is unaffected.

Verification

Reproduced in a uv venv matching the CI install shape (Python 3.12, torch>=2.4,<2.11 from the CPU index, bitsandbytes>=0.45 from PyPI, unsloth_zoo from git main). Confirmed CPU-only: torch 2.10.0+cpu, hasattr(torch._C, "_cuda_getCurrentRawStream") == False. The spawned child fails with the identical AttributeError before the change and returns its RESULT line after.

tests/studio/install, same venv, before and after:

	failed	passed	errors
before	3	1399	12
after	3	1411	0

The 3 failures are the same ones on both sides (test_managed_node_runtime, unrelated to this change and to bitsandbytes). test_rocm_rdna_routing.py on its own goes from 11 passed + 12 errors to 23 passed. test_rocm_arch_table_parity.py (76 passed) and test_xpu_spoof_pipeline.py are unchanged before and after.

[danielhanchen]

Validated on staging replications onto main, which run the real Repo tests (CPU) job. Three states isolate the two independent causes:

tree	failed	passed	errors	job
main alone	1	2855	12	failure
main (pre-#7468) + this PR	1	2895	0	failure
current main (#7468 merged) + this PR	0	2896	0	success

The middle row is this change on its own: the 12 errors go to zero, and the single remaining failure was the font-scale test that #7468 has since fixed. With both in place the job is green:

2896 passed, 73 skipped, 23 deselected, 22 warnings, 40 subtests passed

Full batch on the last row: Backend CI, Core, Lint CI and Notebooks CI all success.