You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(python): follow sys.modules shims to the module they install - #1133
No linked issue. This came up indexing a real backend that keeps ~70 flattened-import compatibility shims at its root. The behaviour and the fix are described in full below.
What & why
Some Python codebases keep old flat import paths working with a shim module that swaps itself out of sys.modules:
After import billing, the caller holds app/services/billing.py. The parser stopped at the shim file:
Call site
Before
After
import billing (IMPORTS_FROM)
billing.py
app/services/billing.py
billing.charge(1)
bare charge, unresolved (charge is not top-level in the shim)
app/services/billing.py::charge
billing.run_model(1) (re-exported by the real module)
bare, unresolved
app/services/engine.py::run_model
from billing import charge
billing.py::charge (no such node)
app/services/billing.py::charge
from billing import charge as pay; pay()
billing.py::pay
app/services/billing.py::charge
TESTED_BY from a test calling billing.charge
bare source charge
app/services/billing.py::charge
billing.refund(1) (not defined anywhere)
bare
bare (unchanged, as it should be)
On the codebase that prompted this, a module with 39 importers showed 1 (the shim), and a function with 59 callers showed 3.
Changes
parser.py: follow module-alias shims during resolution._python_module_alias_target() maps a resolved Python module file to the module it installs as sys.modules[__name__]. It is applied wherever Python imports become files: _resolve_module_to_file, _python_import_origin (so star exports and re-export origins see through shims too), and _python_submodule_file. Detection uses ast and accepts only the literal, module-level form, importlib.import_module("dotted.name") or a bare import_module(...). A computed or relative name, a swap inside a block, or a target outside the repository leaves the file as the module. Chains of shims are followed up to 5 hops, with a cycle guard. Targets excluded through exclude_files (used by forget) are not followed, so forget gives the same result as a build without those files. Results are cached per parser instance, and exclude_files clears the cache.
parser.py: Python as aliases keep the original name.from m import a as b; b() was qualified as m.py::b. The existing JS original-name key now records Python aliases too, so _resolve_call_target asks for a. The prefixed key starts with _, so star-export discovery already filters it out. This is a general fix, not specific to shims.
parser.py and graph.py: module receivers follow re-exports.pkg.helper() where pkg only re-exports helper now stamps receiver_module with the defining file (via the existing _python_reexport_origin). The bare-endpoint resolver accepts a candidate in the receiver's own file as evidence, because the call-site file does not import the defining file directly. Candidates are still narrowed to top-level names in exactly that file first, so this does not loosen the receiver rule.
There is no schema change and no new post-processing pass: the graph resolver needs no knowledge of shims.
Known limitation
Importers now depend on the real module, so update does not re-parse them when only the shim file changes. That is the same dependency rule as every other import. Re-pointing a shim to a different module needs a build. This is noted in the CHANGELOG entry.
How it was tested
New module tests/test_python_module_alias_shims.py (22 tests). It covers:
a simple shim;
a shim of a shim;
a re-export through a module receiver, with and without a shim;
from shim import x and from shim import x as y, plus a plain as alias;
TESTED_BY;
a missing name that stays bare;
four swaps that must not be followed (computed, relative, conditional, outside the repository);
a shim cycle;
review follow-ups: several swaps (the last one wins), a literal swap overwritten by one that can't be followed, from shim import * direct and recursive, a renamed re-export next to an unrelated function of the same name, and a later unaliased import that rebinds an alias;
build/update parity: a post-processing pass run twice is a no-op, and after a new function is added to the real module, incremental_update resolves an untouched caller's shim.new() and matches a fresh full_build.
Ten of the 16 fail on staging without the change. The six that pass there are the guard cases.
uv sync --extra dev
uv run pytest tests/ --tb=short -q
# Run with five modules split out (see note below):# 4802 passed, 880 skipped, 3 xfailed, 2 xpassed# Each of those five on its own: watch_refresh_without_index 5, watch_robustness 61,# postprocessing 20, embedding_refresh_skip_notice 30, incremental 139: all passed
uv run ruff check code_review_graph/
# All checks passed!
uv run mypy code_review_graph/ --ignore-missing-imports --no-strict-optional
# Success: no issues found in 77 source files
bandit -r code_review_graph/ -c pyproject.toml # via uvx --from 'bandit[toml]'# No issues identified.
Note on the full run. Locally, uv run pytest tests/ sometimes stops partway with a KeyboardInterrupt raised from unittest/mock.py. A test in one of test_watch_refresh_without_index.py, test_watch_robustness.py, test_postprocessing.py, test_embedding_refresh_skip_notice.py or test_incremental.py lets a mocked KeyboardInterrupt escape. Running those five modules together reproduced it 3 times in 4 on staging without this change, so it predates this PR. Each module passes on its own, and the rest of the suite passes with them excluded.
tests/test_python_star_imports.py: the two read-count tests now expect the helper module to be read twice per process instead of once. One read checks whether it is a shim and one computes its star exports. Both are cached process-wide, so concurrent parsers still read it only twice in total.
Checklist
Tests added for new functionality
All tests pass: uv run pytest tests/ --tb=short -q
Linting passes: uv run ruff check code_review_graph/
Type checking passes: uv run mypy code_review_graph/ --ignore-missing-imports --no-strict-optional
Lines are at most 100 characters
Docs updated where behavior changed (README, docs/, docstrings): CHANGELOG Unreleased (Added and Fixed) and docstrings
A compatibility shim that does
`sys.modules[__name__] = importlib.import_module("pkg.real")` hands its
importers `pkg/real.py`, but the parser stopped at the shim: the import
edge named the shim, `shim.func()` and the TESTED_BY edge from a test
calling it stayed bare (`func` is not top-level in the shim), and
`from shim import func` qualified as a node that does not exist.
Python module resolution now maps a shim to its target, through chains
of shims, so the import edge, module-receiver evidence, from-imports and
re-export origins all see the real module. Only the literal,
module-level form is followed; detection is cached process-wide by
mtime and size like the star-export cache.
Two general fixes the shim cases depend on:
- `from m import a as b; b()` resolves to `m.py::a`, not `m.py::b`.
- `pkg.name()` where `pkg` only re-exports `name` stamps the defining
file as `receiver_module`, and the bare-endpoint resolver accepts the
receiver's own file as evidence.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Unresolved wildcard imports and incorrect targets from rebinding, renamed re-exports, and multiple module replacements can produce inaccurate graph edges.
This PR improves Python dependency tracking by following sys.modules compatibility shims to their installed modules, so imports and calls can resolve to actual definitions.
Changes:
Adds bounded, cached shim resolution and preserves original names for Python import aliases.
Follows module-receiver re-exports and accepts their defining files during graph resolution.
Adds regression coverage and documents the rebuild requirement after retargeting a shim.
File
Description
tests/test_python_star_imports.py
Updates read-count expectations for cached shim detection.
tests/test_python_module_alias_shims.py
Adds shim, alias, receiver, guard, and build/update regression tests.
code_review_graph/parser.py
Adds shim detection and updates import and receiver resolution.
code_review_graph/graph.py
Accepts receiver-module evidence when resolving bare endpoints.
CHANGELOG.md
Documents resolution improvements and incremental-update limitations.
- `from shim import *` enumerates the real module: star-import
expansion now resolves through `_python_import_origin`, which follows
shims, so direct and recursive wildcard imports reach the definition.
- Shim detection takes the last write to `sys.modules[__name__]`, as
importers do, and declines when that write is not a literal swap or
when any write sits inside a block.
- A renamed re-export (`from .impl import charge as pay`) keeps `pay` on
the re-exporting module instead of sending `pkg.pay()` to an unrelated
`impl.pay`.
- A later unaliased `from b import bar` drops the original-name entry an
earlier `from a import foo as bar` left behind.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Shim detection can mistake a locally defined callable for importlib’s function and redirect graph edges to an unrelated module.
No open issues 🎉
1 previously missed issue
In code that hasn't changed since last review
Reject shadowed import_module callees in module swap detection
code_review_graph/parser.py:15157
The bare-call branch accepts any function named import_module, even a local function rather than the importlib function. For example, def import_module(name): return sys.modules[__name__] followed by sys.modules[__name__] = import_module("target") leaves the original module installed at runtime. This detector instead follows target when that file exists in the repository, attributing import and call edges to an unrelated module. Verify the callee's import binding and reject shadowed or rebound callees before following a swap; apply the same check to the importlib.import_module form. Add a guard regression for a locally defined wrapper while preserving support for from importlib import import_module.
…dule
Shim detection accepted any callable spelled `import_module` or
`importlib.import_module`, so a module defining its own
`def import_module(...)`, or rebinding `importlib`, was read as a shim
and its importers redirected to an unrelated module. The callee name
must now be bound only by an import from importlib (`import importlib`,
`import importlib.util`, or `from importlib import import_module`).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Require an unshadowed sys import before redirecting edges
code_review_graph/parser.py:15169
The detector validates importlib but never checks what sys refers to. With import fake_sys as sys, where fake_sys.modules is an ordinary dictionary, the assignment does not replace the importing module, yet the detector redirects its import and call edges to the named target. Require an unshadowed sys import before accepting the swap, and add a regression for a shadowed or rebound sys.
Resolve shim submodule dependencies to the installed package
code_review_graph/parser.py:15804
Submodule dependencies still use the shim's original name when the shim installs a package. For example, if shim.py installs pkg, _python_package_dir now finds pkg, but _python_from_import_targets emits shim.child for from shim import child. Resolution then searches for shim/child.py, not the existing pkg/child.py, and leaves that dependency unresolved. Importer queries and incremental dependency tracking therefore miss the child file. Resolve these submodule targets relative to the installed package and add a shim-to-package regression that checks the IMPORTS_FROM edge.
Use POSIX paths for graph identity comparisons
tests/test_python_module_alias_shims.py:102
Graph paths use forward slashes on every platform (normalize_file_path in parser.py:911–925), but str(file_path.resolve()) uses backslashes on Windows. This filter therefore returns no edges, so the new positive regression tests fail even when shim resolution is correct. Use .resolve().as_posix() for graph identities throughout this test module, including _paths() and the expected qualified names in individual tests.
- Read a module for shim detection only when its resolved path is inside
the repository, so a symlink to an outside file is never opened.
- Require `sys` to be bound only by `import sys`; `import x as sys` or a
rebinding makes `sys.modules[__name__] = ...` replace nothing.
- `from shim import child`, where the shim installs a package, depends
on that package's `child` file: the target is the submodule's file
when the dotted name would point under the shim, and Python module
resolution accepts an absolute in-repository path.
- Tests compare graph identities as POSIX paths, so they hold on
Windows.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Human assessment is needed because re-export resolution can bypass repository boundaries and read external files through symlinks, alongside alias-resolution and rebinding concerns.
Legend icons were stretched to 104px by the page-wide svg rule, so the legend
filled the screen and a sixth node kind pushed it under the filter panel. This
is older than the train, and the icons are now 16px. Two tests also depended on
their platform: Python 3.10 accepts the deep expression (#1133), and Windows
paths have backslashes (#1108).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Linked issue
No linked issue. This came up indexing a real backend that keeps ~70 flattened-import compatibility shims at its root. The behaviour and the fix are described in full below.
What & why
Some Python codebases keep old flat import paths working with a shim module that swaps itself out of
sys.modules:After
import billing, the caller holdsapp/services/billing.py. The parser stopped at the shim file:import billing(IMPORTS_FROM)billing.pyapp/services/billing.pybilling.charge(1)charge, unresolved (chargeis not top-level in the shim)app/services/billing.py::chargebilling.run_model(1)(re-exported by the real module)app/services/engine.py::run_modelfrom billing import chargebilling.py::charge(no such node)app/services/billing.py::chargefrom billing import charge as pay; pay()billing.py::payapp/services/billing.py::chargebilling.chargechargeapp/services/billing.py::chargebilling.refund(1)(not defined anywhere)On the codebase that prompted this, a module with 39 importers showed 1 (the shim), and a function with 59 callers showed 3.
Changes
parser.py: follow module-alias shims during resolution._python_module_alias_target()maps a resolved Python module file to the module it installs assys.modules[__name__]. It is applied wherever Python imports become files:_resolve_module_to_file,_python_import_origin(so star exports and re-export origins see through shims too), and_python_submodule_file. Detection usesastand accepts only the literal, module-level form,importlib.import_module("dotted.name")or a bareimport_module(...). A computed or relative name, a swap inside a block, or a target outside the repository leaves the file as the module. Chains of shims are followed up to 5 hops, with a cycle guard. Targets excluded throughexclude_files(used byforget) are not followed, soforgetgives the same result as a build without those files. Results are cached per parser instance, andexclude_filesclears the cache.parser.py: Pythonasaliases keep the original name.from m import a as b; b()was qualified asm.py::b. The existing JS original-name key now records Python aliases too, so_resolve_call_targetasks fora. The prefixed key starts with_, so star-export discovery already filters it out. This is a general fix, not specific to shims.parser.pyandgraph.py: module receivers follow re-exports.pkg.helper()wherepkgonly re-exportshelpernow stampsreceiver_modulewith the defining file (via the existing_python_reexport_origin). The bare-endpoint resolver accepts a candidate in the receiver's own file as evidence, because the call-site file does not import the defining file directly. Candidates are still narrowed to top-level names in exactly that file first, so this does not loosen the receiver rule.There is no schema change and no new post-processing pass: the graph resolver needs no knowledge of shims.
Known limitation
Importers now depend on the real module, so
updatedoes not re-parse them when only the shim file changes. That is the same dependency rule as every other import. Re-pointing a shim to a different module needs abuild. This is noted in the CHANGELOG entry.How it was tested
New module
tests/test_python_module_alias_shims.py(22 tests). It covers:from shim import xandfrom shim import x as y, plus a plainasalias;from shim import *direct and recursive, a renamed re-export next to an unrelated function of the same name, and a later unaliased import that rebinds an alias;incremental_updateresolves an untouched caller'sshim.new()and matches a freshfull_build.Ten of the 16 fail on
stagingwithout the change. The six that pass there are the guard cases.Note on the full run. Locally,
uv run pytest tests/sometimes stops partway with aKeyboardInterruptraised fromunittest/mock.py. A test in one oftest_watch_refresh_without_index.py,test_watch_robustness.py,test_postprocessing.py,test_embedding_refresh_skip_notice.pyortest_incremental.pylets a mockedKeyboardInterruptescape. Running those five modules together reproduced it 3 times in 4 onstagingwithout this change, so it predates this PR. Each module passes on its own, and the rest of the suite passes with them excluded.tests/test_python_star_imports.py: the two read-count tests now expect the helper module to be read twice per process instead of once. One read checks whether it is a shim and one computes its star exports. Both are cached process-wide, so concurrent parsers still read it only twice in total.Checklist
uv run pytest tests/ --tb=short -quv run ruff check code_review_graph/uv run mypy code_review_graph/ --ignore-missing-imports --no-strict-optionaldocs/, docstrings): CHANGELOGUnreleased(Added and Fixed) and docstrings🤖 Generated with Claude Code