fix: Strip trailing separators from --backend-directory path#443
Open
nsiddharth wants to merge 1 commit into
Open
fix: Strip trailing separators from --backend-directory path#443nsiddharth wants to merge 1 commit into
nsiddharth wants to merge 1 commit into
Conversation
A trailing path separator on --backend-directory (e.g. `--backend-directory=/opt/tritonserver/backends/`) produced `/opt/tritonserver/backends//python`, which failed the equality check against the resolved backend location. runtime_modeldir was then set to the backend path instead of "DEFAULT", so every Python model failed to load with `ModuleNotFoundError: No module named 'model'`. Strip trailing separators from the backend directory before the comparison so the default Python backend is detected regardless of a trailing slash. Fixes triton-inference-server/server#6730 Signed-off-by: nsiddharth <n.siddharth@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A trailing path separator on
--backend-directory(e.g.--backend-directory=/opt/tritonserver/backends/) causes every Python-backend model to fail to load with:Why
In
TRITONBACKEND_Initialize(src/python_be.cc), the backend directory is concatenated asdefault_backend_dir_string + os_slash + "python". With a trailing slash this yields/opt/tritonserver/backends//python, which is then compared for equality against the resolved backendlocation(/opt/tritonserver/backends/python). The//vs/mismatch makes the check fail, soruntime_modeldiris set to the backend path instead of""(DEFAULT). The stub is then launched as if a custom Python-based backend runtime lives there and tries to import a module namedmodel, which does not exist.Fix
Strip any trailing path separators from
default_backend_dir_stringbefore the comparison, so the default Python backend is detected regardless of a trailing slash.Verification
Reproduced and verified in a CPU-only
nvcr.io/nvidia/tritonserver:25.02-py3container (arm64), rebuildinglibtriton_python.sofrom this branch and loading a minimal Python model:--backend-directory/opt/tritonserver/backends/(trailing slash)UNAVAILABLE—ModuleNotFoundError: No module named 'model'; stubruntime_modeldir=/opt/tritonserver/backends/pythonREADY;runtime_modeldir=DEFAULT/opt/tritonserver/backends(no slash)READYREADY(no regression)Fixes triton-inference-server/server#6730