Skip to content

Move dependencies from requirements.txt to an optional packaging extra #638

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kkraus14 merged 7 commits into from
May 16, 2025
Merged

Move dependencies from requirements.txt to an optional packaging extra #638

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
23d48a9
Move dependencies from requirements.txt to an optional packaging extra
kkraus14 May 15, 2025
6018239
move cuda_core test requirements to pyproject.toml optional dependenc…
kkraus14 May 15, 2025
c5a4956
Need to expand wildcard to use the bracket syntax
kkraus14 May 15, 2025
4e5ea46
Add cython to test dependencies for now
kkraus14 May 15, 2025
3ad86cb
Merge branch 'dependency_update' of github.com:kkraus14/cuda-python i…
kkraus14 May 15, 2025
eb07724
add missing distutils dependency, used in cython files directives
kkraus14 May 15, 2025
d5e0d31
distutils is part of setuptools
leofang May 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 3 additions & 5 deletions .github/workflows/test-wheel-linux.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -318,14 +318,13 @@ jobs:
pushd "${CUDA_BINDINGS_ARTIFACTS_DIR}"
if [[ "${{ matrix.LOCAL_CTK }}" == 1 ]]; then
ls $CUDA_PATH
pip install *.whl
pip install $(ls *.whl)[test]
else
pip install $(ls *.whl)[all]
pip install $(ls *.whl)[all,test]
fi
popd

pushd ./cuda_bindings
pip install -r requirements.txt
${SANITIZER_CMD} pytest -rxXs -v tests/

# It is a bit convoluted to run the Cython tests against CTK wheels,
Expand Down Expand Up @@ -356,11 +355,10 @@ jobs:
fi
TEST_CUDA_MAJOR="$(cut -d '.' -f 1 <<< ${{ matrix.CUDA_VER }})"
pushd "${CUDA_CORE_ARTIFACTS_DIR}"
pip install $(ls *.whl)["cu${TEST_CUDA_MAJOR}"]
pip install $(ls *.whl)["cu${TEST_CUDA_MAJOR}","test-cu${TEST_CUDA_MAJOR}"]
popd

pushd ./cuda_core
pip install -r "tests/requirements-cu${TEST_CUDA_MAJOR}.txt"
${SANITIZER_CMD} pytest -rxXs -v tests/

# It is a bit convoluted to run the Cython tests against CTK wheels,
Expand Down
8 changes: 3 additions & 5 deletions .github/workflows/test-wheel-windows.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -246,14 +246,13 @@ jobs:
if ('${{ matrix.LOCAL_CTK }}' -eq '1') {
Get-ChildItem $env:CUDA_PATH
echo $PATH
pip install (Get-ChildItem -Filter *.whl).FullName
pip install "$((Get-ChildItem -Filter *.whl).FullName)[test]"
} else {
pip install "$((Get-ChildItem -Filter *.whl).FullName)[all]"
pip install "$((Get-ChildItem -Filter *.whl).FullName)[all,test]"
}
Pop-Location

Push-Location ./cuda_bindings
pip install -r requirements.txt
pytest -rxXs -v tests/
# skip Cython tests for now (NVIDIA/cuda-python#466)
Pop-Location
Expand All @@ -273,11 +272,10 @@ jobs:
}
$TEST_CUDA_MAJOR = '${{ matrix.CUDA_VER }}' -split '\.' | Select-Object -First 1
Push-Location $env:CUDA_CORE_ARTIFACTS_DIR
pip install "$((Get-ChildItem -Filter *.whl).FullName)[cu${TEST_CUDA_MAJOR}]"
pip install "$((Get-ChildItem -Filter *.whl).FullName)[cu${TEST_CUDA_MAJOR},test-cu${TEST_CUDA_MAJOR}]"
Pop-Location

Push-Location ./cuda_core
pip install -r "tests/requirements-cu${TEST_CUDA_MAJOR}.txt"
pytest -rxXs -v tests/
Pop-Location

Expand Down
2 changes: 1 addition & 1 deletion cuda_bindings/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ This subpackage adheres to the developing practices described in the parent meta

## Testing

Latest dependencies can be found in [requirements.txt](https://github.com/NVIDIA/cuda-python/blob/main/cuda_bindings/requirements.txt).
Testing dependencies can be installed using the `[test]` optional dependency identifier. For example, `pip install -v -e .[test]`.

Multiple testing options are available:

Expand Down
11 changes: 10 additions & 1 deletion cuda_bindings/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# SPDX-License-Identifier: LicenseRef-NVIDIA-SOFTWARE-LICENSE

[build-system]
requires = ["setuptools>=77.0.0", "cython", "pyclibrary"]
requires = ["setuptools>=77.0.0", "cython>=3.0", "pyclibrary>=0.1.7"]
build-backend = "setuptools.build_meta"

[project]
Expand Down Expand Up @@ -37,6 +37,15 @@ all = [
"nvidia-nvjitlink-cu12>=12.3",
]

test = [
"cython>=3.0",
"setuptools",
"numpy>=1.21.1",
"pytest>=6.2.4",
"pytest-benchmark>=3.4.1",
"llvmlite"
Copy link
Collaborator

@rwgk rwgk May 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, but just to point out our options:

  • This could be left commented out (status quo), with
# Uncomment to generate MINIMAL_NVVMIR_BITCODE_STATIC for test_nvvm.py (see PR #443).
  • If we choose to make this an unconditional test dependency: I'll work on a follow-on PR to simplify the logic in cuda_bindings/tests/test_nvvm.py. I believe we can then remove the MINIMAL_NVVMIR_BITCODE_STATIC dictionary entirely.

Copy link
Collaborator Author

@kkraus14 kkraus14 May 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is something I wanted to discuss. I could pull it out into a separate optional dependency group of something like test-nvvm or something of the likes. One of my motivations for uncommenting it was to have it start being tested in CI.

Copy link
Collaborator

@rwgk rwgk May 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My vote, based on the quick experiment below (pip install llvmlite takes less than 1 second):

  • Make llvmlite an unconditional test dependency, as you have it right now.
  • I'll do the follow-on PR (quickly, should be easy to squeeze in).
0db6015-lcedt.nvidia.com:~ $ python -m venv junk
0db6015-lcedt.nvidia.com:~ $ . junk/bin/activate
(junk) 0db6015-lcedt.nvidia.com:~ $ pip install --upgrade pip
...
Successfully installed pip-25.1.1
(junk) 0db6015-lcedt.nvidia.com:~ $ time pip install llvmlite
Collecting llvmlite
  Using cached llvmlite-0.44.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.0 kB)
Using cached llvmlite-0.44.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (42.4 MB)
Installing collected packages: llvmlite
Successfully installed llvmlite-0.44.0

real    0m0.867s
user    0m0.662s
sys     0m0.151s

]

[project.urls]
Repository = "https://github.com/NVIDIA/cuda-python"
Documentation = "https://nvidia.github.io/cuda-python/"
Expand Down
11 changes: 0 additions & 11 deletions cuda_bindings/requirements.txt
View file
Open in desktop

This file was deleted.

3 changes: 3 additions & 0 deletions cuda_core/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,9 @@ dependencies = [
[project.optional-dependencies]
cu11 = ["cuda-bindings==11.8.*"]
cu12 = ["cuda-bindings==12.*"]
test = ["cython>=3.0", "setuptools", "pytest>=6.2.4"]
test-cu11 = ["cuda-core[test]", "cupy-cuda11x", "nvidia-cuda-runtime-cu11"] # runtime headers needed by CuPy
test-cu12 = ["cuda-core[test]", "cupy-cuda12x", "nvidia-cuda-runtime-cu12"] # runtime headers needed by CuPy

[project.urls]
homepage = "https://nvidia.github.io/cuda-python/"
Expand Down
4 changes: 0 additions & 4 deletions cuda_core/tests/requirements-cu11.txt
View file
Open in desktop

This file was deleted.

4 changes: 0 additions & 4 deletions cuda_core/tests/requirements-cu12.txt
View file
Open in desktop

This file was deleted.

Loading