Repository navigation
Benchmark examples: replace uv pip in the Docker wrappers and matrix helpers with uv run - #502
Merged
Merged
Conversation
…helpers with uv run The Docker wrappers of examples 07 to 12 run the example with uv run --no-project --with (no venv, no uv pip), through bench_exec.py, which prints the interpreter and every installed package version before it becomes python -u <script>. The Milvus runner image fills its uv cache at build time and runs offline. _matrix_helpers.sh no longer installs the wheel into the host environment; the matrix launcher runs under uv run --with the wheel. Closes #393 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Bandit flags the test's own subprocess call; the argv is fixed (sys.executable plus a repo script), no shell, no untrusted input. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #393.
Follow-up to #381 and PR #392. The benchmark examples no longer use
uv pipor a hand-made venv.07to12, Docker wrappers:python -m venv+ activate +uv pip installis replaced by oneuv run --no-project --python <interpreter> --with <package> ... python bench_exec.py <script> <args>. Every package and every pin is unchanged (ladybug==0.19.0stays). The ArcadeDB wheel is a--withof the same call. The unusedpython3-venvapt package is dropped from the Debian-based branches (the postgres and pgvector containers), and the uv bootstrap there now runs withpython3because there is no venvpythonany more.bindings/python/examples/bench_exec.py: prints one[bench-env] python=...line and one[bench-env] name==versionline per installed package (importlib.metadata, thepip freezeequivalent), thenos.execvspython -u <script>. The measured process is the interpreter that printed the versions, in the same environment, so the log of every run records what it ran with.11and12):uv pip install --systemis replaced by a build-timeuv run --with numpy --with pymilvus --with psutilthat fillsUV_CACHE_DIR=/opt/uv-cache, and the runs useuv run --offlineagainst that cache, so a run cannot resolve other versions than the image holds (as before, the versions are frozen when the image is built). The image tag is bumped topy312-v2so an existingv1image is not reused.examples/scripts/_matrix_helpers.sh: the local wheel is no longer installed into the host environment or the user site.matrix_prepare_local_arcadedb_wheelsetsMATRIX_PYTHON=(uv run --no-project --python <python3> --with <wheel> python3), the sixrun_*_matrix.shscripts start the launcher with it, and the helper logs the launcher's python,arcadedb-embeddedandjpype1versions.uv run --withlayers over the active virtual environment (checked), so the launcher still sees the dependencies of that environment, such aslxml.benchmarks/experiments/CAMPAIGN.md: theuv pip install --python .venv <wheel> jpype1==1.7.1note now saysuv add jpype1==1.7.1at the repo root (not run here; it records the pin inpyproject.tomlanduv.lock). Docs: the sixdocs/examples/07to12pages andci-setup.md.Same versions before and after, in
python:3.12-slimwithlxml duckdb arcadedb-embedded==26.10.2(both commands built from the realuv_bootstrap_commandsanduv_run_command): the old flow (python -m venv,uv pip install,python bench_exec.py) and the new flow print identical[bench-env]lines, which is the same CPython 3.12.15, lxml 6.1.3, duckdb 2.0.0, arcadedb-embedded 26.10.2, jpype1 1.7.1 (and the samepackagingandpip), and the probe script ran in both. Also run: the Debianbookworm-slim+ aptpython3path (the postgres and pgvector pattern, nopython3-venv), whereuv run --python python3 --with lxml --with psycopg[binary]printed python 3.11.2, lxml 6.1.3, psycopg 3.3.6; and the Milvus pattern (cache filled by oneuv runas root, thenuv run --offlineas uid 4242 from that cache) printed numpy 2.5.4, pymilvus 3.0.2, psutil 7.2.2. All containers were pinned to cores 0 to 3 with a 2 GB limit.No speed claim: this changes how the environment is built, not what the measured process executes, and no timed run was made. The interpreter start of the measured process is now
uv runthenexecvinstead of the venvpython, outside the script's own timers. The issue asked for a shorttools/ab.py --noiseround per changed example if a measured phase includes interpreter start; that is left to the benchmark queue (none of the examples' timers include it), so I did not run it.Checks run: pre-commit (black, isort, shfmt and the rest) on every changed file, again after the formatter's own edits;
bash -non_matrix_helpers.shand the sixrun_*_matrix.sh; shellcheck (same 9 findings with and without the change, none new);py_compileof07to12; pytest on the newtest_bench_exec.py(the environment print, the exec into the script, and a guard againstuv pipand venvs in the six wrappers) and the existingtest_example11_degree_matching.py(5 passed, run against the published 26.10.2 wheel). Not run: the full wrappers against a real dataset, the Milvus runner image build itself, and_matrix_helpers.shagainst a freshly built wheel (needs the Linux wheel build); CI covers the examples.Remaining
uv pipoutside this scope, not touched:e2e-python/README.md(upstream's suite),docs/superpowers/plans/2026-07-08-bolt-ci-nightly-matrix-4891.mdanddocs/superpowers/specs/2026-07-08-bolt-ci-nightly-matrix-4891-design.md(historical plans). Theci-setup.mdsentence that says there is nouv pipin the workflows is a statement, not a command.🤖 Generated with Claude Code