Repository navigation
Stage generator: the paper-2611 profile narrows the 2026-11 subset (core lanes, fp32 dense, N=3) - #509
Merged
Conversation
…ore lanes, fp32 dense, N=3 target) The paper's experiments must finish by 2026-11-10, so make_2611_stages.py takes a named profile, `--profile paper-2611` (the default), with the drops listed and proved by `--check` instead of silently missing: - core lanes only (graph, cross-model, documents, dense): no sparse, time series or lifecycle cell, for the comparators and the ArcadeDB chain; `--second-list` needs `--profile full` - Neo4j and Memgraph keep graph and cross-model, but no dense cell - the dense lane is fp32 only for every engine: no `_int8` and no `_pq` arm, ArcadeDB's included (embedded and served fp32 stay) - target N=3 (TARGET_REPS); the passes 4 and 5 stay after pass 3 in #503's order, with the ArcadeDB marker gate unchanged `--check` prints each dropped reason with its lane-cell count and arms, runs profile_problems on both chains whatever the arguments select, and `--check --drafts` lints written scripts for the same. `--profile full` is the whole subset as before. stage_drafts_2611.sha256 is regenerated with `test_stage_generator_2611.py --rewrite`. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Narrows the 2026-11 DBBench subset to what the paper needs (the owner's decisions of 2026-10-25; the paper's experiments must finish by 2026-11-10). The narrowing is a named profile,
make_2611_stages.py --profile paper-2611, which is now the default. Every dropped cell is listed by--checkwith its reason, so nothing is silently missing.What the profile does
--second-listis refused unless--profile fullis given._int8and no_pqarm, comparators' and ArcadeDB's. ArcadeDB keepsarcadedb_dense_embeddedandarcadedb_dense_server. The drop is derived fromrunner.LANES(QUANTIZED_DENSEis the arm-name rule), so a newly registered int8 arm is dropped without anyone typing its name.TARGET_REPS).--repsstill defaults to 5, so the chain keeps Stage generator: exact pass order at N=5 (T1 1-3, T2 1-3, then T1/T2 alternating) and the ArcadeDB go marker #503's order with passes 4 and 5 after pass 3 (they run only if time is left), and the ArcadeDB marker gate is unchanged.--projectnow prices the profile's target repetitions (3) unless--project-reps 5is given.--profile fullis the whole subset as it was before.Dropped cells (lane cells per pass, both chains; each cell counted once, under the first reason)
neo4j_dense,neo4j_dense_int8,memgraph_dense,memgraph_dense_int8arcadedb_dense_embedded_int8,arcadedb_dense_server_int8,qdrant_dense_int8,mongodb_dense_int8,arangodb_dense_int8,sqlite_vec_dense_int8(and the_pqarms whenBENCH_DENSE_PQ=1)Why: the second list is out of scope for the paper. Quantized arms are not like-for-like across engines, and quantization is out of the paper's scope: fp32 for every engine, ArcadeDB's own INT8 and BINARY out by the owner's rule, jvector PQ, BQ and NVQ studied after the paper.
Proof and tests
--checkprints the dropped cells and runsprofile_problemson both chains (comparators and ArcadeDB) whatever the arguments select: no lane of the second list, no quantized dense arm, no Neo4j or Memgraph dense cell, ArcadeDB's dense arms exactly its fp32 pair, no stale drop rule, no arm-name skip that would also drop an arm of another lane. The order proofs (plan_problems, schedule and prefixes) run as before.--check --drafts DIRlints written scripts for the same (lane read back from--lanes, arms fromBACKENDS).stage_drafts_2611.sha256is regenerated withtest_stage_generator_2611.py --rewrite(229 scripts, from 319: the second list and the dropped arms are gone). The hash cases now use the default profile.test_stage_generator_2611.pypin the dropped arms by name, the drafts of both chains, the refusal of--second-list, theBENCH_DENSE_PQ=1case, the profile proof catching a planted dropped arm or lane, the draft lint, the order with passes 4 and 5 after 3, and the projection default. Tests that used--second-listthrough the CLI now pass--profile full.make_2611_stages.py --check,--check --draftson a generated folder (0 problems),version_pin_check.py(0 disagreements), and the benchmarks pytest CI runs (1073 passed, 24 skipped).fairness_check.pyneedsresults/runs.jsonland does not apply to this change.No benchmark ran, no docker, no mini.
🤖 Generated with Claude Code