latent-inspector-py

Patch-token geometry benchmarks for released vision encoders

_{Figure 1. Per-model means with 95% bootstrap confidence intervals for nine patch-token metrics on the 202-image Caltech101 (seed 17) benchmark. Inferential comparisons live in the checked-in bootstrap and sign-flip summaries.}

---

Abstract

latent-inspector-py is the research-grade Python toolkit behind the manuscript Patch-Token Geometry of Four Released Vision Encoders: A Multi-Dataset Artifact Study. It measures patch-token geometry and pairwise alignment of complete released checkpoints under a fixed measurement contract, and ships the code, configs, checked-in benchmark snapshots, LaTeX source, figures, and compiled PDF needed to audit every reported number.

The repository is deliberately narrow in scope. It evaluates released checkpoints under explicit backend and token-alignment contracts; it does not claim that objective family alone causes the observed geometry.

Status. Research artifact, version 0.1.0. The manuscript is a benchmark report, not a peer-reviewed venue paper. All headline numbers are grounded in checked-in artifacts under paper/results/, and pipeline is reproducible with this repository code.

---

Table of Contents

1. Headlines
2. Results at a glance
3. Models
4. Datasets
5. Installation
6. Quickstart
7. Command-line interface
8. Reproducing the paper
9. Project layout
10. Development
11. Citation
12. Acknowledgements

---

Headlines

Scope	Evidence
Models	DINOv2 ViT-L/14, I-JEPA ViT-H/14, V-JEPA 2 ViT-L/16 still-image adapter, EUPE ViT-B/16
Detailed run	202-image Caltech101 (seed 17) with image bootstrap, parent-bucket block bootstrap, paired sign-flip tests, FDR summaries, sample-size sensitivity
Publication snapshot	6 checked-in runs across Caltech101, Imagenette, Oxford-IIIT Pets, DTD
Volume	3,606 selected images · 14,424 per-image model rows · 21,636 pairwise rows
Supported claim	Under recorded runtime backends, EUPE is the compressed / high-coherence extreme and the still-image V-JEPA 2 adapter is the high-LID extreme across the checked-in snapshot
Caveat	Pairwise similarity remains backend-, token-contract-, and dataset-snapshot-specific until null baselines and alternative alignment contracts are reported

---

Results at a glance

Headline extrema across the checked-in artifacts (paper/results/). All numbers are reproduced verbatim from the manuscript abstract; brackets are 95% image-bootstrap confidence intervals.

Quantity	Detailed run (Caltech101, seed 17, n=202)	6-run mean (4 datasets)
EUPE effective rank	39.68 [39.39, 39.96]	40.42
EUPE spatial coherence	0.921 [0.918, 0.923]	0.917
EUPE participation ratio	—	6.11
V-JEPA 2 local intrinsic dim. (LID)	11.63 [11.52, 11.73]	11.81
Largest mean linear CKA pair	DINOv2 ↔ V-JEPA 2	0.448
Smallest mean linear CKA pair	I-JEPA ↔ EUPE	0.121

Pairwise scores are reported under the backend-prefix token protocol and are interpreted as contract diagnostics — not as invariant model-affinity claims.

---

Models

All four backbones are evaluated under identical study bookkeeping, each using its own native preprocessing pipeline. The V-JEPA 2 row uses the still-image ONNX adapter registered in this repository, not a native video-clip evaluation.

Model	Backbone	Training objective	Source	arXiv
DINOv2	ViT-L/14	Image self-distillation	facebook/dinov2-large	2304.07193
I-JEPA	ViT-H/14	Image latent prediction	facebook/ijepa_vith14_1k	2301.08243
V-JEPA 2	ViT-L/16 still-image adapter	Video latent prediction	facebook/vjepa2-vitl-fpc64-256	2506.09985
EUPE	ViT-B/16	Multi-teacher proxy distillation	facebook/EUPE-ViT-B	2603.22387

Inspect the live registry — including ONNX digests, parity tolerances, and validation state:

uv run lipy models --format terminal

---

Datasets

Every dataset id resolves to a checked-in config/publication/ entry pinned by sampling seed and dataset fingerprint.

Dataset	Family	Selection	Citation
Caltech101	object-centric	balanced-parent, seeds 17 / 23 / 31, n=202	Fei-Fei et al., 2004
Imagenette	broad natural image	seed 17, n=1,000	derivative of Deng et al., 2009
Oxford-IIIT Pets	fine-grained object	seed 17, n=1,000	Parkhi et al., 2012
DTD	texture-centric	seed 17, n=1,000	Cimpoi et al., 2014

Datasets must be downloaded under the terms of their respective licenses. config/publication/datasets.yaml records the canonical archive URLs, archive members, and target paths used by the benchmark.

---

Installation

Requires Python 3.11+ and uv.

git clone https://github.com/AbdelStark/latent-inspector-py.git
cd latent-inspector-py
uv sync --python 3.11 --group dev

Verify:

uv run lipy --help
uv run lipy models --format terminal

Optional Rust parity backend. Set LATENT_INSPECTOR_RUNTIME_BACKEND=rust when the sibling Rust reference binary is on PATH to enable the parity-verified path. The Python-native and ONNX paths run with no extra setup.

---

Quickstart

The smoke path generates deterministic demo assets and exercises the packaging, ONNX export, and validation plumbing without requiring any benchmark dataset:

make smoke

Inspect a single image with a single model:

uv run lipy inspect path/to/image.png \
  --model dinov2-vit-l14 \
  --format json

Compare the four released encoders on the same image:

uv run lipy compare path/to/image.png \
  --models dinov2-vit-l14,ijepa-vit-h14,vjepa2-vitl-fpc2-256,eupe-vit-b16 \
  --format json

Run a small, deterministic local study:

make demo-study

---

Command-line interface

Top-level subcommands of lipy (run uv run lipy <command> --help for flags):

Command	Purpose
models	Print the model registry (terminal / JSON)
inspect	Per-image patch-token geometry for one model
compare	Per-image side-by-side geometry across models
benchmark	Single-image timed forward-pass benchmark
embed	Persist embeddings for downstream consumers
profile	Aggregate geometry over a dataset directory
neighbors	Nearest-neighbor search inside an image set
similarity	Cross-model pairwise alignment over a dataset
drift	Checkpoint-to-checkpoint geometry drift
validate	Backend parity & contract validation
export onnx	Export a registered model to ONNX
study run	Run a config-driven study end-to-end
tui	Interactive terminal UI

Every study writes a deterministic bundle: manifest, dataset selection, per-image metrics, pairwise rows, summaries, and plots. Paper claims are grounded in these artifacts, not in notebook state.

---

Reproducing the paper

Each manuscript table and figure is backed by a checked-in artifact. The checked-in publication snapshot can be re-validated without re-running any GPU job.

Rebuild the manuscript PDF (requires Tectonic):

./scripts/build_paper.sh

The compiled PDF is committed under paper/build/paper.pdf so reviewers can read the manuscript without rebuilding.

---

Project layout

latent-inspector-py/
├── src/latent_inspector_py/        # typed Python package
│   ├── analysis/                   # metrics, null baselines
│   ├── backends/                   # ONNX, native, transformers, Rust parity
│   ├── cli/                        # `lipy` Typer app
│   ├── export/                     # ONNX export pipeline
│   ├── models/                     # registry + contracts
│   ├── validation/                 # parity validators
│   └── study.py                    # config-driven study runner
├── config/                         # study configs
│   └── publication/                # paper-grade benchmark index
├── scripts/                        # paper claims, summaries, HF launchers
├── paper/
│   ├── tex/                        # LaTeX source (paperstyle, references.bib)
│   ├── refs/                       # BibTeX archive
│   ├── results/                    # checked-in artifact bundles
│   └── build/paper.pdf             # compiled manuscript
├── docs/                           # benchmark docs and figures
├── tests/                          # pytest suite
└── Makefile                        # local quality + research gates

---

Development

Single-command quality gate (format check, lint, mypy, pytest):

make check

Equivalent explicit commands:

uv lock --check
uv run ruff format --check src tests scripts
uv run ruff check src tests scripts
uv run mypy src
uv run pytest

Full release-readiness gate, including paper-claim validation and a wheel/sdist build:

make release-check

CI runs the same gate on every push and pull request — see .github/workflows/ci.yml. Contribution guidelines, including the project's claim discipline, live in CONTRIBUTING.md.

---

Citation

If this repository or its benchmarks inform your work, please cite both the manuscript and the software artifact:

@misc{bakhta2026patchtoken,
  author       = {Bakhta, Abdelhamid},
  title        = {Patch-Token Geometry of Four Released Vision Encoders:
                  A Multi-Dataset Artifact Study},
  year         = {2026},
  howpublished = {Manuscript and artifacts},
  url          = {https://github.com/AbdelStark/latent-inspector-py},
  note         = {Compiled PDF: \url{https://github.com/AbdelStark/latent-inspector-py/blob/main/paper/build/paper.pdf}}
}

@software{bakhta2026latentinspector,
  author  = {Bakhta, Abdelhamid},
  title   = {latent-inspector-py: Patch-Token Geometry Benchmarks for
             Released Vision Encoders},
  year    = {2026},
  version = {0.1.0},
  url     = {https://github.com/AbdelStark/latent-inspector-py}
}

---

Acknowledgements

This benchmark would not exist without the released checkpoints and methodological work it stands on. We thank the authors of DINOv2 (Oquab et al., 2024), I-JEPA (Assran et al., 2023), V-JEPA 2 (Assran et al., 2025), and EUPE (Zhu et al., 2026), and the authors of the geometric and similarity tooling (RankMe, alignment/uniformity, linear CKA, Procrustes, and the Levina–Bickel LID estimator) that this study combines into a single auditable measurement pipeline. Any errors in interpretation are the responsibility of this repository, not of the upstream work.