Part of ModePot. Project website Created by Tugrul Guner
A custom embedded database preview for Python.
RefPot is an experimental alpha: a native C++ redo-and-checkpoint engine with Python CRUD and explicit transactions. Install the latest published version with `python -m pip install refpot`. Package and platform qualification remain limited.
Quick start · Why RefPot · Package contract · Documentation · Deep reference · Examples · Agent skill · Community · Contributing
Important
Experimental alpha. Package and platform qualification remain limited; no general performance claim or production durability guarantee is made.
RefPot is an early custom embedded database engine: a native C++ redo-and-checkpoint core exposed through a small Python Database / Row API. The package preview already supports fixed-schema CRUD and explicit transactions; it is not merely a research design. SQL and ORM are future design direction, not shipped interfaces.
Requires Python 3.11–3.14. Install the experimental alpha from PyPI:
python -m pip install refpotSave this complete program as quickstart.py and run it with python quickstart.py:
from pathlib import Path
from tempfile import TemporaryDirectory
from refpot import Database, Row
with TemporaryDirectory(prefix="refpot-quickstart-") as directory:
path = Path(directory) / "example.rp" # Parent directory must already exist.
with Database(path) as db:
db.insert(1, 42, "answer")
db.update(1, value=43)
assert db.get(1) == Row(1, 43, "answer")
with db.transaction():
db.insert(2, 99)
try:
with db.transaction():
db.update(1, value=100)
raise RuntimeError("roll back this transaction")
except RuntimeError:
pass
assert db.get(1) == Row(1, 43, "answer")
db.delete(2)
db.checkpoint()
with Database(path) as db:
assert db.get(1) == Row(1, 43, "answer")
print(db.get(1))python quickstart.pyThe complete quick start demonstrates CRUD, transaction commit and rollback, checkpointing, and persistence after close/reopen. No output is prescribed; run it to see the actual Row representation. The same program is available as examples/quickstart.py.
The database starts empty. Keys and integer values are signed 64-bit integers; text is UTF-8 limited to 31 encoded bytes. The database path names a file, not a directory, and its parent directory must already exist. One process exclusively owns each database; only its creating thread may use the handle.
The native engine uses versioned snapshots and a WAL. Snapshot and WAL recovery are each bounded to 64 MiB. Incomplete unacknowledged tails are discarded; complete transactions with both copies corrupt are rejected; uncertain writes poison the writer. These implementation details are not broad recovery qualification or a physical power-loss guarantee. Concurrent readers, multiwriter use, Windows, existing-format compatibility, SQL and ORM are unsupported. See the full package contract and package recommendation.
Open the full-size editable SVG · Detailed package contract
Transactions stage per-row changes without full-map copies. Commit preparation serializes the final change set before WAL I/O; only after both checksummed redo copies are synchronized does nonallocating publication update committed memory. Recovery validates/prepares a frame before applying row-local changes. These mechanisms preserve the API and storage framing, and do not by themselves establish package performance or broader qualification.
Task-oriented guides are available for CRUD, transactions and rollback, persistence and checkpointing, errors and ownership, plus the runnable CRUD example, transaction example, and complete quick-start program.
The package contract documents the exact Database and Row API, validation, errors, ownership, capacity and unsupported behavior. Hosted guides: API reference · package reference.
The current package architecture records implementation behavior, not an independently qualified guarantee. Recovery internals explain file ownership, snapshot/WAL behavior and current limits. The separate planned execution diagram remains explicitly planned, not shipped.
See project status, design direction, roadmap, contributing, and release policy.
The standalone CRUD example exercises insert, update, delete, context-managed close, and reopen. The transaction example demonstrates read-your-writes, commit, rollback, checkpoint, and reopen. Run them after installation:
python examples/crud.py
python examples/transactions.pyThe full task-first docs live at RefPot documentation with package quick start, package reference, status and next steps, and design direction.
Detailed findings, raw evidence and reproduction · Hosted historical benchmark report · Full-size chart · Next package recommendation
The historical findings are separate from package performance. The 3.04–5.95× result applies only to the historical update harness; its reopen result remains 16/16 losses. Package results are reported separately below.
The preserved planned architecture visual shows a future SQL/ORM direction, not this preview. Neither SQL nor ORM is implemented.
- Engine first. Establish correctness and performance before building the ORM.
- One semantic boundary. Direct operations must not bypass validation or constraints.
- Explicit guarantees. Internal plan selection must not silently weaken durability, isolation, transaction boundaries, or result semantics.
- Bounded maintenance. Include checkpointing, reclamation, and recovery debt in design and measurement, rather than hiding them outside favorable timings.
- Simple public usage. Keep storage layout, encoding, and routing choices private unless a user genuinely needs an operational control.
Language, storage format, index structure, isolation model, and SQL compatibility scope remain decisions to earn through coherent experiments. A native-language rewrite alone is not a differentiator. No SQLite SQL, file-format, or C API compatibility is promised.
The actual-package campaign uses the public Python API and prepared SQLite over six workload/size cells. See the canonical measured results and reproduction guide. Provenance identifies locally built wheels carrying 0.1.1 metadata; these are neither published PyPI 0.1.1 wheels nor a 0.1.2 package artifact. The overall 2× SQLite target remains unmet: three of six measured cells are below 2× (both singleton/batch writes at 1,000 rows and singleton updates at 10,000 rows). Point reads exceed 2× in both sizes, and 10,000-row batch updates exceed 2×; these wins do not satisfy the complete target.
Join the ModePot Discord for design discussions, implementation questions, early ideas, and database use cases across the family. Use GitHub Issues for scoped proposals and reproducible findings. The experimental alpha is not yet broadly qualified.
See CONTRIBUTING.md for setup and contribution workflow. Repository-local coding-agent guidance is in AGENTS.md, with the installable RefPot database skill for consumer-facing API boundaries. The first useful work is helping define a small, testable engine contract and qualification matrix. Correctness cases, crash and I/O-failure scenarios, reproducible benchmark methods, and low-ceremony usage designs are more useful now than a broad speculative API.
Discuss substantial architecture or public-contract changes in GitHub Issues before implementing them. Do not introduce a SQLite wrapper as the engine, publish invented benchmark output, or present research prototypes as production capabilities.
The logo source is SVG; the README uses its PNG rendering.
RefPot is distributed under the MIT License.
See CHANGELOG.md for changes and release guidance for release policy.