.venv/bin/python -m pytest -q -p no:cacheprovider tests # everything
.venv/bin/python -m pytest -q -p no:cacheprovider tests -n 8 # parallel (xdist)
.venv/bin/python -m pytest -q -p no:cacheprovider tests/views # one area
.venv/bin/python -m pytest -q -p no:cacheprovider tests -k lmdb # one backendThe full suite takes about 3 minutes serial and under a minute with -n 8. Every test runs on the project venv; the only configuration is the slow marker in pyproject.toml, excluded by default.
Three files drive the storages and views with random operations against a reference model, and a fourth fires interrupts into writes; all four run in two tiers:
| file | drives |
|---|---|
conformance/test_s8_model.py |
a Hypothesis state machine over one storage: transactions begun on the main thread, a loop thread or a worker thread, batches, snapshots, drop, close and reopen, bulk rows across scan pages, big values, odd and oversized keys, against a dict model with a pending overlay |
conformance/test_s1_randomized.py |
threads, coroutines and spawned processes under random load: bank sum, insert race on a scanned range, atomic multi-key visibility, with the default and a 20ms busy timeout, and with SIGKILL of writer processes on sqlite and lmdb |
views/test_views_model.py |
random ops on five views (dict, log indexed, fractional indexed, list, set) against python models, every view and every raw storage row checked after each op |
_backends/storages/test_interrupt_fuzz.py |
a signal handler raising at random into sqlite and lmdb writes, each run its own process: nothing reported as landed is lost, nothing lands twice; the long runs add a side writer and a peer process |
fastruns in the default suite: a fixed seed, a few short examples, about 5 seconds per backend across the model files, plus the short interrupt runs. The same seed draws the same operations every run.slowis excluded by default throughaddopts = -m 'not slow'inpyproject.tomland runs with-m slow: about 1 to 2 minutes per backend. Any explicit-mon the command line replaces that default, so-m "not functional"includes the slow tier; addand not slowwhen that matters.CONFORMANCE_MODEL_EXAMPLESandCONFORMANCE_MODEL_SEEDsize and seed the two state machines,CONFORMANCE_MODEL_SECONDSscales the concurrent runs (30 is the default).
.venv/bin/python -m pytest -q -p no:cacheprovider -m slow tests # the slow tier
CONFORMANCE_MODEL_EXAMPLES=500 .venv/bin/python -m pytest -q -p no:cacheprovider -m slow tests/views/test_views_model.py -k lmdbA failure prints the shrunk sequence of steps that reproduces it. Each run also prints its op counts per rule (-s), so a starved rule is visible. Fast tier, per backend: storage model about 800 ops with put 200, delete 70, commit 70, snapshot reads 120, close and reopen under 15; views model about 400 ops spread over 45 rules.
| directory | what it covers | backends |
|---|---|---|
conformance/ |
the kv contract as executable cases, one run per backend. Transactions, snapshots, errors, process model, notifications, cost. See its README | sqlite, lmdb, rocksdb, mem; notify on the in-memory and redis transports |
_backends/storages/ |
per backend behaviour below the contract: locks, threads, fork, close races, map growth, secondaries. _write_slot_cases.py is the shared writer slot suite each backend runs |
one file per backend |
_backends/publishers/, _backends/observers/, _backends/notify/ |
publisher queue, retry and drops; observer readiness, reconnect, backlog; the redis routing function | in-memory and redis |
_backends/test_notify_*.py |
notify end to end: gap after subscribe, backpressure, reconnect | in-memory and redis |
views/ |
every view against storage: behaviour on every backend, op counts on the seeking backends (sqlite, lmdb, rocksdb). test_dict_views_hardening.py pins the contract rules for dict views: EMPTY, existence, kind change, cost |
all four |
functional/ |
container and view behaviour through the public API, watches on subtrees and absent containers | mem |
unit/ |
pure units: key and path navigation, container types and markers, view registry and capabilities, fractional index keys | none |
tkv/ |
the tkv protocols: codecs, filters, transport, compliance suites a backend can import | compliance on sqlite, lmdb, rocksdb, mem; the rest none |
support/ |
shared helpers: counting.py (op counter for cost tests, open_backend), valkey.py (a private valkey for redis tests), dict_view.py, set_members.py |
- Every test that touches a file backend uses
tmp_path. Nothing is left on disk. - Redis tests use
TEST_REDIS_URLwhen set, else start a private valkey undertmp_pathand skip if none can be started. - Cost tests count storage ops, not time. They assert that an op costs the same at 10 and at 1000 siblings, or that
lenreads children only. Seesupport/counting.py. - Concurrency tests force the race (a pause between read and write, a barrier before the write) and retry only the two retryable error classes. A stalled thread fails the case; the conformance suite joins such threads after the storage closed.
- Timing budgets are generous (100ms to 300ms) and hold under 3x load. A failure on time is a real stall, not noise.
- Known contract failures are strict xfails in
conformance/known_failures.py. An unexpected pass fails the run, so a fix deletes the row in the same commit. - Notify tests wait on
publisher.flush()andobserver.flush()or an event, not on sleeps, except for a few negative checks inconformance/test_s5_notify.pythat wait a quiet period to assert nothing arrived. - Multi-process cases spawn workers (
spawnstart method) and skip on single-process backends (rocksdb, mem).
Add a spec to conformance/backends.py (factory, process model, writer slot flag) and a line to support/counting.open_backend. Run tests/conformance and tests/views. Known gaps go into known_failures.py with an owner, as strict xfails.