Changelog
mempill follows semantic versioning. The first published release is 0.2.0. The latest
published release on crates.io and
PyPI is 0.3.0. 0.4.0 below is unreleased — it exists
on the main branch only. See Installation for install
instructions and how to use the unreleased API today.
0.4.0 — Unreleased
Section titled “0.4.0 — Unreleased”536 Rust (+3 ignored) + 88 Postgres-gated + ~165 Python + 19 MCP tests · 0 warnings
Breaking change — per-agent SQLite entry points
Section titled “Breaking change — per-agent SQLite entry points”mempill-sqlite’s public constructors are now per-agent, replacing the prior path-based
API. The raw path-based connection::open is no longer public.
| Before (removed) | After (0.4.0) |
|---|---|
open_default(path) (Rust, mempill-sqlite) |
open_default_for_agent(base_dir, agent_id) |
open_with_oracle(path, oracle) (Rust, mempill-sqlite) |
open_with_oracle_for_agent(base_dir, agent_id, oracle) |
mempill.open(path) (Python) |
mempill.open_for_agent(base_dir, agent_id) |
mempill.open_oracle(path, oracle) (Python) |
mempill.open_oracle_for_agent(base_dir, agent_id, oracle) |
MEMPILL_DB_PATH (mempill-mcp env var, full file path) |
MEMPILL_DB_DIR (base directory) |
The database file is now always derived as base_dir/agent_{agent_id}.db, which makes it
structurally impossible for two different agent_id values to collide on the same file.
agent_id is validated against [A-Za-z0-9_-]; an invalid agent_id raises
SqliteStoreError::InvalidAgentId (Rust) / StorageError (Python) before any file I/O.
open_default_in_memory() / open_with_oracle_in_memory() / mempill.open_in_memory() /
mempill.open_oracle_in_memory() are unaffected — in-memory engines have no file to
collide on.
PostgreSQL entry points are unaffected. open_postgres / open_postgres_with_oracle
keep their existing signatures — PostgreSQL was already agent-scoped via advisory locking,
not file naming.
Migration for existing pre-0.4.0 databases: a pre-0.4.0 shared-file database is not
auto-migrated. Point MEMPILL_DB_DIR / base_dir at a fresh directory, or manually copy
the existing database file to base_dir/agent_{agent_id}.db before first use with the new
API.
See Quickstart (Rust), Quickstart (Python), and MCP Integration for the updated examples.
Configurable PostgreSQL connection pool
Section titled “Configurable PostgreSQL connection pool”- New
PoolConfigstruct (max_size: u32,connection_timeout: Duration) and an additivePostgresPersistenceStore::with_pool_config(conn_str, pool_config)constructor. PostgresPersistenceStore::newis unchanged and delegates towith_pool_configwithPoolConfig::default()— existing callers see no behavior change. Defaults match the prior hardcoded values:max_size = 20,connection_timeout = 5s.max_size == 0is rejected withPostgresStoreError::Configbefore any network or database connection is attempted.
See PostgreSQL Backend for usage.
Ledger-scope correctness fix
Section titled “Ledger-scope correctness fix”- Write and audit paths (
ingest_claim,reconcile,submit_adjudication,sweep_adjudications) now use an uncapped, claim-scopedload_ledger_for_claimslookup instead of a capped, agent-wideload_ledger(agent_id, None, 10_000)call. - What this fixes: on an agent with more than 10,000 ledger rows, disposition-changing ledger entries beyond the cap window were previously invisible to fold/state-guard logic, which could produce an incorrect belief. Correctness of these paths is now independent of agent ledger history size.
As-of / bi-temporal correctness benchmark
Section titled “As-of / bi-temporal correctness benchmark”-
New reproducible, assertion-based benchmark exercising both the valid-time and transaction-time axes independently, including the 3-way succession chain and the honest Contested case (a genuine conflict that is not silently resolved).
-
Single reproduction command, no external data or environment-specific setup required:
Terminal window cargo run --release --example asof_correctness_benchmark -p mempill -
This is a correctness benchmark only — every scenario is a hard assertion against a documented expected outcome. No timing, latency, or throughput numbers are captured or published.
-
Full results: As-Of / Bi-Temporal Correctness Benchmark.
0.3.0 — published
Section titled “0.3.0 — published”Published on crates.io and PyPI — the latest published release.
Valid-time as-of query (valid_at)
Section titled “Valid-time as-of query (valid_at)”valid_at: Option<DateTime<Utc>>added toQueryMemoryRequest— a point-in-time valid-time filter independent ofas_of_tx_time. Enables queries such as “who was CEO on 2021-06-01?” wherevalid_atselects by real-world validity window andas_of_tx_timeselects by when the engine recorded the claim. The two axes are now fully independent.- Exposed in the Rust core API, the Python wheel (
query_memorydict), and the MCPquery_memorytool (valid_at?parameter).
Date granularity (per-endpoint, display-only)
Section titled “Date granularity (per-endpoint, display-only)”mempill now records the precision of every valid-time boundary independently, and renders it back honestly — no fabricated precision is ever inserted.
What shipped:
-
DateGranularityenum with four levels:Year,Month,Day,Instant. Added additively toValidTimeas two independent nullable fields:start_granularityandend_granularity. The start and end boundaries of a valid-time interval can carry different precision. -
Honest display. A render helper (
format_valid_time_endpoint) formats each boundary according to its recorded granularity:Granularity Display format Example YearYYYY"2020"MonthYYYY-MM"2020-03"DayYYYY-MM-DD"2020-03-15"InstantYYYY-MM-DD(day precision)"2024-05-15"None(legacy)YYYY-MM-DD(fallback)"2020-03-01"A
Month-granularity boundary is never displayed as"2020-03-01"— doing so would imply day precision the caller never supplied. -
Pre-rendered display strings. Every belief slot exposes
valid_from_displayandvalid_until_display, plus the rawstart_granularity/end_granularitytags inside itsvalid_timeobject. Python and MCP surface these directly in everyquery_memoryresponse, so callers do not need to implement display logic.
Ingest contract — infer vs. explicit:
-
Ergonomic Rust
remember()/RememberOptions— infers granularity automatically from the supplied date string via theparse_valid_time_dateparser:"2020"→Year,"2020-03"→Month,"2020-03-15"→Day. No explicit granularity field required. -
Structured / raw ingest (raw
IngestClaimRequestin Rust, Pythoningest_claimdict, MCPingest_claimtool) — granularity is not inferred. The caller must setstart_granularityandend_granularityexplicitly inside thevalid_timeblock. If omitted, the fields default toNone(legacy / no granularity declared). -
Legacy rows — claims ingested before this feature have
Nonegranularity on both ends. They display using the YYYY-MM-DD fallback. No migration or backfill is required.
Persistence and conformance:
-
Both SQLite (V3 migration) and PostgreSQL (V3 migration) store the two nullable granularity columns. The shared
run_granularity_conformanceharness inmempill-coreproves cross-adapter parity across three scenarios (Month/open, Day/Year, None/None) alongside the existingrun_persistence_conformance,run_history_conformance, andrun_valid_at_conformancesuites. Per-adapter round-trip tests (granularity_roundtrip) cover all fourDateGranularitylevels. -
The fold (succession) correctly preserves granularity through the full write → read cycle.
Transaction-time as-of correctness
Section titled “Transaction-time as-of correctness”- Ledger/disposition lookup and claim load (
load_subject_line) now honor anas_of_tx_timecutoff end-to-end. Claims ingested after the as-of timestamp are excluded at the persistence layer, not filtered post-hoc. This meansas_of_tx_timecorrectly rewinds only the transaction-time axis;valid_atindependently controls the valid-time axis.
Subject-scoped enumeration (query_subject)
Section titled “Subject-scoped enumeration (query_subject)”- New read path:
query_subjectreturns the resolved belief for every predicate known about a subject, without the caller needing to know the predicate names upfront. One entry per distinct predicate, sorted lexicographically for stable, deterministic output. - Bi-temporal aware: accepts the same
valid_atandas_of_tx_timeparameters asquery_memory, applied per-predicate during the fold. - Each entry carries:
predicate,value,status(Resolved/Contested/NoBelief/TimingUncertain),valid_from_display,valid_until_display,provenance,claim_ref, andconf(value confidence). - Exposed in the Rust core API (
EngineHandle::query_subject) and the Python wheel (engine.query_subject({...})). Not yet exposed as an MCP tool.
Date granularity on history()
Section titled “Date granularity on history()”The per-endpoint date-granularity feature (see Date granularity
above) now covers query_history / history() — previously the only honest-display read
path that dropped stored precision.
HistoryEntry(Rust core DTO) gained two additive fields:valid_from_granularityandvalid_until_granularity(Option<DateGranularity>), populated from the underlying claim’sValidTime. The Rust facade’shistory()passes both through verbatim (HistoryEntryis a direct re-export of the core type — no facade-level change was needed).- Derived-endpoint rule for
valid_until_granularity. Unlikevalid_from_granularity(always the entry’s ownstart_granularity),valid_untilis a derived bound — the canonical ordering key of the successor claim in the timeline (supersession bounding), not a value stored on this entry. The honest choice is therefore the successor’s granularity, not this entry’s own (never-populated)end_granularity:- If the successor’s ordering key is its
valid_time.start(successor’s valid-time confidence meets the engine threshold),valid_until_granularity= the successor’sstart_granularity. - If the successor’s ordering key falls back to its
transaction_time(low valid-time confidence),valid_until_granularityisNone— a machine-assigned transaction timestamp has no user-supplied date precision to report. - The last (open-ended /
Current) entry always hasvalid_until_granularity = None.
- If the successor’s ordering key is its
- Python enrichment.
query_historyresponses are enriched the same wayquery_memoryresponses are (enrich_query_historymirrorsenrich_query_memoryindisplay.rs): every entry gains pre-renderedvalid_from_display/valid_until_displaystrings, rendered via the sameformat_valid_time_endpointhelper — identical rendering rules to belief reads ("2020-03"for Month,"2020"for Year,"2020-03-15"for Day/Instant/legacy). The Python ergonomicHistoryEntrydataclass and.pyistubs were updated to include all four new fields;stubtestremains clean. - Conformance: a new
run_history_granularity_conformanceharness (mirrors the existingrun_granularity_conformance/run_history_conformancesuites) exercisesQueryHistoryUseCaseend-to-end against both SQLite and Postgres (16 and 18), including a three-way Month→Day→Year succession and an explicit assertion that the derivedvalid_until_granularitycarries the successor’s precision, never the predecessor’s. - Non-breaking: purely additive fields; no renames or removals.
Python TypedDicts
Section titled “Python TypedDicts”ValidTimeDict,BeliefProjection,FactDict,BeliefSlotadded tomempill.typesfor IDE completion and mypy.
0.5.0 and beyond — planned
Section titled “0.5.0 and beyond — planned”- Compliance wedge (audit/export tooling for regulated verticals) — planned
- TypeScript bindings (napi-rs) — planned
- Vector search integration (sqlite-vec / pgvector) — planned
- PostgreSQL TLS support — planned (NoTls only today)
- API/authoritative-source oracle reference implementation — planned
pdoc-generated Python API reference athttps://api.mempill.dev/python/— planned
0.2.0 — first published release
Section titled “0.2.0 — first published release”446 Rust + ~70 Postgres (feature-gated) + 135 Python tests · 0 warnings (clippy --all-targets -D warnings + missing_docs) · MSRV Rust 1.88 · Apache-2.0
(Current main counts: 536 Rust (+3 ignored) + 88 Postgres-gated + ~165 Python + 19 MCP — see 0.4.0 above.)
Cross-adapter conformance: SQLite + PostgreSQL 16 + PostgreSQL 18.4 via testcontainers.
This release ships the full mempill stack end-to-end:
Rust core engine (mempill-core)
Section titled “Rust core engine (mempill-core)”EngineHandle<P, O, V>— the sole async entry point. Eight deterministic engine components C1–C8:- C1: Provenance Gate — validates and classifies provenance.
- C2: Canonical Fold — read-time belief computation (I8: canonical, never stored).
- C3: Conflict Detector — detects
Functionalcardinality conflicts. - C4: Reconciler — attempts deterministic conflict resolution.
- C5: Read Path (
query_memory,query_audit). - C6: Amplification Guard — blocks
RecallReEntryechoes. - C7: Adjudication Gate — oracle escalation when C4 cannot resolve.
- C8: Ledger — append-only audit log (I1: no UPDATE/DELETE).
- All port traits:
PersistencePort,OraclePort(withNoOpOracle),VectorPort(withNoOpVector). - Application use-cases:
IngestClaimUseCase,QueryMemoryUseCase,ReconcileUseCase,AuditUseCase,SubmitAdjudicationUseCase,SweepAdjudicationsUseCase. - Public DTOs:
IngestClaimRequest/Response,QueryMemoryRequest/Response,ReconcileRequest/Response,AuditQueryRequest/Response. mempill-types:ProvenanceLabel,Disposition(12-state),Cardinality,Confidence,Criticality,ValidTime,BeliefProjection,LedgerEntry,ClaimRef,AgentId.- 11 invariants enforced: I1 (append-only), I2 (low-conf → Contested), I3 (fold not stored), I5 (oracle proposal only), I7 (overlap → Contested), I8 (canonical fold), I9 (atomic commit), and others.
Facade crate (mempill)
Section titled “Facade crate (mempill)”cargo add mempillre-exportsmempill-core+mempill-sqlitefor the common embedded case.
SQLite adapter (mempill-sqlite)
Section titled “SQLite adapter (mempill-sqlite)”SqlitePersistenceStore,DefaultEnginetype alias.open_default(path),open_default_in_memory().- Mandatory PRAGMAs at connection open:
WAL + synchronous=FULL + foreign_keys=ON.
PostgreSQL adapter (mempill-postgres)
Section titled “PostgreSQL adapter (mempill-postgres)”PostgresPersistenceStoreimplementingPersistencePortviapostgres+r2d2connection pool (max 20 connections).open_postgres(conn_str, oracle, vector, config)andopen_postgres_with_oracle(...)constructors.PostgresEngine<O, V>type alias.- Per-agent write serialization:
pg_advisory_xact_lock(hashtext(agent_id)::bigint). - OCC belt-and-suspenders:
UNIQUE(agent_id, stream_seq)onledger_entries. requires_global_write_serialization()returnsfalse— no global application lock; true per-agent concurrency across agents sharing one database.- Schema embedded at compile time via
refinery::embed_migrations!; runs automatically on first connection. - Cross-adapter conformance harness (
run_persistence_conformance,run_oracle_conformance): SQLite and Postgres pass the same behavioral test suite. - Known limitation:
NoTlsonly — TLS planned for a future release. Do not expose the connection over an untrusted network.
Oracle resolution loop
Section titled “Oracle resolution loop”submit_adjudicationwithAffirm/Deny/Unknownverdicts — both SQLite and PostgreSQL adapters.- TTL on pending adjudications;
sweep_expired_adjudications()orphan sweep. EngineHandle::list_pending_adjudications(agent_id)— returns the durable pending queue with incumbent and challenger values decoded.PyOracleEngine.list_pending_adjudications(agent_id=...)— Python binding.
Valid-time succession
Section titled “Valid-time succession”ConflictType::Succession— when a new claim’s valid-time window starts after the incumbent’s ends, and both havevalid_time_confidence ≥ 0.7, the engine commits the new claim asCommittedCheap(notContested) and supersedes the old one cleanly.- Fold selects the claim whose valid-time window contains the query instant (now or
as_of_tx_time) —query_memoryreturns the temporally-correct belief without additional filtering by the caller. - Succession matrix test suite: now/past/boundary/gap/n-chain correctness, cross-adapter conformance (SQLite + Postgres 16 + Postgres 18).
- Overlapping windows →
Contested(no regression). - Low-confidence valid-time (
valid_time_confidence < 0.7) →Contested. - Claims without
valid_time→Contestedwhen conflicting. - Note:
as_of_tx_timecontrols the transaction-time axis. A separatevalid_atparameter for the valid-time axis is available on main and ships in 0.3.0 (see above).
Bi-temporal history read
Section titled “Bi-temporal history read”history()(Rust facade) /history()(Python ergonomic tier) retrieves the full ordered claim timeline for a(subject, predicate)subject-line.- Returns
History/HistoryResultwithentries: Vec<HistoryEntry>(Rust) orentries: list[HistoryEntry](Python), ordered oldest→newest. - Each
HistoryEntrycarries:value,valid_from,valid_until,status(Current|Superseded),provenance,value_confidence,claim_ref,valid_from_granularity,valid_until_granularity(0.4.0 — see Date granularity onhistory()below). history().current()returns the singleCurrententry — guaranteed to agree withrecall()(same canonical fold).history().is_empty()— true when no claims exist for the subject-line.- Python
Historyis iterable:for e in history(engine, agent, subject, predicate). - Point-in-time valid-time querying via
valid_atis available on main and ships in 0.3.0.
Python wheel (mempill-python)
Section titled “Python wheel (mempill-python)”- PyO3/maturin wheel (PyO3 0.29, Python ≥ 3.11, abi3).
mempill.open(path),mempill.open_in_memory(),mempill.open_oracle(path, oracle),mempill.open_oracle_in_memory(oracle).- Duck-typed oracle protocol:
request_adjudication(self, agent_id: str, request: dict) -> str. submit_adjudication({handle_id, verdict, evidence_provenance}).sweep_expired_adjudications().mempill.typesmodule:Disposition(12-statestrenum),ProvenanceLabel(factory),IngestClaimRequest,IngestClaimResponse,QueryMemoryRequest,QueryMemoryResponse,ReconcileRequest,ReconcileResponse,AuditQueryRequest,AuditQueryResponseTypedDicts.- Exception hierarchy:
MempillError,ValidationError,NotFoundError,ConflictError,StorageError,ConfigError,InternalError. - Toolchain: maturin 1.14.1, PyO3 0.29.
MCP adapter (mempill-mcp)
Section titled “MCP adapter (mempill-mcp)”- FastMCP stdio server (FastMCP, mcp 1.28, pinned
<2). Four tools:ingest_claim,query_memory,reconcile,audit. MEMPILL_AGENT_IDrequired environment variable.MEMPILL_DB_PATHoptional (in-memory if absent).status_reasonfield on non-committed dispositions.- Friendly provenance string normalisation (
"External:UserAsserted"etc.).
Demo (mempill-demo)
Section titled “Demo (mempill-demo)”- Console + LangGraph agents.
HumanOraclereference implementation — stateless duck-typed oracle; generates a UUID handle per conflict; engine stores all conflict state in the durable pending queue./reviewREPL command — interactive human-in-the-loop adjudication:c(hallenger) → Affirm,i(ncumbent) → Deny,abstain → Unknown,skip → defer.--selftestflag for non-interactive CI validation.- Verified: deferred conflicts survive engine restart (same
handle_idafter close+reopen).
Development history
Section titled “Development history”The following notes track internal milestones during development toward 0.2.0. They are recorded for historical context only — the authoritative current state is the 0.2.0 section above.
| Internal phase | Highlights | Rust tests at that point |
|---|---|---|
| Core engine + SQLite | 8 engine components, 12-state disposition model, port traits | 290 |
| Python wheel + MCP | PyO3 wheel, FastMCP adapter | — |
| PostgreSQL adapter | topology-b, cross-adapter conformance (SQLite + PG16 + PG18) | 311 |
| Oracle resolution loop + HITL | submit_adjudication, HumanOracle, /review REPL |
430 |
| Valid-time succession | ConflictType::Succession, fold fix, succession matrix tests |
461 |
| Additional coverage | Oracle conformance, edge cases, cleanup | 462 |
| Ergonomic Tier-1 API + Postgres test gating | remember/recall + enrichment; PG tests feature-gated |
424 default · 69 PG · 107 Python |
| Bi-temporal history read | history() / query_history; full timeline with Current/Superseded status |
443 default · ~70 PG · 135 Python |
| Publish-hardening (API guidelines + Cargo checklist) | #[non_exhaustive] enum sweep, facade types::/engine:: modules, leak removal, compiled doctest, missing_docs gate |
446 default · ~70 PG · 135 Python (0.2.0 baseline) |
License
Section titled “License”Apache-2.0. Note: a commercial licensing option is planned for future releases.