Skip to content

Changelog

mempill follows semantic versioning. The first published release is 0.2.0. The latest published release on crates.io and PyPI is 0.3.0. 0.4.0 below is unreleased — it exists on the main branch only. See Installation for install instructions and how to use the unreleased API today.


536 Rust (+3 ignored) + 88 Postgres-gated + ~165 Python + 19 MCP tests · 0 warnings

Breaking change — per-agent SQLite entry points

Section titled “Breaking change — per-agent SQLite entry points”

mempill-sqlite’s public constructors are now per-agent, replacing the prior path-based API. The raw path-based connection::open is no longer public.

Before (removed) After (0.4.0)
open_default(path) (Rust, mempill-sqlite) open_default_for_agent(base_dir, agent_id)
open_with_oracle(path, oracle) (Rust, mempill-sqlite) open_with_oracle_for_agent(base_dir, agent_id, oracle)
mempill.open(path) (Python) mempill.open_for_agent(base_dir, agent_id)
mempill.open_oracle(path, oracle) (Python) mempill.open_oracle_for_agent(base_dir, agent_id, oracle)
MEMPILL_DB_PATH (mempill-mcp env var, full file path) MEMPILL_DB_DIR (base directory)

The database file is now always derived as base_dir/agent_{agent_id}.db, which makes it structurally impossible for two different agent_id values to collide on the same file. agent_id is validated against [A-Za-z0-9_-]; an invalid agent_id raises SqliteStoreError::InvalidAgentId (Rust) / StorageError (Python) before any file I/O. open_default_in_memory() / open_with_oracle_in_memory() / mempill.open_in_memory() / mempill.open_oracle_in_memory() are unaffected — in-memory engines have no file to collide on.

PostgreSQL entry points are unaffected. open_postgres / open_postgres_with_oracle keep their existing signatures — PostgreSQL was already agent-scoped via advisory locking, not file naming.

Migration for existing pre-0.4.0 databases: a pre-0.4.0 shared-file database is not auto-migrated. Point MEMPILL_DB_DIR / base_dir at a fresh directory, or manually copy the existing database file to base_dir/agent_{agent_id}.db before first use with the new API.

See Quickstart (Rust), Quickstart (Python), and MCP Integration for the updated examples.

  • New PoolConfig struct (max_size: u32, connection_timeout: Duration) and an additive PostgresPersistenceStore::with_pool_config(conn_str, pool_config) constructor.
  • PostgresPersistenceStore::new is unchanged and delegates to with_pool_config with PoolConfig::default() — existing callers see no behavior change. Defaults match the prior hardcoded values: max_size = 20, connection_timeout = 5s.
  • max_size == 0 is rejected with PostgresStoreError::Config before any network or database connection is attempted.

See PostgreSQL Backend for usage.

  • Write and audit paths (ingest_claim, reconcile, submit_adjudication, sweep_adjudications) now use an uncapped, claim-scoped load_ledger_for_claims lookup instead of a capped, agent-wide load_ledger(agent_id, None, 10_000) call.
  • What this fixes: on an agent with more than 10,000 ledger rows, disposition-changing ledger entries beyond the cap window were previously invisible to fold/state-guard logic, which could produce an incorrect belief. Correctness of these paths is now independent of agent ledger history size.
  • New reproducible, assertion-based benchmark exercising both the valid-time and transaction-time axes independently, including the 3-way succession chain and the honest Contested case (a genuine conflict that is not silently resolved).

  • Single reproduction command, no external data or environment-specific setup required:

    Terminal window
    cargo run --release --example asof_correctness_benchmark -p mempill
  • This is a correctness benchmark only — every scenario is a hard assertion against a documented expected outcome. No timing, latency, or throughput numbers are captured or published.

  • Full results: As-Of / Bi-Temporal Correctness Benchmark.


Published on crates.io and PyPI — the latest published release.

  • valid_at: Option<DateTime<Utc>> added to QueryMemoryRequest — a point-in-time valid-time filter independent of as_of_tx_time. Enables queries such as “who was CEO on 2021-06-01?” where valid_at selects by real-world validity window and as_of_tx_time selects by when the engine recorded the claim. The two axes are now fully independent.
  • Exposed in the Rust core API, the Python wheel (query_memory dict), and the MCP query_memory tool (valid_at? parameter).

Date granularity (per-endpoint, display-only)

Section titled “Date granularity (per-endpoint, display-only)”

mempill now records the precision of every valid-time boundary independently, and renders it back honestly — no fabricated precision is ever inserted.

What shipped:

  • DateGranularity enum with four levels: Year, Month, Day, Instant. Added additively to ValidTime as two independent nullable fields: start_granularity and end_granularity. The start and end boundaries of a valid-time interval can carry different precision.

  • Honest display. A render helper (format_valid_time_endpoint) formats each boundary according to its recorded granularity:

    Granularity Display format Example
    Year YYYY "2020"
    Month YYYY-MM "2020-03"
    Day YYYY-MM-DD "2020-03-15"
    Instant YYYY-MM-DD (day precision) "2024-05-15"
    None (legacy) YYYY-MM-DD (fallback) "2020-03-01"

    A Month-granularity boundary is never displayed as "2020-03-01" — doing so would imply day precision the caller never supplied.

  • Pre-rendered display strings. Every belief slot exposes valid_from_display and valid_until_display, plus the raw start_granularity / end_granularity tags inside its valid_time object. Python and MCP surface these directly in every query_memory response, so callers do not need to implement display logic.

Ingest contract — infer vs. explicit:

  • Ergonomic Rust remember() / RememberOptions — infers granularity automatically from the supplied date string via the parse_valid_time_date parser: "2020"Year, "2020-03"Month, "2020-03-15"Day. No explicit granularity field required.

  • Structured / raw ingest (raw IngestClaimRequest in Rust, Python ingest_claim dict, MCP ingest_claim tool) — granularity is not inferred. The caller must set start_granularity and end_granularity explicitly inside the valid_time block. If omitted, the fields default to None (legacy / no granularity declared).

  • Legacy rows — claims ingested before this feature have None granularity on both ends. They display using the YYYY-MM-DD fallback. No migration or backfill is required.

Persistence and conformance:

  • Both SQLite (V3 migration) and PostgreSQL (V3 migration) store the two nullable granularity columns. The shared run_granularity_conformance harness in mempill-core proves cross-adapter parity across three scenarios (Month/open, Day/Year, None/None) alongside the existing run_persistence_conformance, run_history_conformance, and run_valid_at_conformance suites. Per-adapter round-trip tests (granularity_roundtrip) cover all four DateGranularity levels.

  • The fold (succession) correctly preserves granularity through the full write → read cycle.

  • Ledger/disposition lookup and claim load (load_subject_line) now honor an as_of_tx_time cutoff end-to-end. Claims ingested after the as-of timestamp are excluded at the persistence layer, not filtered post-hoc. This means as_of_tx_time correctly rewinds only the transaction-time axis; valid_at independently controls the valid-time axis.

Subject-scoped enumeration (query_subject)

Section titled “Subject-scoped enumeration (query_subject)”
  • New read path: query_subject returns the resolved belief for every predicate known about a subject, without the caller needing to know the predicate names upfront. One entry per distinct predicate, sorted lexicographically for stable, deterministic output.
  • Bi-temporal aware: accepts the same valid_at and as_of_tx_time parameters as query_memory, applied per-predicate during the fold.
  • Each entry carries: predicate, value, status (Resolved / Contested / NoBelief / TimingUncertain), valid_from_display, valid_until_display, provenance, claim_ref, and conf (value confidence).
  • Exposed in the Rust core API (EngineHandle::query_subject) and the Python wheel (engine.query_subject({...})). Not yet exposed as an MCP tool.

The per-endpoint date-granularity feature (see Date granularity above) now covers query_history / history() — previously the only honest-display read path that dropped stored precision.

  • HistoryEntry (Rust core DTO) gained two additive fields: valid_from_granularity and valid_until_granularity (Option<DateGranularity>), populated from the underlying claim’s ValidTime. The Rust facade’s history() passes both through verbatim (HistoryEntry is a direct re-export of the core type — no facade-level change was needed).
  • Derived-endpoint rule for valid_until_granularity. Unlike valid_from_granularity (always the entry’s own start_granularity), valid_until is a derived bound — the canonical ordering key of the successor claim in the timeline (supersession bounding), not a value stored on this entry. The honest choice is therefore the successor’s granularity, not this entry’s own (never-populated) end_granularity:
    • If the successor’s ordering key is its valid_time.start (successor’s valid-time confidence meets the engine threshold), valid_until_granularity = the successor’s start_granularity.
    • If the successor’s ordering key falls back to its transaction_time (low valid-time confidence), valid_until_granularity is None — a machine-assigned transaction timestamp has no user-supplied date precision to report.
    • The last (open-ended / Current) entry always has valid_until_granularity = None.
  • Python enrichment. query_history responses are enriched the same way query_memory responses are (enrich_query_history mirrors enrich_query_memory in display.rs): every entry gains pre-rendered valid_from_display / valid_until_display strings, rendered via the same format_valid_time_endpoint helper — identical rendering rules to belief reads ("2020-03" for Month, "2020" for Year, "2020-03-15" for Day/Instant/legacy). The Python ergonomic HistoryEntry dataclass and .pyi stubs were updated to include all four new fields; stubtest remains clean.
  • Conformance: a new run_history_granularity_conformance harness (mirrors the existing run_granularity_conformance / run_history_conformance suites) exercises QueryHistoryUseCase end-to-end against both SQLite and Postgres (16 and 18), including a three-way Month→Day→Year succession and an explicit assertion that the derived valid_until_granularity carries the successor’s precision, never the predecessor’s.
  • Non-breaking: purely additive fields; no renames or removals.
  • ValidTimeDict, BeliefProjection, FactDict, BeliefSlot added to mempill.types for IDE completion and mypy.

  • Compliance wedge (audit/export tooling for regulated verticals) — planned
  • TypeScript bindings (napi-rs) — planned
  • Vector search integration (sqlite-vec / pgvector) — planned
  • PostgreSQL TLS support — planned (NoTls only today)
  • API/authoritative-source oracle reference implementation — planned
  • pdoc-generated Python API reference at https://api.mempill.dev/python/ — planned

446 Rust + ~70 Postgres (feature-gated) + 135 Python tests · 0 warnings (clippy --all-targets -D warnings + missing_docs) · MSRV Rust 1.88 · Apache-2.0

(Current main counts: 536 Rust (+3 ignored) + 88 Postgres-gated + ~165 Python + 19 MCP — see 0.4.0 above.)

Cross-adapter conformance: SQLite + PostgreSQL 16 + PostgreSQL 18.4 via testcontainers.

This release ships the full mempill stack end-to-end:

  • EngineHandle<P, O, V> — the sole async entry point. Eight deterministic engine components C1–C8:
    • C1: Provenance Gate — validates and classifies provenance.
    • C2: Canonical Fold — read-time belief computation (I8: canonical, never stored).
    • C3: Conflict Detector — detects Functional cardinality conflicts.
    • C4: Reconciler — attempts deterministic conflict resolution.
    • C5: Read Path (query_memory, query_audit).
    • C6: Amplification Guard — blocks RecallReEntry echoes.
    • C7: Adjudication Gate — oracle escalation when C4 cannot resolve.
    • C8: Ledger — append-only audit log (I1: no UPDATE/DELETE).
  • All port traits: PersistencePort, OraclePort (with NoOpOracle), VectorPort (with NoOpVector).
  • Application use-cases: IngestClaimUseCase, QueryMemoryUseCase, ReconcileUseCase, AuditUseCase, SubmitAdjudicationUseCase, SweepAdjudicationsUseCase.
  • Public DTOs: IngestClaimRequest/Response, QueryMemoryRequest/Response, ReconcileRequest/Response, AuditQueryRequest/Response.
  • mempill-types: ProvenanceLabel, Disposition (12-state), Cardinality, Confidence, Criticality, ValidTime, BeliefProjection, LedgerEntry, ClaimRef, AgentId.
  • 11 invariants enforced: I1 (append-only), I2 (low-conf → Contested), I3 (fold not stored), I5 (oracle proposal only), I7 (overlap → Contested), I8 (canonical fold), I9 (atomic commit), and others.
  • cargo add mempill re-exports mempill-core + mempill-sqlite for the common embedded case.
  • SqlitePersistenceStore, DefaultEngine type alias.
  • open_default(path), open_default_in_memory().
  • Mandatory PRAGMAs at connection open: WAL + synchronous=FULL + foreign_keys=ON.
  • PostgresPersistenceStore implementing PersistencePort via postgres + r2d2 connection pool (max 20 connections).
  • open_postgres(conn_str, oracle, vector, config) and open_postgres_with_oracle(...) constructors.
  • PostgresEngine<O, V> type alias.
  • Per-agent write serialization: pg_advisory_xact_lock(hashtext(agent_id)::bigint).
  • OCC belt-and-suspenders: UNIQUE(agent_id, stream_seq) on ledger_entries.
  • requires_global_write_serialization() returns false — no global application lock; true per-agent concurrency across agents sharing one database.
  • Schema embedded at compile time via refinery::embed_migrations!; runs automatically on first connection.
  • Cross-adapter conformance harness (run_persistence_conformance, run_oracle_conformance): SQLite and Postgres pass the same behavioral test suite.
  • Known limitation: NoTls only — TLS planned for a future release. Do not expose the connection over an untrusted network.
  • submit_adjudication with Affirm/Deny/Unknown verdicts — both SQLite and PostgreSQL adapters.
  • TTL on pending adjudications; sweep_expired_adjudications() orphan sweep.
  • EngineHandle::list_pending_adjudications(agent_id) — returns the durable pending queue with incumbent and challenger values decoded.
  • PyOracleEngine.list_pending_adjudications(agent_id=...) — Python binding.
  • ConflictType::Succession — when a new claim’s valid-time window starts after the incumbent’s ends, and both have valid_time_confidence ≥ 0.7, the engine commits the new claim as CommittedCheap (not Contested) and supersedes the old one cleanly.
  • Fold selects the claim whose valid-time window contains the query instant (now or as_of_tx_time) — query_memory returns the temporally-correct belief without additional filtering by the caller.
  • Succession matrix test suite: now/past/boundary/gap/n-chain correctness, cross-adapter conformance (SQLite + Postgres 16 + Postgres 18).
  • Overlapping windows → Contested (no regression).
  • Low-confidence valid-time (valid_time_confidence < 0.7) → Contested.
  • Claims without valid_timeContested when conflicting.
  • Note: as_of_tx_time controls the transaction-time axis. A separate valid_at parameter for the valid-time axis is available on main and ships in 0.3.0 (see above).
  • history() (Rust facade) / history() (Python ergonomic tier) retrieves the full ordered claim timeline for a (subject, predicate) subject-line.
  • Returns History / HistoryResult with entries: Vec<HistoryEntry> (Rust) or entries: list[HistoryEntry] (Python), ordered oldest→newest.
  • Each HistoryEntry carries: value, valid_from, valid_until, status (Current | Superseded), provenance, value_confidence, claim_ref, valid_from_granularity, valid_until_granularity (0.4.0 — see Date granularity on history() below).
  • history().current() returns the single Current entry — guaranteed to agree with recall() (same canonical fold).
  • history().is_empty() — true when no claims exist for the subject-line.
  • Python History is iterable: for e in history(engine, agent, subject, predicate).
  • Point-in-time valid-time querying via valid_at is available on main and ships in 0.3.0.
  • PyO3/maturin wheel (PyO3 0.29, Python ≥ 3.11, abi3).
  • mempill.open(path), mempill.open_in_memory(), mempill.open_oracle(path, oracle), mempill.open_oracle_in_memory(oracle).
  • Duck-typed oracle protocol: request_adjudication(self, agent_id: str, request: dict) -> str.
  • submit_adjudication({handle_id, verdict, evidence_provenance}).
  • sweep_expired_adjudications().
  • mempill.types module: Disposition (12-state str enum), ProvenanceLabel (factory), IngestClaimRequest, IngestClaimResponse, QueryMemoryRequest, QueryMemoryResponse, ReconcileRequest, ReconcileResponse, AuditQueryRequest, AuditQueryResponse TypedDicts.
  • Exception hierarchy: MempillError, ValidationError, NotFoundError, ConflictError, StorageError, ConfigError, InternalError.
  • Toolchain: maturin 1.14.1, PyO3 0.29.
  • FastMCP stdio server (FastMCP, mcp 1.28, pinned <2). Four tools: ingest_claim, query_memory, reconcile, audit.
  • MEMPILL_AGENT_ID required environment variable.
  • MEMPILL_DB_PATH optional (in-memory if absent).
  • status_reason field on non-committed dispositions.
  • Friendly provenance string normalisation ("External:UserAsserted" etc.).
  • Console + LangGraph agents.
  • HumanOracle reference implementation — stateless duck-typed oracle; generates a UUID handle per conflict; engine stores all conflict state in the durable pending queue.
  • /review REPL command — interactive human-in-the-loop adjudication: c(hallenger) → Affirm, i(ncumbent) → Deny, abstain → Unknown, skip → defer.
  • --selftest flag for non-interactive CI validation.
  • Verified: deferred conflicts survive engine restart (same handle_id after close+reopen).

The following notes track internal milestones during development toward 0.2.0. They are recorded for historical context only — the authoritative current state is the 0.2.0 section above.

Internal phase Highlights Rust tests at that point
Core engine + SQLite 8 engine components, 12-state disposition model, port traits 290
Python wheel + MCP PyO3 wheel, FastMCP adapter
PostgreSQL adapter topology-b, cross-adapter conformance (SQLite + PG16 + PG18) 311
Oracle resolution loop + HITL submit_adjudication, HumanOracle, /review REPL 430
Valid-time succession ConflictType::Succession, fold fix, succession matrix tests 461
Additional coverage Oracle conformance, edge cases, cleanup 462
Ergonomic Tier-1 API + Postgres test gating remember/recall + enrichment; PG tests feature-gated 424 default · 69 PG · 107 Python
Bi-temporal history read history() / query_history; full timeline with Current/Superseded status 443 default · ~70 PG · 135 Python
Publish-hardening (API guidelines + Cargo checklist) #[non_exhaustive] enum sweep, facade types::/engine:: modules, leak removal, compiled doctest, missing_docs gate 446 default · ~70 PG · 135 Python (0.2.0 baseline)

Apache-2.0. Note: a commercial licensing option is planned for future releases.