SUM — Verifiable Bidirectional Knowledge Distillation

A cross-runtime trust surface for LLM-rendered text.

SUM focuses on one load-bearing claim: Python, Node, and modern browsers produce byte-identical Ed25519 signatures over the same JCS-canonical bytes for signed render receipts.

Each hosted render returns a detached-JWS receipt (sum.render_receipt.v1) that can be verified offline against JWKS.

View Repository • PyPI: sum-engine • Live demo (worker) • Proof Boundary

What Ships Today

• sum-engine on PyPI — pip install 'sum-engine[sieve]' — sum attest, sum verify, sum render, sum resolve, sum ledger, sum inspect, sum schema, plus the v0.7.0 additions: sum transform list / apply <name> (registry-dispatched transforms with sum.transform_receipt.v1), sum verify --explain (layered per-dimension report), and sum compliance check --regime <id> (six-regime validator), plus the later meaning-layer subcommands sum mint-meaning, sum mint-chain, and sum verify-meaning (research tier). 170 catalogued features in docs/FEATURE_CATALOG.md (151 production, 18 scaffolded, 1 designed), sourced from meta/repo_manifest.json at v0.12.0. v0.12.0 was tagged on 2026-10-21 and published to PyPI on 2026-10-22; SUM main’s manifest records pypi_published_version: 0.12.0 (updated after the tag, SUM#548). v0.11.0 was tagged on GitHub on 2026-09-16 and never published to PyPI, and SUM’s changelog marks it superseded by 0.12.0. The v0.8.0 pip install 'sum-engine[verify]' extra — a dependency-light (cryptography + joserfc, no numpy/torch) receipt verifier that checks SUM’s signed receipts and replays a meaning-risk certificate’s conformal bound offline.
• Cloudflare Worker APIs (/api/render, /api/transform, /api/complete, /api/qid) plus verification surfaces (/.well-known/jwks.json, /.well-known/revoked-kids.json). Per-IP rate limiter + BYO-key gate on public LLM-axis routes (5/24h operator-keyed demo, 100/hr with BYO key).
• Browser demo for in-browser attest/verify; same bytes verify under the Node verifier in standalone_verifier/.
• Cross-runtime trust triangle in CI (make xruntime + make xruntime-adversarial) — K1–K4 valid-input agreement, A1–A6 rejection-class equivalence, plus the 20-fixture transform K-matrix locking accept + reject for sum.transform_receipt.v1 across Python ↔ Node ↔ browser.
• MCP server (sum-mcp) exposing extract / attest / verify / inspect / render / schema over stdio, plus a meaning layer added after v0.8.1: verify_receipt (all five receipt schemas), meaning_diff, depth_frontier, and the BYO-key-only mint_meaning_receipt / mint_chain_receipt. The server never generates, stores, or logs key material.
• Certified multi-hop chains (sum.chain_receipt.v1, research tier) — one signed, offline-replayable certificate over an ordered sequence of sum.meaning_risk_receipt.v1 hops, with the per-hop bounds composed into an integer-exact Bonferroni budget and a mandatory budget_scope field the verifier fails closed without. The first real one covers two transforms over 32 CC0 BillSum bills; SUM performed only hop 2, and certifies the meaning-loss of a transform it did not perform in hop 1.
• T0 altitude panel on the live demo — one real BillSum bill descending four altitudes with the strict-NLI measured loss and a per-rung kept / dropped / added claim readout, under a “measured, not certified” header.
• Multi-provider cascade — LiveLLMAdapter.from_model routes by model-id prefix across OpenAI / Anthropic (Worker) / HF Inference Providers / NVIDIA NIM / Groq / Cerebras / Ollama / llama.cpp / local: (Modal, Fireworks, vLLM-on-anything). Free-tier options first-class.
• Replay-defense window — opt-in signed_at_out_of_window check on all four verifier surfaces (Python render / Python transform / JS render / JS transform). Default-off preserves archival use.
• Negative-control corpus (T5) — 20 hand-authored documents across five failure modes; runner exits 1 when the bench succeeds on inputs it should fail on.

Truthfulness Boundary (Explicit)

SUM makes a hard distinction between:

• Mechanically proven surfaces (canonical codec, cross-runtime signature equivalence, verifier parity), and
• Empirical LLM behavior (extraction quality, slider preservation, narrative round-trip fidelity).

The full LLM narrative round-trip (text → LLM-extract → axioms → LLM-generate → prose' → LLM-extract → axioms') was closed empirically across three measured corpora on 2026-04-28 under a two-layer intervention (canonical-first generator prompt + constrained-decoding extractor + lemma-exclusion of source-predicate lemmas from the canonical-padding set). Combined results: seed_v1 recall 1.0000 / drift 0.00%, seed_v2 0.9750 / 5.00%, seed_long_paragraphs 0.9972 / 0.57%. The unprompted baseline (no intervention) stands at the documented 107.75% drift / 0.12 recall on seed_v1 — preserved as the “no-intervention” reference point. Empirical closure on three corpora is not the same as corpus-independent proof: the deterministic round-trip is mechanically proven (§1.1); the LLM round-trip is closed empirically (§2.5). Full attribution and bench receipts in docs/PROOF_BOUNDARY.md.

Technical Surface

• Core package: Python 3.10+ (sum-engine)
• Trust/verification: Ed25519 + JCS canonicalization + detached JWS receipts
• Runtime parity: Python CLI verifier, Node verifier, in-browser verifier
• Infra: Cloudflare Worker for hosted render/verification endpoints
• Protocol surfaces: CanonicalBundle format, render receipts, MCP tool server

Why It Matters

SUM is built for cases where downstream systems need proof that output was produced and signed by a specific issuer under a reproducible canonical format — without pretending that cryptographic attestation alone proves semantic truth.

That explicit separation between attestation guarantees and semantic-quality benchmarks is the core product stance.

License

Apache 2.0.