InfiniteContext — Tiered Memory for AI

Status: public, MIT-licensed, early-stage (v0.1.0). The library is inspectable and exercised by a Jest test suite with CI on Node 20/22, but it’s pre-1.0 with no stable release. It’s distributed as source on GitHub — there is no published npm package under this name (that name on the registry belongs to an unrelated author). The “What’s Needed To Tighten” section is the honest gap list.

InfiniteContext is a TypeScript memory layer that lets AI applications work past the context window — storing, organizing, and retrieving large amounts of information across storage tiers instead of cramming everything into one prompt. It ships as a Node library and a CLI (infinite-context), with OpenAI wired in for embeddings and summarization.

Three Hierarchies

The system layers at three distinct levels:

1. Storage tiers — a Memory Router fronts pluggable providers (local disk today, Google Drive via the cloud provider, with a common StorageProvider interface for adding more), so hot and cold memory live where they belong.
2. Multi-level summarization — a SummarizationEngine compresses content into summaries at several levels of abstraction, so long histories can be served without replaying everything verbatim.
3. Hierarchical bucket organization — content is filed into a domain/topic bucket tree and retrieved through a HierarchicalRetriever, with automatic categorization deciding where new content lands.

What’s Implemented Today

• Vector retrieval — exact flat index artifacts that persist and support rebuild / merge / split / save / load, plus an approximate HNSW backend (hnswlib-node, with graceful flat fallback) that the hierarchical retriever switches to per-trace above a ~2,000-item threshold for sub-linear search at scale. A committed, key-free benchmark (docs/retrieval-benchmark.md) measures the HNSW backend against exact flat search on synthetic random unit vectors — a near-worst-case ANN workload, since uniformly-distributed high-dimensional neighbours are nearly equidistant. It reports a trade-off, not a free win: HNSW latency stays sub-millisecond at every corpus size while flat search scales linearly (1.3× p50 speedup at N=1,000, 5.5× at 10k, 24.2× at 100k), but at a fixed efSearch=100 recall@10 falls from 0.995 (1k) to 0.766 (10k) to 0.307 (100k) — at 100k the approximate index returns roughly 3 of the true top-10. Recall is tunable by raising efSearch/M at the cost of latency; the benchmark documents the knob, not a tuned operating point.
• Hierarchical bucket store — domain→topic organization with a multi-strategy retriever, covered by tests.
• Automatic categorization — a PromptCategorizer routes content using keyword, vector-similarity, and adaptive strategies, with a category cache.
• Multi-level summarization — OpenAI-driven summaries at configurable abstraction levels (real chat.completions calls), with a deterministic extractive fallback when no LLM client is configured.
• Negation-aware retrieval — Widdows orthogonal negation, so “X but not Y” queries subtract the negated subspace (with a design note connecting it to compositional/DisCoCat meaning).
• Pluggable storage — local and Google Drive providers behind one interface.
• Memory governance with signed receipts — every class of stored memory (chunks, hand-authored profile facts, and auto-extracted profile aggregates) answers to one safety pipeline for redaction, deletion with tombstones, retention, and export. Each governed mutation and export emits an append-only Ed25519 governance receipt over canonically-serialized bytes, hash-chained to its predecessor so altering a past receipt invalidates every receipt after it. Signing keys rotate, with retired keys retained in an on-disk keyring and exposed as a JWKS accessor, so receipts signed before a rotation still verify. Erasure is durable: mutations rewrite the stored blob itself, so a deleted or redacted chunk stays gone across a restart rather than being reloaded from an untouched provider blob. A receipt attests that an operation occurred and was signed by the holder of a given key at a given time — it asserts nothing about the truthfulness of the content, and anyone holding the private key could forge a consistent chain.
• Robustness layer — transaction management with rollback, data-integrity verification, backup/recovery, and data portability (JSON / JSONL / CSV import-export).

The receipt model is borrowed deliberately from SUM, which uses Ed25519 attestation receipts over canonical bytes for text transformations; the governed operations differ (prose↔axiom transforms there, memory deletion / redaction / export here) but the mechanism and its stated proof boundary carry over. The two ADRs record that reasoning, and docs/AUDIT.md records the audit that caught the receipts attesting deletions which had not yet reached the storage layer — the gap the durable-erasure work closed.

Technical Stack

• Language: TypeScript (Node library + infinite-context CLI)
• Vector index: exact flat (persisted artifacts) + approximate HNSW via hnswlib-node for retrieval at scale (IVF reserved)
• Embeddings / summarization: OpenAI (text-embedding-3-small by default)
• Storage: local filesystem, Google Drive; SQLite / MongoDB / Redis client deps present for backends
• Tests / CI: Jest, GitHub Actions on Node 20 + 22
• License: MIT
• Docs: API, Architecture, Categorization, Compositional Meaning, Memory Monitoring, Extending, llamafile, macOS setup, ADRs (profile-memory governance, governance receipts), Audit, Retrieval benchmark

What’s Needed For This Entry To Tighten

• A tagged stable release (it’s at v0.1.0). An install path already exists if wanted: the bare npm name infinite-context is taken by another author, but the package’s actual scoped name @ototao/infinite-context is unpublished (free) — so shipping to npm is a choice, not a blocker.
• A hosted demo or a recorded end-to-end run (ingest → store → retrieve) a visitor can verify without cloning.
• A labeled retrieval-quality eval. The committed benchmark above measures HNSW’s fidelity to exact search (approximation quality + speedup), not end-to-end answer quality against ground-truth relevance — that’s the next benchmark to add.