InfiniteContext — Tiered Memory for AI

Status: public, MIT-licensed, early-stage (v0.1.0). The library is inspectable and exercised by a Jest test suite with CI on Node 20/22, but it’s pre-1.0 with no stable release. It’s distributed as source on GitHub — there is no published npm package under this name (that name on the registry belongs to an unrelated author). The “What’s Needed To Tighten” section is the honest gap list.

InfiniteContext is a TypeScript memory layer that lets AI applications work past the context window — storing, organizing, and retrieving large amounts of information across storage tiers instead of cramming everything into one prompt. It ships as a Node library and a CLI (infinite-context), with OpenAI wired in for embeddings and summarization.

Three Hierarchies

The system layers at three distinct levels:

1. Storage tiers — a Memory Router fronts pluggable providers (local disk today, Google Drive via the cloud provider, with a common StorageProvider interface for adding more), so hot and cold memory live where they belong.
2. Multi-level summarization — a SummarizationEngine compresses content into summaries at several levels of abstraction, so long histories can be served without replaying everything verbatim.
3. Hierarchical bucket organization — content is filed into a domain/topic bucket tree and retrieved through a HierarchicalRetriever, with automatic categorization deciding where new content lands.

What’s Implemented Today

• Vector retrieval — exact flat index artifacts that persist and support rebuild / merge / split / save / load, plus an approximate HNSW backend (hnswlib-node, with graceful flat fallback) that the hierarchical retriever switches to per-trace above a ~2,000-item threshold for sub-linear search at scale. A committed, key-free benchmark (docs/retrieval-benchmark.md) measures the HNSW backend against exact flat search: recall@10 = 0.995 at N=1,000 (≈1.3× p50 speedup), scaling to ~24× p50 speedup at 100k vectors — i.e. near-exact results at a fraction of the latency.
• Hierarchical bucket store — domain→topic organization with a multi-strategy retriever, covered by tests.
• Automatic categorization — a PromptCategorizer routes content using keyword, vector-similarity, and adaptive strategies, with a category cache.
• Multi-level summarization — OpenAI-driven summaries at configurable abstraction levels (real chat.completions calls), with a deterministic extractive fallback when no LLM client is configured.
• Negation-aware retrieval — Widdows orthogonal negation, so “X but not Y” queries subtract the negated subspace (with a design note connecting it to compositional/DisCoCat meaning).
• Pluggable storage — local and Google Drive providers behind one interface.
• Robustness layer — transaction management with rollback, data-integrity verification, backup/recovery, and data portability (JSON / JSONL / CSV import-export).

Technical Stack

• Language: TypeScript (Node library + infinite-context CLI)
• Vector index: exact flat (persisted artifacts) + approximate HNSW via hnswlib-node for retrieval at scale (IVF reserved)
• Embeddings / summarization: OpenAI (text-embedding-3-small by default)
• Storage: local filesystem, Google Drive; SQLite / MongoDB / Redis client deps present for backends
• Tests / CI: Jest, GitHub Actions on Node 20 + 22
• License: MIT
• Docs: API, Architecture, Categorization, Compositional Meaning, Memory Monitoring, Extending, llamafile, macOS setup

What’s Needed For This Entry To Tighten

• A tagged stable release (it’s at v0.1.0). An install path already exists if wanted: the bare npm name infinite-context is taken by another author, but the package’s actual scoped name @ototao/infinite-context is unpublished (free) — so shipping to npm is a choice, not a blocker.
• A hosted demo or a recorded end-to-end run (ingest → store → retrieve) a visitor can verify without cloning.
• A labeled retrieval-quality eval. The committed benchmark above measures HNSW’s fidelity to exact search (approximation quality + speedup), not end-to-end answer quality against ground-truth relevance — that’s the next benchmark to add.