| --- |
| title: Agent Memory Evaluation Map |
| emoji: 🧭 |
| colorFrom: indigo |
| colorTo: blue |
| sdk: static |
| app_file: index.html |
| license: mit |
| short_description: Evidence map for agent-memory evaluation without rankings. |
| tags: |
| - agent-memory |
| - evaluation |
| - knowledge-graphs |
| - mcp |
| --- |
| |
| # Agent Memory Evaluation Map |
|
|
| Choose the memory capability you need to evaluate, compare public evidence, and export a reproducible evaluation checklist. |
|
|
| This Space is a research map, not a leaderboard. It does not declare an overall winner or normalize incomparable reported results into one score. `Unknown` means the public source did not verify the field. `Not evaluated` means the artifact is useful context but does not itself report an evaluation for that outcome; neither label means failure. |
|
|
| ## Data policy |
|
|
| - Version 0.2 contains seven public papers already present in the public Mnemoverse Collection. |
| - Every artifact links to its Hugging Face paper page and any verified public code or project page. |
| - Evaluation claims follow the cited full paper version, not just its abstract. When HF bibliographic metadata lags that version, `metadata_source` pins the authoritative arXiv title and author list; otherwise the HF paper API is used. |
| - Code licenses are reported when the linked GitHub repository exposes a SPDX identifier through the GitHub API; dataset licenses are recorded separately when the linked dataset card declares them. A repository license is not assumed to govern the paper, benchmark data, or every dependency. |
| - Mnemoverse-authored work is visibly labeled. |
| - Only already-public evidence is included; no restricted research, concealed result, submission detail, or vendor ranking is included. |
| - Source checks are dated per artifact; the latest update is 2026-09-20, while MemOS and Harness the Memory retain their September 19 check dates. See [`SOURCES.md`](SOURCES.md) and [`data/artifacts.json`](data/artifacts.json). The manifest and recipe field `evidence_verified_at` denote the latest update, not a same-day recheck of every row; exported `artifact_verification_dates` preserve the individual dates. |
|
|
| The MIT license covers original files in this Space. Linked papers, repositories, datasets, and project pages remain under their own licenses and terms; links and factual summaries do not relicense upstream works. |
|
|
| ## Papers represented |
|
|
| The current map covers the seven public papers below. Inclusion means that the Space maps evidence from the paper; it is not an endorsement or an overall ranking. |
|
|
| - [Semantic Level of Detail for Knowledge Graphs: Discovering Abstraction Boundaries via Spectral Heat Diffusion](https://huggingface.co/papers/2603.08965) |
| - [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://huggingface.co/papers/2410.10813) |
| - [Zep: A Temporal Knowledge Graph Architecture for Agent Memory](https://huggingface.co/papers/2501.13956) |
| - [AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agents](https://huggingface.co/papers/2407.04363) |
| - [MemOS: A Memory OS for AI System](https://huggingface.co/papers/2507.03724) |
| - [Graph-based Agent Memory: Taxonomy, Techniques, and Applications](https://huggingface.co/papers/2602.05665) |
| - [Harness the Memory: A Holistic Evaluation of Memory Substrates in Memory Agents](https://huggingface.co/papers/2608.15008) |
|
|
| ## Local preview |
|
|
| Serve this directory over HTTP, for example: |
|
|
| ```bash |
| python3 -m http.server 4173 |
| ``` |
|
|
| Then open `http://127.0.0.1:4173/`. |
|
|
| ## Preflight |
|
|
| Run the dependency-free release check before creating or updating the Space: |
|
|
| ```bash |
| node scripts/verify-space.mjs |
| ``` |
|
|
| Add `--check-urls` when public-network checks are appropriate. The preflight validates the Static Space metadata, local application references, manifest invariants, evaluation-target coverage, HTTPS evidence links, schema identifiers, exact titles/authors against the declared metadata source, and GitHub SPDX identifiers for implementation repositories. It does not validate scientific claims or reproduce experiments. See [`DEPLOYMENT-CHECKLIST.md`](DEPLOYMENT-CHECKLIST.md) for functional release checks. |
|
|
| ## Repository contents |
|
|
| - `index.html`: accessible application shell |
| - `app.js`: filters, comparison, deterministic recipe builder, downloads |
| - `styles.css`: responsive presentation |
| - `data/artifacts.json`: versioned public evidence manifest |
| - `data/schema.json`: JSON Schema for the manifest |
| - `SOURCES.md`: human-readable evidence trail |
| - `scripts/verify-space.mjs`: dependency-free release preflight |
| - `DEPLOYMENT-CHECKLIST.md`: reusable technical release checks |
|
|
| ## Citation and contribution policy |
|
|
| Follow the primary links on each card before using a claim in a paper or system decision. To propose a correction, provide the artifact ID, field, primary URL, and the exact source passage. Fields remain `unknown` until a primary source verifies them. |
|
|
