File size: 4,896 Bytes
05be922 37fa745 05be922 37fa745 05be922 37fa745 05be922 37fa745 281ef6a 37fa745 281ef6a 37fa745 281ef6a 37fa745 281ef6a 37fa745 281ef6a 37fa745 281ef6a 37fa745 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 | ---
title: Agent Memory Evaluation Map
emoji: 🧭
colorFrom: indigo
colorTo: blue
sdk: static
app_file: index.html
license: mit
short_description: Evidence map for agent-memory evaluation without rankings.
tags:
- agent-memory
- evaluation
- knowledge-graphs
- mcp
---
# Agent Memory Evaluation Map
Choose the memory capability you need to evaluate, compare public evidence, and export a reproducible evaluation checklist.
This Space is a research map, not a leaderboard. It does not declare an overall winner or normalize incomparable reported results into one score. `Unknown` means the public source did not verify the field. `Not evaluated` means the artifact is useful context but does not itself report an evaluation for that outcome; neither label means failure.
## Data policy
- Version 0.2 contains seven public papers already present in the public Mnemoverse Collection.
- Every artifact links to its Hugging Face paper page and any verified public code or project page.
- Evaluation claims follow the cited full paper version, not just its abstract. When HF bibliographic metadata lags that version, `metadata_source` pins the authoritative arXiv title and author list; otherwise the HF paper API is used.
- Code licenses are reported when the linked GitHub repository exposes a SPDX identifier through the GitHub API; dataset licenses are recorded separately when the linked dataset card declares them. A repository license is not assumed to govern the paper, benchmark data, or every dependency.
- Mnemoverse-authored work is visibly labeled.
- Only already-public evidence is included; no restricted research, concealed result, submission detail, or vendor ranking is included.
- Source checks are dated per artifact; the latest update is 2026-09-20, while MemOS and Harness the Memory retain their September 19 check dates. See [`SOURCES.md`](SOURCES.md) and [`data/artifacts.json`](data/artifacts.json). The manifest and recipe field `evidence_verified_at` denote the latest update, not a same-day recheck of every row; exported `artifact_verification_dates` preserve the individual dates.
The MIT license covers original files in this Space. Linked papers, repositories, datasets, and project pages remain under their own licenses and terms; links and factual summaries do not relicense upstream works.
## Papers represented
The current map covers the seven public papers below. Inclusion means that the Space maps evidence from the paper; it is not an endorsement or an overall ranking.
- [Semantic Level of Detail for Knowledge Graphs: Discovering Abstraction Boundaries via Spectral Heat Diffusion](https://huggingface.co/papers/2603.08965)
- [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://huggingface.co/papers/2410.10813)
- [Zep: A Temporal Knowledge Graph Architecture for Agent Memory](https://huggingface.co/papers/2501.13956)
- [AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agents](https://huggingface.co/papers/2407.04363)
- [MemOS: A Memory OS for AI System](https://huggingface.co/papers/2507.03724)
- [Graph-based Agent Memory: Taxonomy, Techniques, and Applications](https://huggingface.co/papers/2602.05665)
- [Harness the Memory: A Holistic Evaluation of Memory Substrates in Memory Agents](https://huggingface.co/papers/2608.15008)
## Local preview
Serve this directory over HTTP, for example:
```bash
python3 -m http.server 4173
```
Then open `http://127.0.0.1:4173/`.
## Preflight
Run the dependency-free release check before creating or updating the Space:
```bash
node scripts/verify-space.mjs
```
Add `--check-urls` when public-network checks are appropriate. The preflight validates the Static Space metadata, local application references, manifest invariants, evaluation-target coverage, HTTPS evidence links, schema identifiers, exact titles/authors against the declared metadata source, and GitHub SPDX identifiers for implementation repositories. It does not validate scientific claims or reproduce experiments. See [`DEPLOYMENT-CHECKLIST.md`](DEPLOYMENT-CHECKLIST.md) for functional release checks.
## Repository contents
- `index.html`: accessible application shell
- `app.js`: filters, comparison, deterministic recipe builder, downloads
- `styles.css`: responsive presentation
- `data/artifacts.json`: versioned public evidence manifest
- `data/schema.json`: JSON Schema for the manifest
- `SOURCES.md`: human-readable evidence trail
- `scripts/verify-space.mjs`: dependency-free release preflight
- `DEPLOYMENT-CHECKLIST.md`: reusable technical release checks
## Citation and contribution policy
Follow the primary links on each card before using a claim in a paper or system decision. To propose a correction, provide the artifact ID, field, primary URL, and the exact source passage. Fields remain `unknown` until a primary source verifies them.
|