README.md

metadata

title: Agent Memory Evaluation Map
emoji: 🧭
colorFrom: indigo
colorTo: blue
sdk: static
app_file: index.html
license: mit
short_description: Evidence map for agent-memory evaluation without rankings.
tags:
  - agent-memory
  - evaluation
  - knowledge-graphs
  - mcp

Agent Memory Evaluation Map

Choose the memory capability you need to evaluate, compare public evidence, and export a reproducible evaluation checklist.

This Space is a research map, not a leaderboard. It does not declare an overall winner or normalize incomparable reported results into one score. Unknown means the public source did not verify the field. Not evaluated means the artifact is useful context but does not itself report an evaluation for that outcome; neither label means failure.

Data policy

• Version 0.2 contains seven public papers already present in the public Mnemoverse Collection.
• Every artifact links to its Hugging Face paper page and any verified public code or project page.
• Evaluation claims follow the cited full paper version, not just its abstract. When HF bibliographic metadata lags that version, metadata_source pins the authoritative arXiv title and author list; otherwise the HF paper API is used.
• Code licenses are reported when the linked GitHub repository exposes a SPDX identifier through the GitHub API; dataset licenses are recorded separately when the linked dataset card declares them. A repository license is not assumed to govern the paper, benchmark data, or every dependency.
• Mnemoverse-authored work is visibly labeled.
• Only already-public evidence is included; no restricted research, concealed result, submission detail, or vendor ranking is included.
• Source checks are dated per artifact; the latest update is 2026-09-20, while MemOS and Harness the Memory retain their September 19 check dates. See SOURCES.md and data/artifacts.json. The manifest and recipe field evidence_verified_at denote the latest update, not a same-day recheck of every row; exported artifact_verification_dates preserve the individual dates.

The MIT license covers original files in this Space. Linked papers, repositories, datasets, and project pages remain under their own licenses and terms; links and factual summaries do not relicense upstream works.

Papers represented

The current map covers the seven public papers below. Inclusion means that the Space maps evidence from the paper; it is not an endorsement or an overall ranking.

• Semantic Level of Detail for Knowledge Graphs: Discovering Abstraction Boundaries via Spectral Heat Diffusion
• LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
• Zep: A Temporal Knowledge Graph Architecture for Agent Memory
• AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agents
• MemOS: A Memory OS for AI System
• Graph-based Agent Memory: Taxonomy, Techniques, and Applications
• Harness the Memory: A Holistic Evaluation of Memory Substrates in Memory Agents

Local preview

Serve this directory over HTTP, for example:

python3 -m http.server 4173

Then open http://127.0.0.1:4173/.

Preflight

Run the dependency-free release check before creating or updating the Space:

node scripts/verify-space.mjs

Add --check-urls when public-network checks are appropriate. The preflight validates the Static Space metadata, local application references, manifest invariants, evaluation-target coverage, HTTPS evidence links, schema identifiers, exact titles/authors against the declared metadata source, and GitHub SPDX identifiers for implementation repositories. It does not validate scientific claims or reproduce experiments. See DEPLOYMENT-CHECKLIST.md for functional release checks.

Repository contents

• index.html: accessible application shell
• app.js: filters, comparison, deterministic recipe builder, downloads
• styles.css: responsive presentation
• data/artifacts.json: versioned public evidence manifest
• data/schema.json: JSON Schema for the manifest
• SOURCES.md: human-readable evidence trail
• scripts/verify-space.mjs: dependency-free release preflight
• DEPLOYMENT-CHECKLIST.md: reusable technical release checks

Citation and contribution policy

Follow the primary links on each card before using a claim in a paper or system decision. To propose a correction, provide the artifact ID, field, primary URL, and the exact source passage. Fields remain unknown until a primary source verifies them.