File size: 3,071 Bytes
aa130ea 18c007d aa130ea a2e6249 aa130ea | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 | ---
license: other
license_name: ameforge-proprietary
license_link: LICENSE
language:
- en
pipeline_tag: question-answering
datasets:
- rajpurkar/squad
tags:
- continual-learning
- memory
- retrieval
- research-prototype
---
# Ananke v1
**A memory that learns by reading, with a learned reader.** Research prototype by **AMEFORGE**
(Ontario, Canada).
👉 **Try it:** [Ananke demo Space](https://huggingface.co/spaces/ameforge/ananke)
## What changed since v0
[Ananke v0](https://huggingface.co/AMFORGE/ananke-v0) found the right passages in its memory well,
but picked the short answer inside them with hand-written rules — its main weakness.
In v1:
- **The memory is unchanged.** It still reads text once, keeps learning, cites its sources and
refuses on subjects it never read.
- **The short answer is chosen by a small learned reader**, a proprietary AMEFORGE architecture
(not a pretrained language model). It looks only at the passages the memory retrieved.
- The reader was trained once, on a GPU, on SQuAD v1.1 questions. It runs on a CPU.
Articles used for the demo memory were excluded from its training data.
## Evaluation (honest numbers)
All question sets were written and frozen **before** each evaluation and never used for training.
The final set (41 questions) was evaluated a single time.
| | Ananke v0 (rules) | Ananke v1 (learned reader) |
|---|---|---|
| Final frozen set (41 questions) | 14 | **16** |
| All frozen sets so far (323 questions) | 109 (34 %) | **119 (37 %)** |
| Refusal on never-read subjects (3) | 3 / 3 | 3 / 3 |
What this means:
- v1 is **slightly better** than v0, but the difference is **not statistically significant yet**
(sign test, p ≈ 0.34). Each system answers some questions the other misses.
- The exact short answer is still wrong about **two times out of three**: the quoted passages
matter more than the headline answer.
- When the memory retrieves the wrong passage, the reader cannot recover.
## Limitations
- **Not a chatbot or an LLM.** It does not reason, combine facts or write free text.
- It **believes what it reads**: wrong text in, wrong answer out.
- English works best. Lists ("which countries border…") are often incomplete.
- If a word of the question is known for an unrelated reason, it may answer from an unrelated
passage instead of refusing. **Always read the quotes.**
## Intended use
Research and demonstration of learning by reading. A self-updating, citable memory over a document
collection. Not for medical, legal, financial or safety-critical decisions.
## Weights and code
The engine, the reader and the memory are proprietary to AMEFORGE and are not distributed;
the Space provides public access for testing.
## Data and attribution
- Reader training: [SQuAD v1.1](https://rajpurkar.github.io/SQuAD-explorer/) (Rajpurkar et al., 2016),
CC BY-SA 4.0.
- Demo memory: English Wikipedia articles, CC BY-SA 4.0. Quoted passages are shown with their source.
## Contact
AMEFORGE — Mike Amega · [huggingface.co/AMFORGE](https://huggingface.co/AMFORGE)
|