Hugging Face's logo

Akhrots
/
LAM8B

Text Generation
Transformers
Safetensors
English
lam
memory
long-term-memory
retrieval-augmented-generation
qlora
abstention
hallucination-reduction
locomo
longmemeval
conversational
custom_code
Eval Results (legacy)
Model card Files
xet
 Community
How to use from
Docker Model Runner
docker model run hf.co/Akhrots/LAM8B
Quick Links

LAM8B

LAM8B ๐Ÿ

the memory model that knows when it doesn't know.
Large Akhrots Model ยท by Tronocity Labs
most models would rather hallucinate than admit a gap. this one won't. ๐Ÿ’…

F1 0.606 +61% vs base -77% over-abstentions

8B params Apache 2.0 QLoRA r32 alpha64 1x T4 100% contract valid

โœจ the vibe

your assistant has a memory problem. it either forgets what you told it, or worse โ€” it makes something up and says it with its whole chest.

LAM8B does both halves of the job in one set of weights:

what it does
โœ๏ธ writes decides what's worth storing, and whether a new fact supersedes an old one
๐Ÿ” reads answers from what's stored โ€” or tells you it's not in there

that second half is the whole point. โฌ‡๏ธ

๐Ÿงฉ where it sits: LAM8B lives behind your retriever. your system pulls candidate memories, LAM8B reads them and answers or abstains. it is not a general chat model โ€” don't drop it in as one.

๐Ÿ“ˆ the glow-up

same prompts, same decoding, same harness. only the weights changed.

metric un-tuned base LAM8B
token F1 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 0.376 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘ 0.606 ๐ŸŸข +61%
exact match โ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 0.177 โ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 0.353 ๐ŸŸข 2.0ร—
abstention F1 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 0.541 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘ 0.788 ๐ŸŸข +0.247
abstention precision โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 0.378 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘ 0.714 ๐Ÿ”ฅ +0.336
abstention recall โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘ 0.946 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘ 0.878 ๐ŸŸก โˆ’0.068
wrongly refused 115 / 544 26 / 544 ๐Ÿ”ฅ โˆ’77%
contract-valid output 100% 100% โœ… tie
speed 2.1 s/ex 1.6 s/ex ๐ŸŸข โˆ’24%

n = 544 held-out answer-task examples.

๐ŸŽฏ the party trick: it shuts up when it should

here's the trap. the un-tuned base has 94.6% abstention recall โ€” it catches almost every unanswerable question. sounds elite. it is not.

precision is 37.8%. it refuses constantly. so of course it catches the real gaps โ€” it's refusing everything. 115 out of 544 questions it could have answered, it didn't. that's not "safe." that's a model that shrugs at you.

LAM8B trades 6.7 points of recall for 33.6 points of precision:

                  precision                     recall
base   0.378  โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘       0.946  โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘
LAM8B  0.714  โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘       0.878  โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘
                 โ–ฒ +0.336                       โ–ผ โˆ’0.068

wrongly-refused questions: 115 โ†’ 26. on the dedicated abstention split (n=74) both models hold 100% precision โ€” base scores 0.972 abstain-F1 there, LAM8B 0.935. so the base model's whole problem is on questions that had answers. ๐Ÿ’€

๐Ÿงพ receipts (per category)

sorted by how hard it glowed up. over = answerable questions it wrongly refused.

category n base LAM8B ฮ” F1 over
๐Ÿฅ‡ adversarial 76 โ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 0.183 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘ 0.673 +0.489 44 โ†’ 6
๐Ÿฅˆ temporal 93 โ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 0.303 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘ 0.586 +0.282 18 โ†’ 2
๐Ÿฅ‰ single_hop 140 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘ 0.545 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘ 0.734 +0.189 16 โ†’ 3
open_domain 13 โ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 0.326 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 0.468 +0.141 3 โ†’ 3
multi_hop 94 โ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 0.283 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 0.418 +0.135 21 โ†’ 3
knowledge_update 20 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 0.508 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘ 0.634 +0.125 2 โ†’ 0
single_hop_assistant 21 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘ 0.785 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 0.842 +0.056 0 โ†’ 0
๐Ÿ’€ preference 13 โ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 0.060 โ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 0.067 +0.007 11 โ†’ 9

adversarial is the flex. ๐Ÿฅ‡ same-domain distractors, where refusing takes actual judgement instead of vibes. base model basically gave up (0.183). LAM8B: 0.673, and 44 wrong refusals collapse to 6.

preference is the L. ๐Ÿ’€ it did not improve. see keeping it real.

๐Ÿ“Š exact-match numbers too (click)
category n EM (base) EM (LAM8B)
adversarial 76 0.1053 0.4211
temporal 93 0.0753 0.2473
single_hop 140 0.2643 0.4714
open_domain 13 0.2308 0.3846
multi_hop 94 0.0957 0.1809
knowledge_update 20 0.4000 0.4500
single_hop_assistant 21 0.5238 0.6667
preference 13 0.0000 0.0000

๐Ÿ”ฌ the honest split

source n F1 (base) F1 (LAM8B) EM abstain P abstain R abstain F1 over
LoCoMo 394 โ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 0.372 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘ 0.660 0.150 โ†’ 0.365 0.444 โ†’ 0.803 0.941 โ†’ 0.897 0.604 โ†’ 0.847 80 โ†’ 15
LongMemEval ๐Ÿ”’ 150 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 0.384 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 0.484 0.236 โ†’ 0.326 0.146 โ†’ 0.267 1.000 โ†’ 0.667 0.255 โ†’ 0.381 35 โ†’ 11

๐Ÿ”’ quote this one. those 150 LongMemEval questions were held in reserve and never trained on โ€” longmemeval_oracle and _s share the same 500 questions, so training on either contaminates the other. +0.10 F1 on genuinely unseen data is the real generalization number, not the +0.29 on LoCoMo.

also notice base's 1.000 abstention recall on LongMemEval with 0.146 precision. that's not skill. that's a model saying "idk" to everything. ๐Ÿซ 

โšก quickstart 

pip install transformers torch

from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL = "Akhrots/LAM8B"
tok = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForCausalLM.from_pretrained(
    MODEL,
    trust_remote_code=True,   # โ† required: LAM registers its own model class
    torch_dtype="float16",
    device_map="auto",
)

SYSTEM = (
    "You answer questions using only the user's stored memories, which are "
    "gathered from every app they use.\n"
    "Rules:\n"
    "- Use only the memories provided. Never invent or assume.\n"
    "- When memories conflict, trust the most recent and answer with that.\n"
    "- Combine memories when the answer needs more than one.\n"
    "- Respect timestamps for questions about when, before, after or latest.\n"
    "- Read negation carefully: a memory saying something stopped is not "
    "evidence that it is still true.\n"
    "- If the memories do not contain the answer, reply exactly: "
    "I don't have that in your memory.\n"
    "Answer concisely โ€” no preamble, no restating the question."
)

# memories render as:  [id] (timestamp via source) text
memories = (
    "[m1] (2024-03-02 via slack) Priya moved the launch review to Thursday.\n"
    "[m2] (2024-02-11 via gmail) Priya scheduled the launch review for Tuesday."
)
user = f"Memories:\n{memories}\n\nQuestion: When is the launch review?"

msgs = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": user}]
prompt = tok.apply_chat_template(
    msgs, tokenize=False, add_generation_prompt=True, enable_thinking=False
)
out = model.generate(**tok(prompt, return_tensors="pt").to(model.device), max_new_tokens=128)
print(tok.decode(out[0][len(tok(prompt).input_ids):], skip_special_tokens=True))
# -> "Thursday."   โ† m1 is newer, so m2 loses. that's the supersession behaviour.

serve it (weights are merged fp16, no adapter juggling):

vllm serve Akhrots/LAM8B --served-model-name lam8b --trust-remote-code

๐Ÿšจ three rules, don't skip

0. trust_remote_code=True is mandatory. LAM registers its own model class via modeling_lam.py. without the flag, loading fails outright.

1. the prompt is load-bearing. system prompt, [id] (time via source) memory format, and the exact abstention string I don't have that in your memory. are all part of the trained contract. drift the prompt โ†’ quality drops silently, no error.

2. thinking stays OFF. LAM8B is a hybrid-thinking architecture and was trained with enable_thinking=False. flip it on at serve time and you're measuring a different model than the one that was trained.

๐Ÿงฐ four tasks, one model

task mode output
answer read text, or the exact abstention string
write_op write JSON โ€” insert / supersede / merge / skip
extract write JSON โ€” entities + attributes
resolve read JSON โ€” {"kept": [...], "superseded": [...], "reason": str}

only answer is scored above โ€” see keeping it real for why the write-path numbers aren't published.

๐Ÿ”ง how it was cooked

training config
base open Apache-2.0 weights
method QLoRA, 4-bit NF4, fp16 compute
LoRA r / ฮฑ / dropout 32 / 64 / 0.05
target modules q,k,v,o,gate,up,down_proj (all of them)
LR 1e-4
batch ร— grad-accum 1 ร— 16 (effective 16)
epochs 1
hardware a single free Kaggle T4 ๐Ÿ˜ญ ~9 h/epoch
release format merged fp16 (vLLM / llama.cpp / TGI ready)
the corpus โ€” 12,783 train / 1,640 test
shard train source
write 7,921 synthetic write_op + extract
resolve 2,250 synthetic read-time conflict resolution
locomo 1,582 LoCoMo QA
abstain 720 synthetic hard abstention + answerable controls
longmemeval 315 LongMemEval oracle (MIT)

the one design choice that matters: every hard-abstention example ships with a twin โ€” identical topical clutter, but the queried fact is present. without those controls the model just learns a new shortcut ("in-domain clutter โ†’ refuse") instead of learning to actually check. the only signal separating the labels is whether the fact is in the set.

everything here is --visibility public โ€” synthetic + public benchmarks only. models memorise training data, so real user memories live behind a separate flag and a separate private adapter trained on top of these weights. no one's DMs are in these weights. ๐Ÿ”

how it was scored

deterministic, lexical, no LLM judge, no API key โ€” reproducible and free.

  • token F1 / EM after normalization (lowercase, strip articles + punctuation)
  • abstention P/R matched against the exact contract string, case- and trailing-period-insensitive
  • contract validity = fraction of outputs parsing against their task schema. both models hit 1.000 โ€” the base model's problem is content, not format
  • test file is shuffled before slicing (it's concatenated by source, so slicing raw would make every per-category number a lie)
python scripts/memory_sft/eval_gate.py \
    --model Akhrots/LAM8B \
    --data data/sft/mixed/test.jsonl \
    --out reports/ft.json --compare reports/base.json

๐Ÿซก keeping it real

no model card should be all W's. here are the L's, on the record:

the L
๐Ÿ’€ preference questions are cooked F1 0.067, EM 0.000, 9 of 13 still wrongly refused. no shard targets subjective recall and it shows. don't use this for "what does the user like."
๐Ÿชซ multi-hop is the weakest working category 0.418. up 48% from base, still way under single-hop's 0.734.
๐ŸŽฏ these scores are the reader, not the retriever every number here measures answering given retrieved memories. your end-to-end quality also depends on your retrieval stack โ€” a bad retriever will bottleneck this model no matter how good it is.
๐Ÿ” lexical metrics undersell it token F1 scores "Stripe" and "we use Stripe" as different answers. treat these as run-to-run comparisons, not absolute quality.
โœ๏ธ write-path numbers aren't published the write_op/extract/resolve eval shares generators with its training data, so it measures the generator as much as the model. validating against real production memories is the top open risk.
๐Ÿงฉ abstention doesn't fully transfer 0.381 abstain-F1 on LongMemEval vs 0.847 on LoCoMo. training teaches total absence; LongMemEval tests partial information ("you mentioned X but not Y"). different problem shape.
โณ one epoch a T4 caps one Kaggle session at ~1 epoch over this corpus. whether epochs 2โ€“3 leave gains on the table is untested.
๐ŸŒ english only both benchmarks and every synthetic generator are English.

๐Ÿ“œ license & credits

Apache-2.0. LAM8B is a fine-tune of open, Apache-2.0-licensed weights and the derivative is released under the same license. training data is synthetic + LoCoMo + LongMemEval (MIT).

@misc{lam8b,
  title  = {LAM8B: a joint read/write memory model with calibrated abstention},
  author = {Tronocity Labs},
  year   = {2026},
  url    = {https://huggingface.co/Akhrots/LAM8B}
}

one T4 ยท zero GPU budget ๐Ÿ

if it saved you from a hallucinated answer, leave a โค๏ธ on the repo.

Downloads last month
19
Safetensors
Model size
8B params
Tensor type
F16
ยท
Inference Providers NEW
Text Generation
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Dataset used to train Akhrots/LAM8B

Evaluation results