Add SVG model card and architecture preview; weights uploading

README.md

@@ -0,0 +1,110 @@
+---
+license: apache-2.0
+library_name: transformers
+pipeline_tag: text-generation
+base_model: summerMC/Looped-Qwen-pre
+language:
+- ja
+- en
+tags:
+- experimental
+- custom_code
+- graph-neural-network
+- graph-memory
+- looped-transformer
+- research
+---
+
+> **Upload in progress:** model weights are still transferring. Do not load until the final model commit appears.
+
+<div align="center">
+  <img src="./assets/header.svg" alt="LoopedQwen × GraphEngram" width="100%" />
+  <br />
+  <img src="./assets/tags.svg" alt="Experimental · JA + EN · 24 loops · 48,491 nodes · 13.63M memory parameters · custom code" width="100%" />
+  <br /><br />
+  <strong>Looped computation meets learned graph memory.</strong><br />
+  <sub>A self-contained bilingual research snapshot by summerMC.</sub>
+  <br /><br />
+  <a href="#architecture">Architecture</a> · <a href="#loading">Quickstart</a> · <a href="#evaluation">Evaluation</a> · <a href="#limitations">Limitations</a>
+</div>
+
+---
+
+> **Research prototype, NOT a production chatbot.** Conversation, arithmetic and code-generation tests remain poor, often producing unrelated numbers. Do not interpret lower language-model loss as general reasoning or instruction-following quality.
+
+This repository contains a consolidated snapshot of a repaired LoopedQwen backbone, learned graph-memory modules, and the matching token graph. No separate base-model or adapter download is needed when loading this package. The graph module cannot be algebraically folded into a dense weight matrix: “merged” means a self-contained custom architecture, not a LoRA-only merge.
+
+GraphEngram is an independent experimental graph-attention mechanism, not the official DeepSeek Engram implementation or a reproduction of its paper results.
+
+## Architecture
+
+<img src="./assets/architecture.svg" alt="Causal routing, learned graph attention and LoopedQwen fusion" width="100%" />
+
+- Pinned base: `summerMC/Looped-Qwen-pre`, revision `eaa834598f3e6b479d1f72b51b6933422b9b8a9c`.
+- Six shared GDN groups, four inner steps each: 24 effective steps, hidden size 2048.
+- Prior backbone adapters are consolidated into the exported backbone parameters.
+- Memory inserted before outer groups 0 and 3.
+- 48,491 shared token nodes and 88,076 directed adjacency edges.
+- Node width 128; 13,626,882 learned memory parameters.
+- Causal exact token bigram/trigram routing, up to 8 candidates, one-hop graph attention with degree cap 8.
+- Graph topology is fixed; node embeddings, attention projections and fusion gates are learned.
+- Graph construction uses training documents only. Old English node IDs and weights were preserved during Japanese expansion.
+
+## Loading
+
+Use a recent compatible Transformers environment. Export/evaluation used PyTorch 2.11.0 and Transformers 5.20.2. `flash-linear-attention` is optional acceleration; backend differences can affect numerical outputs.
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+repo = "summerMC/LoopedQwen-GraphEngram-ja-en"
+tok = AutoTokenizer.from_pretrained(repo)
+model = AutoModelForCausalLM.from_pretrained(
+    repo, trust_remote_code=True, dtype=torch.bfloat16, device_map="cuda"
+).eval()
+ids = tok.apply_chat_template(
+    [{"role": "user", "content": "こんにちは。"}],
+    add_generation_prompt=True, enable_thinking=False,
+    return_tensors="pt", return_dict=True
+)["input_ids"].to("cuda")
+out = model.generate(ids, max_new_tokens=32, do_sample=False, use_cache=False)
+print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))
+```
+
+Inspect custom code before enabling `trust_remote_code`. Generation is greedy, unpadded batch size 1 only. Cached decoding, concurrent forwards and production serving integrations are unsupported. Each output token recomputes the prefix. The graph checksum is checked during loading. Keep `graph.json` with the weights and configuration.
+
+Toggle memory for ablations: `model.network.enabled = False`.
+
+## Training provenance
+
+- English source: `HuggingFaceFW/fineweb`, `sample-10BT`, revision `9bb295ddab0e05d785b879661af7260fed5140fc`.
+- Japanese source: `HuggingFaceFW/fineweb-2`, `jpn_Jpan`, revision `af9c13333eb981300149d5ca60a8e9d659b276b9`.
+- Dataset cards identify ODC-BY. Check dataset/source-content terms for downstream redistribution and use. The token graph itself retains training-text fragments; it is not a privacy-preserving representation.
+- Whole-document hashing separates train/dev/test before truncation. No near-duplicate or base-pretraining contamination audit was performed.
+- Base was frozen during graph training. The immediate bilingual continuation trained 36,864 Japanese + 36,864 English tokens, not the entire downloaded corpus.
+- Earlier graph stages and the repaired base are described in the included reports/design document.
+- Parent optimizer was not retained for the first bilingual continuation; weights were resumed with a fresh optimizer.
+
+## Evaluation
+
+Standalone reloaded bilingual snapshot, eight held-out documents per language, full documents evaluated in independent chunks:
+
+| Corpus | Target tokens | GNN off NLL | GNN on NLL |
+|---|---:|---:|---:|
+| Japanese | 4,895 | 4.5203 | 4.4736 |
+| English | 4,912 | 4.0961 | 4.0122 |
+
+Eight additional development prompts: math 0/2, required code functions 0/2, extraction 1/2 for both on/off. Conversation remains inappropriate/repetitive. Generated code was not executed. These are small diagnostics, not standardized benchmark or Opus-equivalence results.
+
+## Limitations
+
+- Weak instruction following and numerical-answer bias in the repaired backbone.
+- No established improvement in Japanese/English conversation, math or code generation.
+- No demonstrated learning-speed advantage over official Engram, RAG or ordinary fine-tuning at matched quality.
+- No proven compression advantage or lossless reconstruction of the complete source corpus from the model graph.
+- No billion/trillion-token context guarantee; lexical routing has limited recall and CPU overhead.
+- Frozen random-initialized new node vectors require learning; storing a text node is not equivalent to learning its facts.
+- Source text may contain errors, personal information or malicious instructions. No dedicated filtering/safety training was performed for this prototype.
+
+The model is published for reproducibility and architecture experiments, not capability claims.