Qwen3.8-27B-V13-b1

DavidAU/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored converted to the qwen_dense_v13 architecture (Qwen3.5 dense with episodic KV memory and latent foresight). The weights are the base model's, unchanged; the architecture code ships with the model, so load it with trust_remote_code=True.

Architecture

• Backbone: the base model's Qwen3.5 dense 27B: 64 layers (48 Gated DeltaNet linear-attention layers and 16 full-attention layers) and its vision tower.
• Episodic KV memory: calls write their keys and values into a persistent store and later calls recall them at the full-attention layers. The store lives under $QWEN_DENSE_V13_MEMORY_HOME and is named after the weights, so the same weights find the same store. Supported operations include read-only queries (model.reopen_memory(write_policy="none")), model.memory_stats() and forgetting an episode (model.open_memory_bank().forget_episode(...)).
• Latent foresight (world model): one predictor per full-attention layer (16 predictors, about 0.1B parameters). They are not stored in the checkpoint. They start in their closed form at load time, so the load report lists world_model.* as newly initialized; this is expected. The runtime fits them by itself once a store holds 256 transitions, and again whenever that number has doubled. It keeps a fit only if it predicts the held-out transitions better than the closed form, and stores the fit with the memory, where every instance that opens the store picks it up. model.fit_foresight() triggers a fit on demand.
• In training mode, or with memory_enabled=False, the forward pass is the plain Qwen3.5 forward.
• MTP head: the base model's multi-token-prediction head is included for serving engines that use it for speculative decoding. Transformers ignores it.
• Memory settings: chunk_size 4096 and memory_tokens 32768 (qwen_dense_v13_memory_config in config.json). Both are eight times the V13 defaults of 512 and 4096: selection works over 4096-token spans of a prompt, and up to 32768 recalled tokens per layer and KV head are attended.
• Chat template: chat_template.jinja keeps thinking always on and keeps reasoning in every turn. reasoning_effort takes low, medium or xhigh, and defaults to medium.

Evaluation

Measured with the Transformers code and the V13 default memory settings (chunk_size 512, memory_tokens 4096), before this release changed them.

Test	Result
Memory recall, 40 facts written once each, read-only queries: after write	0 / 40
… from a new model instance opening the same store	0 / 40
… after forgetting half of the episodes: forgotten / kept	0 / 20 · 0 / 20
Generation, 6 prompts on one shared store, greedy, 384 tokens: ended / no stray <think> / no repetition loop	6 / 6 · 6 / 6 · 6 / 6 (the tool call is parsed)
Held-out NLL (60 conversations, 33,029 tokens, memory off)	0.545

In the recall test, each fact is a separate call (Registry entry <name>: the vault code is <code>.); the query is the same sentence without the code, answered greedily in 10 tokens. After the writes, the answers take the shape of the stored codes (7X9-PLM2 instead of the 100000000 produced with an empty store), so the store is read, but the written values are not returned.

Usage

import os
import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer

os.environ["QWEN_DENSE_V13_MEMORY_HOME"] = "/path/on/a/local/disk/v13-memory"
repo = "LTM-Lab/Qwen3.8-27B-V13-b1"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(
    repo, trust_remote_code=True, dtype=torch.bfloat16, device_map="cuda"
).eval()

messages = [{"role": "user", "content": "Summarize the trade-offs of event sourcing in three bullet points."}]
ids = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", reasoning_effort="medium"
).to(model.device)
out = model.generate(ids, max_new_tokens=2048, do_sample=True, temperature=1.0, top_p=0.95, top_k=20)
print(tokenizer.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

• Requirements: Transformers 5.x and flash-linear-attention for the Gated DeltaNet kernels. causal-conv1d is optional; without it a slower PyTorch fallback is used.
• Keep the memory store on a local disk. The store uses SQLite, which can stall on network or virtual filesystem mounts.
• Inline control tags in system or user messages (<|think_low|>, <|think_medium|>, <|think_max|>) also set the reasoning effort; high, max and extreme map to xhigh.

Serving with SGLang

Stock SGLang does not know the Qwen3_5V13ForConditionalGeneration architecture. The image lovewin10/sglang-v13:nightly-dev-cu13-20260924-ffac53d7-v13 is lmsysorg/sglang:nightly-dev-cu13-20260924-ffac53d7 patched to run this architecture with its episodic memory and world model. It drives the model's own V13 code, and keeps CUDA graphs, the prefix cache, EAGLE speculative decoding (MTP draft) and the overlap scheduler on. Mount a writable folder at /memory for the store.

docker run --rm -it --gpus all --ipc=host -p 8000:8000 `
  -v "<folder containing Qwen3.8-27B-V13-b1>:/model:ro" `
  -v "<folder for the memory store>:/memory" `
  lovewin10/sglang-v13:nightly-dev-cu13-20260924-ffac53d7-v13 `
  sglang serve --model-path /model/Qwen3.8-27B-V13-b1 --trust-remote-code `
  --host 0.0.0.0 --port 8000 --mem-fraction-static 0.78 --served-model-name EVE-27B `
  --chat-template /model/Qwen3.8-27B-V13-b1/chat_template.jinja `
  --reasoning-parser qwen3-thinking --tool-call-parser qwen3_coder `
  --attention-backend flashinfer --linear-attn-backend triton --mamba-ssm-dtype float32 `
  --speculative-algorithm EAGLE --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 `
  --enable-linear-replayssm-spec --mamba-radix-cache-strategy extra_buffer_lazy `
  --context-length 262144 --max-prefill-tokens 32768 --max-running-requests 8 `
  --kv-cache-dtype fp8_e4m3 --max-mamba-cache-size 16

Tested on one 96 GB GPU with this release's memory settings:

• Memory across requests. A locker code given in one chat is answered correctly in a new chat. This held after server restarts; the answer's reasoning cites the time header of the stored memory.
• Near-identical facts. These can be confused. After Registry entry Morivane: the vault code is 5QK-82TR. was written, the query for Morivane returned the code stored earlier for another entry (17D-27JI).
• Parsing. Reasoning and tool calls come back parsed (reasoning_content, tool_calls).
• Speed, single request.
  • Memory on: about 39 tokens/s (1,602 tokens in 41.5 s).
  • QWEN_DENSE_V13_MEMORY=0: about 52 tokens/s, with EAGLE acceptance length 2.1–2.7.
• GPU memory. The memory takes 10.1 GiB of step buffers for 4 concurrent requests, plus a 4 GiB decoded-memory cache, from the GPU memory left after --mem-fraction-static 0.78.

Limitations

• With many near-identical facts in the store (for example dozens of "entry X: code Y" lines), recall can return the value of a similar entry.
• The base model is an uncensored merge, and this model inherits its behavior.
• Not evaluated on standard benchmarks.