Etsu-Embedding-0.8B-Memory

Name: Etsu — from the Japanese reading of 会通 (etsū), the Buddhist term for the convergence of disparate teachings. From Zheng Qiao's (郑樵) Tongzhi (《通志·总序》):

百川异趋，必会于海，然后九州无浸淫之患；万国殊途，必通诸夏，然后八荒无壅滞之忧。会通之义大矣哉！

A hundred rivers run different courses, yet all must reach the sea; ten thousand states take different roads, yet all converge upon the Central States — great indeed is the meaning of huitong (convergence). An embedding model does the same: it maps heterogeneous inputs into one shared vector space.

An embedding model fine-tuned from Qwen/Qwen3.5-0.8B-Base for agent memory retrieval and related domain-specific semantic retrieval.

This is the in-domain specialist sibling of Rebine/Etsu-Embedding-0.8B. Same base model, same architecture, same encoding protocol; different training data and recipe. The two are separate, coexisting releases — this one is trained on a re-built memory-retrieval dataset and is stronger on the target domain, while the earlier release generalizes better out of domain.

Model Overview

• Model Type: Text Embedding
• Supported Languages: Chinese, English
• Number of Parameters: 752M (752,393,024)
• Context Length: 2048 tokens
• Embedding Dimension: 1024 (supports MRL dimensions 128, 256, 512, 768, 1024)

Model Details

Property	Value
Base model	Qwen/Qwen3.5-0.8B-Base
Architecture	Qwen3.5 hybrid text model
Layers	24 (18 linear-attention + 6 full-attention)
Hidden size	1024
Intermediate size	3584
Vocabulary size	248,320
Embedding output	1024 dimensions
Pooling	Last-token pooling
Normalization	L2 normalization (after Matryoshka truncation)
Similarity	Cosine similarity or normalized dot product
Training precision	BF16
MRL dimensions	128, 256, 512, 768, 1024
License	Apache-2.0

Intended Use

Chinese/English semantic retrieval for agent memory, personal knowledge bases, conversation archives, technical documentation, and related domain-specific search, including short keyword queries and natural questions.

Encoding Protocol

Queries:

Instruct: Given a query, retrieve relevant passages that answer the query
Query: {query}

Memories/passages: no instruction prefix.

Use last-token pooling (the last non-pad position of attention_mask) and L2 normalization. If using Matryoshka truncation, truncate to 128, 256, 512, or 768 dimensions before the final L2 normalization.

Use right padding: under left padding the "last non-pad token" is not the true end of the text.

This is not a sentence-transformers package with a built-in pooling wrapper; implement the pooling and query prefix as described above.

import torch
import torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer

model_id = "Rebine/Etsu-Embedding-0.8B-Memory"
INSTRUCTION = "Given a query, retrieve relevant passages that answer the query"

tok = AutoTokenizer.from_pretrained(model_id)
tok.padding_side = "right"
model = AutoModel.from_pretrained(model_id, dtype=torch.bfloat16, device_map="cuda").eval()

@torch.no_grad()
def encode(texts, is_query=False):
    if is_query:
        texts = [f"Instruct: {INSTRUCTION}\nQuery: {t}" for t in texts]
    enc = tok(texts, padding=True, truncation=True, max_length=2048,
              return_tensors="pt").to(model.device)
    hidden = model(**enc, use_cache=False).last_hidden_state
    idx = enc["attention_mask"].sum(dim=1) - 1
    vec = hidden[torch.arange(hidden.size(0)), idx][:, :1024]
    return F.normalize(vec.float(), dim=-1)

Evaluation

Full-corpus retrieval, dense cosine, K = 1/3/6/10 (K = 6 matches OpenClaw's maxResults = 6). Values are Recall@6 unless stated otherwise. Every row uses the same tokenizer protocol, query instruction, pooling, normalization, corpus and query set.

vs. Qwen/Qwen3-Embedding-0.6B

Set	Domain	Cases	Memory R@6	Qwen3-Embedding-0.6B R@6
valid	in-domain (synthetic queries)	913	83.46%	70.43%
golden	in-domain (real queries)	100	78.00%	69.00%
locomo	out-of-domain, English	1,977	67.53%	74.41%
locomo_zh	out-of-domain, Chinese	1,977	62.32%	72.18%

vs. Rebine/Etsu-Embedding-0.8B (previous recipe)

Set	Memory R@6	Rebine R@6
valid	83.46%	80.28%
golden (real queries)	78.00%	74.00%
locomo	67.53%	75.77%
locomo_zh	62.32%	72.53%

Matryoshka dimensions (Recall@6)

Dim	valid	golden
128	66.16%	65.00%
256	75.03%	70.00%
512	80.94%	71.00%
768	83.35%	77.00%
1024	83.46%	78.00%

Headline metrics at 1024 dims

Set	R@1	R@3	R@6	R@10	MRR@6
valid	58.49%	75.36%	83.46%	86.86%	67.59%
golden	49.00%	69.00%	78.00%	82.00%	59.80%

Known Limitations

• In-domain specialist. On out-of-domain benchmarks (LoCoMo en/zh) it scores below both the previous 0.8B release and the general-purpose 0.6B baseline. For cross-domain/general retrieval use Rebine/Etsu-Embedding-0.8B.
• No refusal behaviour. Queries with no answer in the corpus still return results; "no answer" handling must be implemented by a score threshold upstream.
• Retrieval quality is coupled to chunking: training passages were cut at ~800 tokens. Changing the chunk size degrades results.
• Inputs longer than 2048 tokens are truncated on the right; with last-token pooling a truncated input does not pool over its true ending.

Quantizations

GGUF versions: Rebine/Etsu-Embedding-0.8B-Memory-GGUF — F16, Q8_0, and Q4_K_M (recommended for deployment).

Base Model and License

Derived from Qwen/Qwen3.5-0.8B-Base and distributed under the base model's Apache-2.0 license. Please review the base model license and usage requirements.