KAUSTAcademy/RecipeMatching_RecipeArm_Qwen3-Embedding-4B

RecipeMatching recipe arm, 4B (Qwen3-Embedding-4B)

A checkpoint from the paper Recipe-Matching, Not Equivalence (Ali Habibullah, Mohammad Alshiekh, Yazan Alshoibi, Salman Khan and Naeemullah Khan, 2026), shared so the paper's evaluations of this model can be re-run without retraining. It is not one of the three models the paper releases as its main artefacts. Code, data and results: https://github.com/KAUST-Academy/recipe-matching-not-equivalence.

What it is. The recipe arm of the paper's 4B replication: a rank-16 LoRA adapter over Qwen3-Embedding-4B, trained on the 6,145 rows of data/llm_pairs/pairs.jsonl, the 0.6B recipe arm's training file. Its twin, the 4B verified arm, was trained with the same settings and is in the GitHub repository under models/ctrl-cas-6145-4b/final/. Seed 42; the full training record, command line included, is run_config.json in this repository, and every training file named here is in the GitHub repository.

Results of this checkpoint (seed 42, R@1 / R@5 / R@10; a single run, the one the paper reports; the trained 4B arms were not scored on the medium tier):

Evaluation	R@1	R@5	R@10
MathNet-Retrieve easy tier (15,000 queries, 117,088 documents)	77.77	98.41	99.33
MathNet-Retrieve hard tier	6.03	74.29	86.79
Cross-language duplicates (strict)	70.99	86.77	89.06

Usage. An adapter-only repository: with peft installed, SentenceTransformer loads the adapter on top of Qwen/Qwen3-Embedding-4B. Encode queries with the query prompt, the setting every number above uses.

# pip install sentence-transformers peft
import torch
from sentence_transformers import SentenceTransformer

# Adapter-only repo: with peft installed, SentenceTransformer loads the LoRA
# adapter on top of its base model, Qwen/Qwen3-Embedding-4B.
model = SentenceTransformer("KAUSTAcademy/RecipeMatching_RecipeArm_Qwen3-Embedding-4B", model_kwargs={"torch_dtype": torch.bfloat16})
model.max_seq_length = 1024  # the evaluation's input cap

queries = ["Find all real numbers x such that x^2 - 5x + 6 = 0."]
documents = ["Determine every real x satisfying x^2 - 5x + 6 = 0."]

# Queries use the stored "query" prompt, as in the paper's evaluation; documents use no prompt.
q = model.encode(queries, prompt_name="query", normalize_embeddings=True)
d = model.encode(documents, normalize_embeddings=True)
print(model.similarity(q, d))

Citation

@misc{habibullah2026recipematchingequivalence,
      title={Recipe-Matching, Not Equivalence}, 
      author={Ali Habibullah and Mohammad Alshiekh and Yazan Alshoibi and Salman Khan and Naeemullah Khan},
      year={2026},
      eprint={2609.31927},
      archivePrefix={arXiv},
      primaryClass={cs.IR},
      url={https://arxiv.org/abs/2609.31927}, 
}