Yida-Model-Qwen

Merged full-weight checkpoint of a Qwen3-14B LoRA supervised fine-tune. Hub files are merged bfloat16 weights, not a PEFT adapter.

This model is for research and engineering evaluation. It is not a medical device and must not be used as the sole basis for clinical decisions.

Model details

Item Value
Developed by rwang220
Model type Causal LM (Qwen3ForCausalLM)
Parameters 14.77B (14,768,307,200)
Precision bfloat16
Architecture 40 layers, hidden size 5120, GQA 40/8, intermediate size 17408, vocab 151936
Context max_position_embeddings 40,960. SFT max length 8,192
License Apache 2.0, inherited from Qwen3-14B

Training

Fine-tuned with ms-swift LoRA, then merged into full weights.

Item Value
Immediate base An internal continued Qwen3-14B checkpoint (Qwen3-14B_30G), same architecture as Qwen/Qwen3-14B. This is not a direct fine-tune of the public Qwen3-14B weights.
Run qwen_14b_7.31/v0-20260731-064328
LoRA rank 16, alpha 32, dropout 0.05, targets q/k/v/o/gate/up/down_proj
Optim AdamW, lr 1e-4, cosine, warmup ratio 0.05
Schedule 4 epochs, 97,232 steps, max length 8192
Batch per-device 1, gradient accumulation 8
Best eval loss 0.5906 at step 96,000
Final eval loss 0.5907
Final eval token acc 0.8067

Data

An internal Chinese medical instruction mix. Most categories were capped at 10,000 samples; MedEthics used a higher cap.

How to use

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "rwang220/Yida-Model-Qwen"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
messages = [{"role": "user", "content": "请用通俗语言解释高血压的常见注意事项。"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=1024, temperature=0.6, top_p=0.95, top_k=20)
print(tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
vllm serve rwang220/Yida-Model-Qwen --reasoning-parser qwen3

Limitations

  • Can hallucinate guidelines, doses, diagnoses, and citations.
  • Training data is internal and not released with this repository.
  • Reported numbers are training eval loss and token accuracy only. No clinical benchmark is claimed.
Downloads last month
317
Safetensors
Model size
15B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support