vetbot-v3 / README.md
lini1990's picture
Upload folder using huggingface_hub
133e7cb verified
|
Raw History Blame Contribute Delete
692 Bytes
# VetBot V3 - Qwen2.5-72B LoRA Adapter
Fine-tuned Qwen2.5-72B-Instruct for veterinary medicine Q&A.
## Training Summary
| Parameter | Value |
|-----------|-------|
| Base Model | Qwen/Qwen2.5-72B-Instruct |
| Method | LoRA (r=128, alpha=256) |
| Trainable Params | 1.68B / 74.4B (2.26%) |
| Hardware | 4x NVIDIA H200 (141GB each) |
| Training Time | 8.46 hours |
| Epochs | 3 |
| Train Samples | 41,506 |
| Final Loss | 0.572 |
| Token Accuracy | ~94% |
## Files
- `fsdp_checkpoint/` - FSDP2 distributed checkpoint
- `adapter_config.json` - LoRA configuration
- `tokenizer*` - Qwen2.5-72B tokenizer
## Note
This is an FSDP2 checkpoint format. Load with HuggingFace accelerate library.