Radha-9B / README.md
Advisio's picture
Upload README.md with huggingface_hub
7346720 verified
|
Raw History Blame Contribute Delete
4.11 kB
---
license: apache-2.0
base_model: Qwen/Qwen3.5-9B
base_model_relation: finetune
library_name: transformers
pipeline_tag: text-generation
language:
- en
tags:
- radha
- instruction-tuning
- lora
- structured-output
- gguf
- text-generation
---
# Radha-9B
**Radha** is an instruction-tuned assistant specialized for structured tasks,
grounded extraction, and format-faithful output. It is a fine-tune of
[`Qwen/Qwen3.5-9B`](https://huggingface.co/Qwen/Qwen3.5-9B) (Apache-2.0) and
keeps that base model's two operating modes: a **thinking** mode for reasoning
and a **non-thinking** mode for fast, direct answers.
> **Provenance (read this):** Radha is **fine-tuned from Qwen/Qwen3.5-9B**. It is
> a derivative work, distributed under Apache-2.0, with attribution in the
> `NOTICE` file. The base architecture is Qwen3.5; we did not train a model from
> scratch. We believe in honest provenance — this card is the authoritative
> source on Radha's lineage.
## What's different from the base
- **Sharpened structured-task behavior** across 10 task families: JSON
classification, grounded extraction, table aggregation, multi-step arithmetic,
summarization with action items, normalization/transformation, Python
bug-fixing, constraint ordering, format-adherence under adversarial prompts,
and grounded answerable/unanswerable QA.
- **Reasoning preserved.** The fine-tune masks the reasoning span from the
training loss, so Radha's thinking mode is not degraded — only its direct-answer
behavior is specialized.
- **Radha persona.** Identifies as "Radha." When asked about its internal
architecture it declines in-conversation; lineage is disclosed here, openly.
## Training
- **Method:** bf16 LoRA (rank 16, α 32) via [Unsloth](https://github.com/unslothai/unsloth).
- **Data:** a deterministic, stratified 300K-example subset (30K per task family)
of the `synthetic-distillation-v3` corpus, plus a small identity set. All
examples are short (max ≈ 293 tokens). The corpus is fully synthetic and
template-validated (schema + semantic e2e checks).
- **Hardware:** single NVIDIA A100 80GB (RunPod).
- **Modes:** trained with completion-only loss; reasoning span masked to preserve
thinking mode.
## Evaluation
Radha was checked against the training task families during development. Those
checks are in-distribution validations on synthetic, templated data — useful for
catching regressions, but **not independent benchmarks** and not published here
as capability claims. Evaluate Radha on your own data before relying on it.
## Usage
### Transformers (PyTorch)
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("Advisio/Radha-9B")
model = AutoModelForCausalLM.from_pretrained("Advisio/Radha-9B", torch_dtype="bfloat16", device_map="auto")
messages = [{"role": "user", "content": "Summarize: ..."}]
# Direct answers (fast): enable_thinking=False ; Reasoning: enable_thinking=True
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
out = model.generate(**tok(prompt, return_tensors="pt").to(model.device), max_new_tokens=512)
print(tok.decode(out[0], skip_special_tokens=True))
```
### GGUF (llama.cpp / Ollama, local)
Quantized builds are provided: `radha-9b-Q4_K_M.gguf` (recommended, ~5.6GB) and
`radha-9b-Q5_K_M.gguf`.
```bash
# llama.cpp
llama-cli -m radha-9b-Q4_K_M.gguf -p "Your prompt" -no-cnv
# Ollama
ollama create radha -f Modelfile # FROM ./radha-9b-Q4_K_M.gguf
ollama run radha
```
## Limitations
- Training data is **synthetic and template-driven**; Radha is strongest on the
task shapes it was tuned on and is not a substitute for domain-specific tuning.
- Inherits the base model's general knowledge, biases, and failure modes.
- Not safety-aligned beyond the base model's alignment; add your own guardrails
for production.
## License & attribution
Apache-2.0. Radha is a derivative of `Qwen/Qwen3.5-9B` (© Alibaba Cloud /
Qwen team, Apache-2.0). See `NOTICE`. You must retain attribution when
redistributing.