README.md

---
license: cc-by-nc-4.0
base_model: vinmlops/cpt-v2
tags:
  - sft
  - supervised-fine-tuning
  - instruction-tuning
  - llm
  - transformers
  - educational
language:
  - en
library_name: transformers
pipeline_tag: text-generation
---

# sft-v2 — Instruction-Tuned ML/LLM Assistant (SFT on cpt-v2)

> **⚠️ Educational project.** This model is a personal learning experiment,
> released **for learning and educational purposes only**. See
> [License & Intended Use](#license--intended-use).

`sft-v2` is a **supervised fine-tuned (SFT)** model built on
[vinmlops/cpt-v2](https://huggingface.co/vinmlops/cpt-v2) (a domain-adapted
Qwen3-1.7B). It turns the domain-adapted base into a **chat-style assistant**
that answers questions about large language models, neural networks, and
transformers — and handles general questions too.

This is **stage 2 of 3** in a small end-to-end pipeline:

```
Qwen3-1.7B-Base  ──CPT──▶  cpt-v2  ──SFT──▶  sft-v2  ──DPO──▶  dpo-v1
                                            (this repo)
```

---

## What it is

- **Base:** [vinmlops/cpt-v2](https://huggingface.co/vinmlops/cpt-v2)
  (Qwen3-1.7B-Base + continued pretraining)
- **Method:** Supervised fine-tuning with LoRA, then **merged into a standalone
  model** (bfloat16). Assistant-only loss so the model learns to answer and stop.
- **Format:** A **full standalone model** — load it directly with
  `AutoModelForCausalLM.from_pretrained`, no adapter/PEFT needed.
- **Role:** Instruction-following ML/LLM assistant; the base for the DPO
  preference-tuning stage.

## Recommended system prompt

The model was tuned with this system prompt for domain questions:

> *You are a helpful assistant with expertise in machine learning and large
> language models. Answer the user's question accurately and directly. Explain
> concepts clearly and acknowledge uncertainty when you are unsure.*

General questions were trained **without** a system prompt, so the model also
answers everyday questions naturally when no system prompt is provided.

## How to use

Because this is a merged standalone model, load it directly:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained("vinmlops/sft-v2")
model = AutoModelForCausalLM.from_pretrained("vinmlops/sft-v2", dtype=torch.bfloat16)
model = model.to("cuda" if torch.cuda.is_available() else "cpu").eval()

messages = [
    {"role": "system", "content": "You are a helpful assistant with expertise in machine learning and large language models. Answer the user's question accurately and directly. Explain concepts clearly and acknowledge uncertainty when you are unsure."},
    {"role": "user", "content": "What is attention in a transformer?"},
]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

*(This is a gated repository — you must be granted access and be authenticated
with `hf auth login` to download it.)*

## Training summary

| Item | Value |
|---|---|
| Base | vinmlops/cpt-v2 (Qwen3-1.7B-Base + CPT) |
| Method | Supervised fine-tuning (LoRA), merged to standalone |
| Precision | bfloat16 |
| Loss | Assistant-only (the model learns to answer and stop) |
| Task | Instruction-following Q&A on ML/NN/transformer topics, plus general questions |

The training data was supervised instruction/response pairs built from publicly
available material for **personal, educational training use only**, and is
**not** included or redistributed here. This repository contains only the
resulting model weights.

## Behavior

- Answers instruction-style questions directly and **stops cleanly** (learns to
  emit the end-of-turn token rather than rambling).
- Has a consistent assistant identity and declines clearly harmful requests.
- Retains general instruction-following ability alongside its ML/LLM focus.

## Intended use

- Asking beginner-to-intermediate questions about LLMs, neural networks, and
  transformers.
- Educational demos of how SFT turns a base model into an instruction follower.
- The base for the DPO preference-tuning stage.

## Limitations

- **Small-scale, experimental.** A 1.7B model with light domain adaptation and a
  modest SFT set — helpful for learning, not an authoritative expert.
- **May produce inaccurate, incomplete, or outdated information.** Verify
  anything important; do not use for production or critical decisions.
- Strongest on ML/NN/transformer topics; general-domain performance is more
  limited.
- Inherits biases and limitations of the base model and training data.

## License & Intended Use

**This model is released for learning and educational purposes only.**

- Licensed under **CC-BY-NC-4.0** (Creative Commons Attribution–NonCommercial 4.0).
- You may use, study, and share it for **non-commercial, educational, and
  research purposes**, with attribution.
- **Commercial use is not permitted.**
- Provided **"as is", without warranty** of any kind; it may produce inaccurate
  output. Use at your own risk.
- **Lineage:** [Qwen/Qwen3-1.7B-Base](https://huggingface.co/Qwen/Qwen3-1.7B-Base)
  (Apache-2.0) → [vinmlops/cpt-v2](https://huggingface.co/vinmlops/cpt-v2) →
  this SFT model. Please also respect the base model's license.

## Citation / attribution

If you reference this educational project:

```
vinmlops/sft-v2 — instruction-tuned ML/LLM assistant (SFT on cpt-v2, merged),
an educational LLM/NN/transformer learning project. Non-commercial (CC-BY-NC-4.0).
Lineage: Qwen/Qwen3-1.7B-Base (Apache-2.0) -> cpt-v2 -> sft-v2.
```