data/README.md

# Delta Seed Dataset

This directory contains the small seed dataset used to test and bootstrap Delta Ultra Mini.

- `delta_seed.jsonl`: conversational JSONL records with a `text` field.
- `tokenizer_corpus.txt`: plain text corpus for BPE tokenizer training.
- `corpus.txt`: training text in the same chat-token format.

This dataset is intentionally small. It is useful for validating the tokenizer, dataset loader, training loop, checkpoint saving, and local inference. It is not enough to create a strong general assistant.

## License

MIT, same as the project.

## Format

```jsonl
{"text":"[SYS] You are Delta. [SEP]\n[USR] Question [SEP]\n[ASS] Answer [SEP]"}
```

## Train Tokenizer

```powershell
python scripts/train_tokenizer.py --corpus_files data/tokenizer_corpus.txt --output_path tokenizer.json
```

## Train Model

```powershell
python scripts/train_delta.py --data_path data --output_dir runs/delta-ultra-mini --epochs 1 --batch_size 2 --tokenizer_path tokenizer.json
```