File size: 755 Bytes
ed12f96 | 1 2 3 4 5 6 7 8 9 10 11 12 13 | # Multi-Task Evaluation Example
## Configuring `multi_task.yaml`
- `eval.defaults` defines inference parameters shared by every dataset entry. Override them inside an individual dataset block if needed.
- `eval.datasets` enumerates the datasets to evaluate. Each entry should specify:
- `name`: a short identifier that appears in logs and dashboards.
- `path`: the path to the dataset JSONL file.
- `rm_type`: which reward function to use for scoring.
- `n_samples_per_eval_prompt`: how many candidate completions to generate per prompt.
## IFBench Notes
- When `ifbench` is used, `slime/rollout/rm_hub/ifbench.py` will automatically prepares the scoring environment, so no additional manual setup is required beyond providing the dataset path.
|