Spaces:

Hugdrigo
/

evalflow-ci-lab

Running

App Files Files Community

evalflow-ci-lab / USER_GUIDE.md

Hugdrigo

Upload 44 files

2e85669 verified about 1 month ago

preview code

Raw History Blame Contribute Delete

1.33 kB

# How to use EvalFlow CI Lab

**Quick start:** Open App. Choose `Multinomial Naive Bayes` as baseline, `Hybrid TF-IDF` as candidate, and `Harder challenge · 54` as split. Click `Run release audit`. The candidate should show 77.8% Accuracy and the default research policy should return PASS.

- **Command Center:** see exact metrics and gate results.
- **Pipeline Replay:** review seven stages and the executable CLI command.
- **Paired Regressions:** filter repaired examples, newly incorrect examples and changed predictions. Search by example ID or source text.
- **Release Gates:** try changing minimum Accuracy from 0.75 to 0.95; it should BLOCK. Use reset to return to the documented default.
- **Slice Diagnostics:** use language/label groupings. The challenge split contains only one `mixed` domain, shown as a warning. For domain slices, use the controlled IID split.
- **Robustness Checks:** inspect typo/context flip rates and example coverage. Truncation is destructive and not scored.
- **Run Registry:** downloads are essential because browser sessions are not persisted.
- **CI Integration:** instructions to activate the optional workflow on GitHub.
- **Export & Provenance:** download CSV, JSON and Markdown. Custom evaluation JSON stays in the browser and is marked unverified. Restore original dataset anytime.

	# How to use EvalFlow CI Lab

	Quick start: Open App. Choose `Multinomial Naive Bayes` as baseline, `Hybrid TF-IDF` as candidate, and `Harder challenge · 54` as split. Click `Run release audit`. The candidate should show 77.8% Accuracy and the default research policy should return PASS.

	- Command Center: see exact metrics and gate results.
	- Pipeline Replay: review seven stages and the executable CLI command.
	- Paired Regressions: filter repaired examples, newly incorrect examples and changed predictions. Search by example ID or source text.
	- Release Gates: try changing minimum Accuracy from 0.75 to 0.95; it should BLOCK. Use reset to return to the documented default.
	- Slice Diagnostics: use language/label groupings. The challenge split contains only one `mixed` domain, shown as a warning. For domain slices, use the controlled IID split.
	- Robustness Checks: inspect typo/context flip rates and example coverage. Truncation is destructive and not scored.
	- Run Registry: downloads are essential because browser sessions are not persisted.
	- CI Integration: instructions to activate the optional workflow on GitHub.
	- Export & Provenance: download CSV, JSON and Markdown. Custom evaluation JSON stays in the browser and is marked unverified. Restore original dataset anytime.