Text Classification
syfox
decision-engine
decision-making
decision-model
structured-prediction
system-one
si-substrate
intent-classification
routing
abstention
uncertainty-quantification
selective-prediction
hebbian
energy-based-model
associative-memory
no-transformer
no-neural-network
no-pattern-matching
moderation
spam-detection
multilingual
| # Datasets — sources, splits, licenses | |
| Every state string in this repo is verbatim dataset text. The only edits are | |
| truncation (long fields capped) and label remapping to opaque anchors. | |
| `MANIFEST.json` carries per-file SHA-256 and per-domain row counts; the | |
| preparation recipe (and the three failure-mode fixes it bakes in) is in | |
| `MANIFEST.json -> recipe`. | |
| ## Included corpora (16 domains) | |
| | domain | source | train | cal | hidden | notes | | |
| |---|---|---:|---:|---:|---| | |
| | tickets_en | repo synthetic support corpus (data/big, SHA-256 manifest) | 8,400 | 1,800 | 1,800 | choice, 3 departments | | |
| | tickets_bn/hi/ru | repo synthetic multilingual support corpus | 2,520-2,522 | 539-540 | 539-540 | bengali/devanagari/cyrillic substrates | | |
| | game | repo synthetic game-command corpus | 5,600 | 1,200 | 1,200 | choice, 3 actions | | |
| | guard | repo synthetic guardrail corpus | 5,600 | 1,200 | 1,200 | noul (irreversible, off_task) + choice (scope); original schema kept verbatim | | |
| | bank77 | [PolyAI banking77](https://github.com/PolyAI-LDN/task-specific-dialogues) official splits | 9,000 | 1,003 | 3,080 | 77 intents, official test = hidden | | |
| | sms | [UCI SMS Spam Collection](https://archive.ics.uci.edu/dataset/228/sms+spam+collection) | 3,900 | 836 | 838 | free-for-research corpus | | |
| | clinc | [clinc/clinc_oos](https://huggingface.co/datasets/clinc/clinc_oos) (plus config) | 10,000 | 3,000 | 4,500 | 151 intents incl. out-of-scope | | |
| | enron | [SetFit/enron_spam](https://huggingface.co/datasets/SetFit/enron_spam) | 6,000 | 758 | 1,113 | cal/hidden carved from official test, disjoint | | |
| | emotion | [dair-ai/emotion](https://huggingface.co/datasets/dair-ai/emotion) | 6,000 | 700 | 2,000 | 6 emotions | | |
| | hate | [tdavidson/hate_speech_offensive](https://huggingface.co/datasets/tdavidson/hate_speech_offensive) | 4,078 | 400 | 1,500 | official train only -> stratified carve (seed 77); offensive class capped at 2,500 in train, natural distribution in eval | | |
| | agnews | [fancyzhx/ag_news](https://huggingface.co/datasets/fancyzhx/ag_news) | 6,000 | 400 | 2,000 | 4 topics; cal carved from train | | |
| | polarity | [fancyzhx/amazon_polarity](https://huggingface.co/datasets/fancyzhx/amazon_polarity) | 6,000 | 400 | 2,000 | 2 classes; cal carved from train | | |
| | dbpedia | [fancyzhx/dbpedia_14](https://huggingface.co/datasets/fancyzhx/dbpedia_14) | 6,000 | 400 | 2,000 | 14 entity classes | | |
| | boolq | [google/boolq](https://huggingface.co/datasets/google/boolq) | 6,000 | 500 | 1,500 | question+passage, boolean choice; cal+hidden carved from official validation | | |
| Total: 76,142 train rows (152,284 lessons per epoch; the shipped model was | |
| trained ONE interleaved pass — see docs/EVALUATION.md §Honest reading for | |
| the consolidation analysis). | |
| ## Split discipline | |
| - Hidden files carry `_hidden` in their names; the M1 firewall refuses them | |
| for `learn` and `calibrate` (hidden rows never teach and never set | |
| calibration scalars). | |
| - cal and hidden are disjoint per domain (enforced in `tools/giant_prepare.py`); | |
| cal feeds per-script temperature/Platt fitting only. | |
| - Model/flag selection stays on cal; hidden is scored once for the report. | |
| ## File format (eval + cal) | |
| The shipped `eval/*_hidden.jsonl` and `cal_*.jsonl` rows are SCHEMA-FREE: | |
| `{"state", "domain", "labels"}` (guard/noul rows also carry their | |
| `questions` inline, since noul instructions are the probe carrier). The | |
| question objects (router + class criteria) are rebuilt from | |
| `schemas.json` — see `build_questions()` in the main repo's | |
| `tools/giant_bench.py`. This keeps the repo small; the full-schema corpus | |
| regenerates via `tools/giant_prepare.py` in the main SyFox repo. | |
| ## Licenses | |
| - SyFox engine: MIT (see LICENSE). | |
| - bank77: released by PolyAI for research (check the repository for the | |
| current license text). | |
| - UCI SMS Spam Collection: free for research with attribution. | |
| - HuggingFace datasets (clinc_oos, enron_spam, emotion, hate_speech_offensive, | |
| ag_news, amazon_polarity, dbpedia_14, boolq): each retains its original | |
| publisher license — consult the dataset page before redistributing or | |
| commercial use. | |
| - The repo synthetic corpora are original generated data (MIT, same as the | |
| engine). | |