Text Classification
Transformers
Safetensors
English
modernbert
llm-routing
capability-vector
system-one
text-embeddings-inference
Instructions to use regolo/brick-sysone-V1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use regolo/brick-sysone-V1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="regolo/brick-sysone-V1")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("regolo/brick-sysone-V1") model = AutoModelForSequenceClassification.from_pretrained("regolo/brick-sysone-V1", device_map="auto") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from regolo/brick-sysone-V1: direct link, hf CLI and curl.
- Browser
- Download file 4.31 kB
-
https://huggingface.co/regolo/brick-sysone-V1/resolve/main/README.md
- Command line
-
hf download hf://regolo/brick-sysone-V1/README.md
-
curl -L -o README.md https://huggingface.co/regolo/brick-sysone-V1/resolve/main/README.md
4.31 kB
| license: apache-2.0 | |
| base_model: answerdotai/ModernBERT-large | |
| library_name: transformers | |
| pipeline_tag: text-classification | |
| tags: | |
| - text-classification | |
| - modernbert | |
| - llm-routing | |
| - capability-vector | |
| - system-one | |
| task_categories: | |
| - text-classification | |
| language: | |
| - en | |
| # Brick SystemOne Large v2 — query to capability vector | |
| Fine-tuned `answerdotai/ModernBERT-large` (8192-token context) that maps a raw | |
| query to a six-dimensional capability vector. Each dimension is an independent | |
| sigmoid output in `[0, 1]`; the vector is neither normalized to sum to one nor | |
| rounded. | |
| This checkpoint is the local router model used by | |
| [Brick](https://github.com/regolo-ai/brick-SR1). | |
| ## Output dimensions | |
| The output order is fixed: | |
| 1. `instruction_following` | |
| 2. `coding` | |
| 3. `math_reasoning` | |
| 4. `world_knowledge` | |
| 5. `planning_agentic` | |
| 6. `creative_synthesis` | |
| ## Training | |
| - Base: `answerdotai/ModernBERT-large`, 28 layers, hidden size 1024, 8192 context. | |
| - Data: 68,882 query/label pairs. Labels are cross-model judge means with | |
| difficulty modulation; v2 adds 16k pair-transfer rows (short queries inherit | |
| detailed labels, including 3.1k ultra-short queries under 40 characters) and | |
| 3k garbage rows mapped to near-zero vectors. | |
| - Configuration: LR 8.93e-5, 4 epochs, gradient accumulation 8, bf16, SDPA. | |
| v2 differences from v1: the brevity gap on short prompts drops from 0.27 to | |
| 0.13, and the garbage mean-max drops from 0.36 to 0.006 with no spikes. | |
| ## Evaluation | |
| - Holdout (4,968 rows): pearson_macro **0.9656**, mae 0.058. | |
| - Mapping battery (30 P1, 6 P2, 40 garbage): P1 gap 0.13, P2 direction 6/6. | |
| ## Quickstart | |
| The output is a capability profile, not a distribution over models: each value | |
| is an independent sigmoid in `[0, 1]`. | |
| ### Install | |
| ```bash | |
| python3 -m venv .venv-systemone | |
| .venv-systemone/bin/python -m pip install --upgrade pip | |
| .venv-systemone/bin/python -m pip install --index-url https://download.pytorch.org/whl/cu128 \ | |
| torch==2.11.0 | |
| .venv-systemone/bin/python -m pip install transformers==5.20.2 safetensors==0.8.0 \ | |
| tokenizers==0.23.3 numpy==1.26.4 huggingface-hub==2.2.0 | |
| ``` | |
| Use `https://download.pytorch.org/whl/cu126` for driver 560-569, or | |
| `https://download.pytorch.org/whl/cpu` for CPU-only. The script below picks | |
| BF16 on CUDA and FP32 on CPU, as recommended. | |
| ### Terminal chat | |
| Type a query, read the vector. Save as `chat.py` and run | |
| `.venv-systemone/bin/python chat.py`. | |
| ```python | |
| import torch | |
| from transformers import AutoModelForSequenceClassification, AutoTokenizer | |
| LABELS = ["instruction_following", "coding", "math_reasoning", | |
| "world_knowledge", "planning_agentic", "creative_synthesis"] | |
| REVISION = "9263a397a366ee22921214ddeebab1f5e4e8facd" | |
| device = "cuda" if torch.cuda.is_available() else "cpu" | |
| dtype = torch.bfloat16 if device == "cuda" else torch.float32 | |
| model = AutoModelForSequenceClassification.from_pretrained( | |
| "regolo/brick-sysone-V1", revision=REVISION, dtype=dtype, | |
| attn_implementation="sdpa", | |
| ).to(device).eval() | |
| tokenizer = AutoTokenizer.from_pretrained("regolo/brick-sysone-V1", revision=REVISION) | |
| limit = min(model.config.max_position_embeddings, 8192) | |
| while True: | |
| try: | |
| query = input("query> ").strip() | |
| except (EOFError, KeyboardInterrupt): | |
| break | |
| if not query: | |
| continue | |
| encoded = tokenizer(query, return_tensors="pt", truncation=True, max_length=limit).to(device) | |
| with torch.inference_mode(): | |
| vector = model(**encoded).logits[0].sigmoid().float().tolist() | |
| for label, value in zip(LABELS, vector): | |
| print(f" {label:<20} {value:.4f}") | |
| print(f" vector: [{', '.join(f'{v:.4f}' for v in vector)}]") | |
| ``` | |
| ``` | |
| query> write a python parser that streams csv rows | |
| instruction_following 0.5117 | |
| coding 0.9258 | |
| math_reasoning 0.0903 | |
| world_knowledge 0.1157 | |
| planning_agentic 0.0630 | |
| creative_synthesis 0.0198 | |
| vector: [0.5117, 0.9258, 0.0903, 0.1157, 0.0630, 0.0198] | |
| ``` | |
| Input is truncated at 8192 tokens; longer inputs are outside the training | |
| distribution. The Brick router classifies only the last user message, never | |
| system or assistant turns. Full scripts with `--json`, `--device` and one-shot | |
| mode: `examples/systemone/` in the Brick repository. | |
| ## License | |
| Apache-2.0, matching the base model. See `LICENSE`. | |