Sloth-27B v1
Sloth-27B v1 makes structured decisions from text or JSON context. Supply a question and its allowed answers to obtain a selected answer and a probability distribution over the candidates. Applications include intent routing, relevance judgments and rule-based workflow decisions.
Built on vllm-sr/d3, this release includes the complete model weights and decision readout.
Model
- Backbone: approximately 26.09B parameters, stored in 11 BF16 safetensors shards.
- Decision head: FP32 readout over 255 answer codes, applied at the final prompt position.
- Outputs: candidate probabilities and typed decisions, without free-form text generation.
- Multiple questions: each question is rendered with a copy of the shared state; question prompts can be batched.
- Scope: text decision tasks. Inherited vision weights have not been evaluated for this release.
Usage
The native Transformers interface accepts individual decision requests. Use a Linux GPU environment with a CUDA-compatible PyTorch/torchvision installation, then install the bundled native dependencies:
python -m pip install huggingface_hub
hf auth login # required while this repository is private
hf download cjqhhh/Sloth-27B-v1 --local-dir ./Sloth-27B-v1
python -m pip install -r ./Sloth-27B-v1/requirements-native.txt
From the directory containing the downloaded model:
from transformers import AutoModel
model = AutoModel.from_pretrained(
"./Sloth-27B-v1",
trust_remote_code=True,
device="cuda:0",
batch_size=1,
verify="fast",
)
response = model.system_one(
state={"message": "Please send me the invoice for my order."},
questions={
"route": {
"type": "choice",
"instructions": "Choose the team that should handle this request.",
"criteria": {
"billing": "Invoices, payments and refunds",
"technical": "Product errors and technical support",
"sales": "Product information and purchase enquiries",
},
}
},
)
answer = response["answers"]["route"]
print(answer["choice"])
print(answer["probabilities"])
The example's syntax and request format have been checked against the bundled code; native GPU inference has not been retested for this release. Model weights occupy about 52.2 GB, with additional runtime memory required.
Input and output
| Field | Meaning |
|---|---|
state |
Text or JSON describing the situation |
questions |
A mapping from question IDs to question objects |
type |
choice for candidate selection; noul for a binary decision |
instructions |
The decision to make |
criteria |
For choice, 1–255 candidate keys mapped to descriptions; for noul, optional false/true descriptions |
Results appear in response["answers"], keyed by question ID. A choice answer contains the selected choice and candidate probabilities; a noul answer contains noul, the probability of true.
Candidate logits are divided by the configured temperature (1.0), then normalized with softmax within each question. Probability calibration has not been independently evaluated.
- Downloads last month
- 45