Sloth-27B v1

Sloth-27B v1 makes structured decisions from text or JSON context. Supply a question and its allowed answers to obtain a selected answer and a probability distribution over the candidates. Applications include intent routing, relevance judgments and rule-based workflow decisions.

Built on vllm-sr/d3, this release includes the complete model weights and decision readout.

Model

  • Backbone: approximately 26.09B parameters, stored in 11 BF16 safetensors shards.
  • Decision head: FP32 readout over 255 answer codes, applied at the final prompt position.
  • Outputs: candidate probabilities and typed decisions, without free-form text generation.
  • Multiple questions: each question is rendered with a copy of the shared state; question prompts can be batched.
  • Scope: text decision tasks. Inherited vision weights have not been evaluated for this release.

Usage

The native Transformers interface accepts individual decision requests. Use a Linux GPU environment with a CUDA-compatible PyTorch/torchvision installation, then install the bundled native dependencies:

python -m pip install huggingface_hub
hf auth login  # required while this repository is private
hf download cjqhhh/Sloth-27B-v1 --local-dir ./Sloth-27B-v1
python -m pip install -r ./Sloth-27B-v1/requirements-native.txt

From the directory containing the downloaded model:

from transformers import AutoModel

model = AutoModel.from_pretrained(
    "./Sloth-27B-v1",
    trust_remote_code=True,
    device="cuda:0",
    batch_size=1,
    verify="fast",
)

response = model.system_one(
    state={"message": "Please send me the invoice for my order."},
    questions={
        "route": {
            "type": "choice",
            "instructions": "Choose the team that should handle this request.",
            "criteria": {
                "billing": "Invoices, payments and refunds",
                "technical": "Product errors and technical support",
                "sales": "Product information and purchase enquiries",
            },
        }
    },
)
answer = response["answers"]["route"]
print(answer["choice"])
print(answer["probabilities"])

The example's syntax and request format have been checked against the bundled code; native GPU inference has not been retested for this release. Model weights occupy about 52.2 GB, with additional runtime memory required.

Input and output

Field Meaning
state Text or JSON describing the situation
questions A mapping from question IDs to question objects
type choice for candidate selection; noul for a binary decision
instructions The decision to make
criteria For choice, 1–255 candidate keys mapped to descriptions; for noul, optional false/true descriptions

Results appear in response["answers"], keyed by question ID. A choice answer contains the selected choice and candidate probabilities; a noul answer contains noul, the probability of true.

Candidate logits are divided by the configured temperature (1.0), then normalized with softmax within each question. Probability calibration has not been independently evaluated.

Downloads last month
45
Safetensors
Model size
26B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for cjqhhh/Sloth-27B-v1

Base model

Qwen/Qwen3.8-27B
Finetuned
vllm-sr/d3
Finetuned
(4)
this model

Space using cjqhhh/Sloth-27B-v1 1