license: apache-2.0
language:
- en
- zh
tags:
- decision-model
- judge
- reranker
- calibration
- system-one
pipeline_tag: text-classification
llav — the open decision model (v0.4 text · v0.5-preview image & audio)
llav turns a fuzzy if in software into a typed call: give it a state (ticket, passage, image, audio clip) and a yes/no question, get back a calibrated probability. It speaks the System One protocol (drop-in for Jev-style clients) and is served at https://llav.app (free tier: 10,000 decisions / day / key).
This card describes the models and publishes the benchmark numbers. Weights are not in this repository yet: the text judge weights (llav-0.4-round0, llav-0.4-rounds) and the image/audio adapters open with the v0.5 release. Code, training and evaluation scripts are open now at https://github.com/llachat/llav.
Models
| Model | Base | What it judges |
|---|---|---|
| llav-0.4 text judge | Qwen3-Reranker-4B (LoRA, weight-averaged) | evidence / relevance / yes-no questions about text |
| llav image judge (0.5-preview) | Qwen3-VL-Reranker-8B (LoRA) | yes/no questions about an image (one or two photos) |
| llav audio judge (0.5-preview) | Qwen2.5-Omni-3B thinker (LoRA, bf16) | emotion / intent / sound questions about a clip |
Numbers (measured by us, same candidates, same day; details and caveats at https://llav.app/benchmarks)
| Benchmark | llav | Jev 1.14 | Gemini Decision 1 |
|---|---|---|---|
| Text judge, MuSiQue held-out q50–200 (AUC) | 0.970 single 4B · 0.977 five-model ensemble | 0.969 | 0.966 |
| Calibration, round 0 (Brier / ECE, lower is better) | 0.0095 / 0.004 | 0.0113 / 0.015 | 0.0118 / 0.031 |
| Image judge, blind set (DocVQA / ChartQA / MMStar, 3,120 items, AUC) | 0.830 | 0.984 | 0.962 |
| Audio judge, blind set (CREMA-D emotion, 1,200 clips, AUC) | 0.697 | 0.455 | 0.714 |
Image and audio numbers are blind sets (tasks llav never saw in training). On sets drawn from llav's own training sources it scores 0.950 (image) and 0.952 (audio).
Use
curl https://api.llav.app/v1/systemone -H "Authorization: Bearer $LLAV_KEY" -H "Content-Type: application/json" \
-d '{"state":"Customer: the parcel never arrived and I want my money back.","questions":{"refund":{"type":"noul","instructions":"Is the customer asking for a refund?"}}}'
Self-hosting (MLX on Apple silicon, or one NVIDIA GPU) is documented in the GitHub repo; weights follow with v0.5.
License
Apache-2.0 for code and the released weights. Benchmarks reuse public datasets under their own licenses.