Instructions to use abhi6168/ekvachan-decoder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use abhi6168/ekvachan-decoder with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,87 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: Qwen/Qwen3.5-4B
|
| 4 |
+
library_name: peft
|
| 5 |
+
tags:
|
| 6 |
+
- lora
|
| 7 |
+
- decision-model
|
| 8 |
+
- jev
|
| 9 |
+
- system-one
|
| 10 |
+
- text-classification
|
| 11 |
+
pipeline_tag: text-classification
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# ekVachan-decoder
|
| 15 |
+
|
| 16 |
+
LoRA adapters (r=16, alpha=32, 128 target modules) for
|
| 17 |
+
[Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B), the decoder arm of
|
| 18 |
+
**ekVachan** -- an open, self-hostable alternative to TypeSafe AI's
|
| 19 |
+
[Jev System-One](https://typesafe.ai/blog/introducing-system-one-models-and-jev)
|
| 20 |
+
decision model. Given a `state` and a set of typed `questions` (`choice`,
|
| 21 |
+
`score`, `noul`), it returns calibrated probabilities in one non-autoregressive
|
| 22 |
+
forward pass (a restricted-logit read over single-token letter codes), not a
|
| 23 |
+
generated response.
|
| 24 |
+
|
| 25 |
+
Full source, training pipeline, and evidence trail:
|
| 26 |
+
[github.com/asp616848/better-jev-for-all](https://github.com/asp616848/better-jev-for-all)
|
| 27 |
+
(`README.md` for the summary, `PRD.md` for every measured number and how it
|
| 28 |
+
was produced).
|
| 29 |
+
|
| 30 |
+
## What's in this repo
|
| 31 |
+
|
| 32 |
+
Two named LoRA adapters over the same base model, hot-swapped at serving time
|
| 33 |
+
by `serve.inference.RoutingDecoderModel`:
|
| 34 |
+
|
| 35 |
+
- `ekvachan-decoder-qwen-benchcorpus/` -- text-only, `max_options=26`
|
| 36 |
+
- `ekvachan-decoder-qwen-vision/` -- the strongest (`stage3`) checkpoint,
|
| 37 |
+
vision-capable, `max_options=588`; also answers text-only requests by
|
| 38 |
+
default in this project's reference server (`EKVACHAN_TEXT_ADAPTER=vision`)
|
| 39 |
+
|
| 40 |
+
Each subfolder carries its own `manifest.json` (training config, real
|
| 41 |
+
measured accuracy/Brier/ECE) alongside the adapter weights -- treat that file
|
| 42 |
+
as the source of truth over this card if they ever disagree.
|
| 43 |
+
|
| 44 |
+
## Real, verified numbers
|
| 45 |
+
|
| 46 |
+
Third-party benchmarks, zero training exposure, measured against the actual
|
| 47 |
+
serving class (`RoutingDecoderModel`, eager and CUDA-graph modes identical
|
| 48 |
+
within noise -- see PRD.md 13a.33):
|
| 49 |
+
|
| 50 |
+
| Benchmark | Accuracy |
|
| 51 |
+
|---|---|
|
| 52 |
+
| JevBench (231/231 items) | **70.99%** |
|
| 53 |
+
| jabr-v2 (944/944 items) | **85.49%** |
|
| 54 |
+
|
| 55 |
+
Latency, on our own dedicated reference server (CUDA graphs on by default,
|
| 56 |
+
PRD.md 13a.34): **~112ms p50 / ~131ms p95** on realistic (100-250 token)
|
| 57 |
+
request shapes; ~45ms on short (~15-token) synthetic prompts. This model is
|
| 58 |
+
not yet competitive with the fastest field entries (Von <18ms, Laya ~16ms)
|
| 59 |
+
on raw latency -- see the main repo's README for the full comparison and an
|
| 60 |
+
honest accounting of what CUDA graphs did and didn't fix.
|
| 61 |
+
|
| 62 |
+
## Usage
|
| 63 |
+
|
| 64 |
+
Not a plain `AutoModelForCausalLM.generate()` model -- it requires the
|
| 65 |
+
restricted-logit read mechanism (build a single-letter code-table prompt,
|
| 66 |
+
take the last token's logits, restrict to the option codes, softmax). Use
|
| 67 |
+
`serve.inference.RoutingDecoderModel` from the main repo directly rather than
|
| 68 |
+
reimplementing this:
|
| 69 |
+
|
| 70 |
+
```python
|
| 71 |
+
from serve.inference import RoutingDecoderModel
|
| 72 |
+
|
| 73 |
+
model = RoutingDecoderModel(checkpoints_dir="path/to/this/repo's/local/clone")
|
| 74 |
+
result = model.predict_choice(
|
| 75 |
+
"The customer says the package never arrived.",
|
| 76 |
+
["refund", "replace", "escalate"],
|
| 77 |
+
)
|
| 78 |
+
print(result["choice"], result["probabilities"])
|
| 79 |
+
```
|
| 80 |
+
|
| 81 |
+
Or try it with zero setup in the
|
| 82 |
+
[ekVachan Space](https://huggingface.co/spaces/abhi6168/ekvachan) (note: Space
|
| 83 |
+
latency includes ZeroGPU queueing, not representative of the numbers above).
|
| 84 |
+
|
| 85 |
+
## License
|
| 86 |
+
|
| 87 |
+
Apache-2.0, same as the base model and the main repo.
|