abhi6168 commited on
Commit
7da514f
·
verified ·
1 Parent(s): 7e63cc4

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +87 -0
README.md ADDED
@@ -0,0 +1,87 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: Qwen/Qwen3.5-4B
4
+ library_name: peft
5
+ tags:
6
+ - lora
7
+ - decision-model
8
+ - jev
9
+ - system-one
10
+ - text-classification
11
+ pipeline_tag: text-classification
12
+ ---
13
+
14
+ # ekVachan-decoder
15
+
16
+ LoRA adapters (r=16, alpha=32, 128 target modules) for
17
+ [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B), the decoder arm of
18
+ **ekVachan** -- an open, self-hostable alternative to TypeSafe AI's
19
+ [Jev System-One](https://typesafe.ai/blog/introducing-system-one-models-and-jev)
20
+ decision model. Given a `state` and a set of typed `questions` (`choice`,
21
+ `score`, `noul`), it returns calibrated probabilities in one non-autoregressive
22
+ forward pass (a restricted-logit read over single-token letter codes), not a
23
+ generated response.
24
+
25
+ Full source, training pipeline, and evidence trail:
26
+ [github.com/asp616848/better-jev-for-all](https://github.com/asp616848/better-jev-for-all)
27
+ (`README.md` for the summary, `PRD.md` for every measured number and how it
28
+ was produced).
29
+
30
+ ## What's in this repo
31
+
32
+ Two named LoRA adapters over the same base model, hot-swapped at serving time
33
+ by `serve.inference.RoutingDecoderModel`:
34
+
35
+ - `ekvachan-decoder-qwen-benchcorpus/` -- text-only, `max_options=26`
36
+ - `ekvachan-decoder-qwen-vision/` -- the strongest (`stage3`) checkpoint,
37
+ vision-capable, `max_options=588`; also answers text-only requests by
38
+ default in this project's reference server (`EKVACHAN_TEXT_ADAPTER=vision`)
39
+
40
+ Each subfolder carries its own `manifest.json` (training config, real
41
+ measured accuracy/Brier/ECE) alongside the adapter weights -- treat that file
42
+ as the source of truth over this card if they ever disagree.
43
+
44
+ ## Real, verified numbers
45
+
46
+ Third-party benchmarks, zero training exposure, measured against the actual
47
+ serving class (`RoutingDecoderModel`, eager and CUDA-graph modes identical
48
+ within noise -- see PRD.md 13a.33):
49
+
50
+ | Benchmark | Accuracy |
51
+ |---|---|
52
+ | JevBench (231/231 items) | **70.99%** |
53
+ | jabr-v2 (944/944 items) | **85.49%** |
54
+
55
+ Latency, on our own dedicated reference server (CUDA graphs on by default,
56
+ PRD.md 13a.34): **~112ms p50 / ~131ms p95** on realistic (100-250 token)
57
+ request shapes; ~45ms on short (~15-token) synthetic prompts. This model is
58
+ not yet competitive with the fastest field entries (Von <18ms, Laya ~16ms)
59
+ on raw latency -- see the main repo's README for the full comparison and an
60
+ honest accounting of what CUDA graphs did and didn't fix.
61
+
62
+ ## Usage
63
+
64
+ Not a plain `AutoModelForCausalLM.generate()` model -- it requires the
65
+ restricted-logit read mechanism (build a single-letter code-table prompt,
66
+ take the last token's logits, restrict to the option codes, softmax). Use
67
+ `serve.inference.RoutingDecoderModel` from the main repo directly rather than
68
+ reimplementing this:
69
+
70
+ ```python
71
+ from serve.inference import RoutingDecoderModel
72
+
73
+ model = RoutingDecoderModel(checkpoints_dir="path/to/this/repo's/local/clone")
74
+ result = model.predict_choice(
75
+ "The customer says the package never arrived.",
76
+ ["refund", "replace", "escalate"],
77
+ )
78
+ print(result["choice"], result["probabilities"])
79
+ ```
80
+
81
+ Or try it with zero setup in the
82
+ [ekVachan Space](https://huggingface.co/spaces/abhi6168/ekvachan) (note: Space
83
+ latency includes ZeroGPU queueing, not representative of the numbers above).
84
+
85
+ ## License
86
+
87
+ Apache-2.0, same as the base model and the main repo.