csabag76 commited on
Commit
e556cf0
·
verified ·
1 Parent(s): 101709c

Card: v1 recommended (demo), v2 experimental

Browse files
Files changed (1) hide show
  1. README.md +15 -11
README.md CHANGED
@@ -18,10 +18,11 @@ A Qwen3.5-0.8B fine-tune that turns a job seeker's preferences, written in plain
18
  runs that object on each job or recruiter email and returns **Through / Maybe / Held** with reasons. The model writes
19
  the decision procedure once; it never sees the emails.
20
 
21
- This repository holds two versions. **v2 is the current one** (folder `v2/`). **v1** (the files in the repository root)
22
- is kept unchanged so earlier results can be reproduced.
 
23
 
24
- ## v2 (current)
25
 
26
  Qwen3.5-0.8B, LoRA SFT, merged into the weights and exported as q8_0. It was trained only on synthetic data: invented
27
  job seekers and invented jobs, generated with the second version of the synthetic set ("synth v2"), whose preference
@@ -38,9 +39,12 @@ Gemini Flash); for each user, the candidate tree that did best when **executed**
38
  | `example-output.json` | the tree v2 wrote for that user (greedy, with the grammar). It passes the tree validator but was not scored against gold verdicts, and it is not a faithful reading of the message: it adds a pay ramp the user never asked for, and it has no accept rule for the dream companies |
39
  | `SHA256SUMS` | checksums of the files above |
40
 
41
- ### How to use it
42
 
43
- 1. Decode with the v2 grammar `v2/mm_factors.gbnf`, greedy, `enable_thinking` off in the chat template, up to 4096 new tokens.
 
 
 
44
  2. **Validate** the tree with a semantic validator (arities, names that are declared, lists that exist).
45
  3. If it is not valid, **retry up to 2 times** at temperature 0.3 with seeds 1234+k (k = 1, 2), no top-k/top-p/min-p, with the same grammar, and keep the first valid tree. If none is valid, treat the user as failed; do not run an invalid tree.
46
 
@@ -60,15 +64,15 @@ invalid tree as wrong on every job. A *wrong hold* hides a job that fits the use
60
 
61
  | model | valid trees | agreement (all) | wrong holds per 100 jobs |
62
  |---|---|---|---|
63
- | v1, same setup | 0.98 | 0.694 | 0.52 |
64
- | **v2** | 0.96 | **0.753** | **0.84** |
65
 
66
  Read this table carefully: v2 agrees more often with the gold verdicts, but it has **more** wrong holds per 100 jobs
67
  than v1 under this setup (0.84 against 0.52), and slightly fewer valid trees (0.96 against 0.98).
68
 
69
- - The release bar for wrong holds was 0.30 per 100 jobs. That bar was set from an unconstrained GPU run. Under the
70
- arity grammar and retry on the Mac, neither version meets it, and v2 is further from it than v1. The author decided to
71
- release v2 anyway; this is an override of the bar, not a pass.
72
  - All numbers are on a dev set. **v2 has not been evaluated on a held-out test set yet.**
73
  - Real job postings were used only for qualitative checks, never for the numbers above.
74
  - The v1 numbers in its own card (valid 0.89, agreement 0.773, wrong holds 0.27) were measured with an earlier setup (an unconstrained run, no arity grammar, no retry) and are not comparable with this table.
@@ -86,7 +90,7 @@ The GGUF header was edited to drop Qwen3.5's unused multi-token-prediction layer
86
  `nextn_predict_layers` 0), which wllama otherwise refuses. The tensor data are byte for byte the ones of the trained
87
  export. All data used to train and evaluate is invented. License: Apache-2.0, as for v1.
88
 
89
- ## v1 (repository root, unchanged)
90
 
91
  `tree-compiler-0.8b-q8_0.gguf`, `mm_factors.gbnf` (the earlier grammar without arities), `example-messages.json`,
92
  `example-output.json`, and the 4-bit phone shards under `phone/`. Results on its dev set (valid 0.89, agreement 0.773,
 
18
  runs that object on each job or recruiter email and returns **Through / Maybe / Held** with reasons. The model writes
19
  the decision procedure once; it never sees the emails.
20
 
21
+ This repository holds two versions. **v1 (the files in the repository root) is the recommended one**. Use it with the
22
+ arity grammar and the validate-and-retry step described below; this is what the public demo runs. **v2** (folder `v2/`)
23
+ is an **experimental** retrain: it agrees with the gold verdicts more often, but it hides more good jobs (see Results).
24
 
25
+ ## v2 (experimental)
26
 
27
  Qwen3.5-0.8B, LoRA SFT, merged into the weights and exported as q8_0. It was trained only on synthetic data: invented
28
  job seekers and invented jobs, generated with the second version of the synthetic set ("synth v2"), whose preference
 
39
  | `example-output.json` | the tree v2 wrote for that user (greedy, with the grammar). It passes the tree validator but was not scored against gold verdicts, and it is not a faithful reading of the message: it adds a pay ramp the user never asked for, and it has no accept rule for the dream companies |
40
  | `SHA256SUMS` | checksums of the files above |
41
 
42
+ ### How to use it (v1 and v2 alike)
43
 
44
+ For **v1** (recommended), use the root GGUF with the same arity grammar (`v2/mm_factors.gbnf`) and the same retry; for v2, use the `v2/` GGUF.
45
+
46
+
47
+ 1. Decode with the arity grammar `v2/mm_factors.gbnf`, greedy, `enable_thinking` off in the chat template, up to 4096 new tokens.
48
  2. **Validate** the tree with a semantic validator (arities, names that are declared, lists that exist).
49
  3. If it is not valid, **retry up to 2 times** at temperature 0.3 with seeds 1234+k (k = 1, 2), no top-k/top-p/min-p, with the same grammar, and keep the first valid tree. If none is valid, treat the user as failed; do not run an invalid tree.
50
 
 
64
 
65
  | model | valid trees | agreement (all) | wrong holds per 100 jobs |
66
  |---|---|---|---|
67
+ | **v1, same setup (recommended)** | 0.98 | 0.694 | 0.52 |
68
+ | v2 (experimental) | 0.96 | 0.753 | 0.84 |
69
 
70
  Read this table carefully: v2 agrees more often with the gold verdicts, but it has **more** wrong holds per 100 jobs
71
  than v1 under this setup (0.84 against 0.52), and slightly fewer valid trees (0.96 against 0.98).
72
 
73
+ - The release bar for wrong holds was 0.30 per 100 jobs, set from an unconstrained GPU run. Under the arity grammar and
74
+ retry on the Mac, neither version meets it, and v2 is further from it than v1. Because a wrong hold is the costlier
75
+ error, v1 stays the recommended model; v2 is published as experimental, not as a release that passed the bar.
76
  - All numbers are on a dev set. **v2 has not been evaluated on a held-out test set yet.**
77
  - Real job postings were used only for qualitative checks, never for the numbers above.
78
  - The v1 numbers in its own card (valid 0.89, agreement 0.773, wrong holds 0.27) were measured with an earlier setup (an unconstrained run, no arity grammar, no retry) and are not comparable with this table.
 
90
  `nextn_predict_layers` 0), which wllama otherwise refuses. The tensor data are byte for byte the ones of the trained
91
  export. All data used to train and evaluate is invented. License: Apache-2.0, as for v1.
92
 
93
+ ## v1 (repository root, unchanged; recommended)
94
 
95
  `tree-compiler-0.8b-q8_0.gguf`, `mm_factors.gbnf` (the earlier grammar without arities), `example-messages.json`,
96
  `example-output.json`, and the 4-bit phone shards under `phone/`. Results on its dev set (valid 0.89, agreement 0.773,