model card: v0.2 changelog + known order-sensitivity issue
Browse files
README.md
CHANGED
|
@@ -182,6 +182,24 @@ Thai sets and on NLI/topic tasks, and route low-confidence English yes/no decisi
|
|
| 182 |
| v0.2 | 2026-09-21 | +3,000 SFT steps from v0.1 with a 22k-record synthetic Thai social-sentiment set (4/3/5-class, yes/no, score schemes), re-calibrated | **63.2** | 51.5 |
|
| 183 |
| v0.1 | 2026-09-20 | initial release: Thai CPT 4.47B tokens, 12k-step SFT, calibration | 61.9 | 38.7 |
|
| 184 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 185 |
## License & credits
|
| 186 |
|
| 187 |
Apache-2.0. Built by iApp Technology / OpenThai on Qwen3.5-0.8B-Base (Apache-2.0). Inspired by TypeSafe AI's
|
|
|
|
| 182 |
| v0.2 | 2026-09-21 | +3,000 SFT steps from v0.1 with a 22k-record synthetic Thai social-sentiment set (4/3/5-class, yes/no, score schemes), re-calibrated | **63.2** | 51.5 |
|
| 183 |
| v0.1 | 2026-09-20 | initial release: Thai CPT 4.47B tokens, 12k-step SFT, calibration | 61.9 | 38.7 |
|
| 184 |
|
| 185 |
+
## Changelog
|
| 186 |
+
|
| 187 |
+
**v0.2 β 2026-09-21**
|
| 188 |
+
- Continued fine-tuning for 3,000 steps from v0.1 with a new 22k-record synthetic Thai social-media sentiment set
|
| 189 |
+
(4-class with the "question" class, 3/5-class variants, yes/no flags, 5-level score), then re-calibrated.
|
| 190 |
+
- Public 13-subset macro 61.9 β **63.2**; Wisesight 38.7 β **51.5**; MASSIVE-en 75.7 β 79.1; MASSIVE-th 86.4 β 88.6;
|
| 191 |
+
MultiNLI 85.6 β 87.3 (ECE 0.035 β 0.016); Aegis2 58.0 β 61.6; PubMedQA 53.6 β 56.4.
|
| 192 |
+
- Regression: SIB-200 Thai 77.5 β 74.0 (sentiment set was weighted 4Γ; will be lowered next round). Unchanged: SQuAD2, SummEval-relevance.
|
| 193 |
+
- Known issue (measured on v0.2): answers can depend on the *order* of the options, mildly for β€ 10 options
|
| 194 |
+
(5β12% of arg-max choices flip under a random permutation, mean probability shift 0.05β0.14) and strongly for very
|
| 195 |
+
large option sets (77-way banking77: 72% flip). A permutation-averaging inference mode is in progress; until then, for
|
| 196 |
+
large option sets query with several option orders and average the probabilities.
|
| 197 |
+
- Live API switched to v0.2 (same checkpoint as these weights).
|
| 198 |
+
|
| 199 |
+
**v0.1 β 2026-09-20**
|
| 200 |
+
- Initial release: Qwen3.5-0.8B text tower, 4.47B-token Thai CPT, 256-slot decision head, 12k-step SFT on 1.8M
|
| 201 |
+
public + 98k synthetic decision records, calibration stage. Public macro 61.9.
|
| 202 |
+
|
| 203 |
## License & credits
|
| 204 |
|
| 205 |
Apache-2.0. Built by iApp Technology / OpenThai on Qwen3.5-0.8B-Base (Apache-2.0). Inspired by TypeSafe AI's
|