Instructions to use tory-labs/vica-laya with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Laya
How to use tory-labs/vica-laya with Laya:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Vica Laya
Vica Laya is a Korean-focused judgment fine-tune of Laya multilingual. Release 2026.11.13.1 contains B3, the epoch-3 Q8_0 model of the direct-judgment experiment, round two, arm B. It was trained from training-checkpoints/mix25-epoch-1 (release 2026.11.06.1) to answer three yes/no questions that the coding agent Vica asks on its direct edit path: check (does an edit do what was asked), locate (is a candidate line the place to change) and done (does the agent's last reply claim the work is finished), with the earlier judgment tasks kept in training. The user ordered this release after B3 passed the gates as revised on 2026-11-13; it did not pass the gates frozen before the experiment. The previous release 2026.11.08.1 is kept unchanged under releases/2026.11.08.1/, and its downloads stay in downloads/2026.11.08.1/.
한국어 문장을 읽고 주어진 선택지에 따라 판단하는 Laya 튜닝 버전입니다. 2026.11.13.1은 코딩 에이전트 Vica가 직접 수정할 때 묻는 세 가지 판단(check: 수정이 요청대로 됐나, locate: 이 줄이 고칠 곳인가, done: 마지막 답이 끝났다고 말하나)을 배운 B3입니다. 선택에 쓰지 않은 시험 분할에서 done은 98/98건으로 기준 모델 Jev(98/98)와 같았고, 처음 여는 test3 871행에서도 238/238건이었습니다. check(176/239, Jev 230/239)와 locate(56/82, Jev 71/82)는 Jev에 못 미쳐, Vica에서는 done만 이 모델이 답합니다. 기존 판단 과제는 이전 모델보다 대체로 나아졌습니다(뉴스 주제 474건 391건, 이전 2026.11.08.1 355건). 정답은 사람이 아니라 Claude가 쓰고 다른 Claude가 검증한 라벨입니다. 이 모델은 2026.11.08.1이 아니라 2026.11.06.1에서 출발해서 2026.11.08.1의 지식그래프·리서치 요약 학습이 들어 있지 않고, 그 자료에서의 정확도는 재지 않았습니다. 85% 목표는 달성하지 못했습니다.
Files and compatibility
downloads/2026.11.13.1/: the downloads. One file per platform holds thevica-layaprogram, the modelvica-laya.gguf(361,797,056 bytes; includes the tokenizer and compiled decision graph) and the license files.SHA256SUMSlists the three files.model.safetensors(float16, 643,835,524 bytes) withrl_agent_config.json,config.json,encoder/config.jsonandtokenizer/: the same release as a float checkpoint for thelayaPython package and for further tuning. It is the original training checkpoint of the exported epoch, copied byte-for-byte from the training run; see Float checkpoint.training-checkpoints/mix25-epoch-1/: the original float checkpoint of release2026.11.06.1and the base checkpoint this release was trained from. Unchanged.releases/2026.11.08.1/anddownloads/2026.11.08.1/: the previous model (이전 모델), exactly as published on 2026-11-08 and 2026-11-09.releases/2026.11.06.1/: the release before it, unchanged.topic-questions.json: the four-way news taxonomy used with this model.example-state.jsonandexample-request.jsonl: an authored Korean security example.evaluation.json: this release's results on the direct-edit jobs, the existing tasks and the reserved rows, the gates and what was not measured.model-manifest.jsonandSHA256SUMS: provenance, the release decision and file integrity.SHA256SUMSlists text files as Git stores them (LF line endings).experiments/direct-judgment/: the scripts, questions, plan and aggregate records (results-20261112.json,results-round2.json,results-round3.json) of the experiment. Private cases, labels and run outputs are not included.
Download
| Platform | File | Bytes | Needs on the computer |
|---|---|---|---|
| macOS arm64 (Metal) | vica-laya-macos-arm64.tar.gz |
334,328,315 | nothing else |
| Linux x86_64 (Vulkan) | vica-laya-linux-x86_64.tar.gz |
351,335,194 | Vulkan loader (libvulkan.so.1); built on Ubuntu 22.04 (glibc 2.35) |
| Windows x86_64 (Vulkan) | vica-laya-windows-x86_64.zip |
351,037,226 | Vulkan loader (vulkan-1.dll, installed with GPU drivers) |
x86_64 builds need a CPU with AVX2, FMA and BMI2 (Intel Haswell 2013, AMD Excavator 2015 or later). The Windows file carries the Microsoft Visual C++ runtime DLLs next to vica-laya.exe. The macOS program is not notarized, so a browser download may be blocked by Gatekeeper until it is allowed. Check a download against downloads/2026.11.13.1/SHA256SUMS before unpacking.
vica-laya is built from ggmlc v0.9.7 source at su-record/vica-laya (commit 4bc3a55, Actions run 43425125282; the runner source is unchanged since 5a75e53, only the pinned model and the rename check changed). It is a compiled laya.cpp decision runner, not a generative LLM runtime; the model is not for llama.cpp or Ollama. Its built-in help text still uses upstream wording.
vica-laya.gguf SHA-256:
c2d69b85ba80daf88f93c6e5bc6b5368d8d0491dcc1d85efeecdf1d5cb3df96e
It holds the same 130 tensors as B3's Q8 export (SHA-256 1babc5537e4ecc05516ea52ef86936980b700f5cc78ccfbeeade839ce417ce4d, not published). Only the model name and the checkpoint in its metadata (tory-labs/vica-laya@2026.11.13.1) were changed; answers were checked identical on the 1,034 direct-judgment calibration questions. For packaging it was staged at the repository root in commit 27561d9 and removed again by the release commit.
Run
Unpack the file for your platform and run from the vica-laya folder:
./vica-laya decide vica-laya.gguf \
--family multilingual \
--state-file example-state.json \
--questions-file topic-questions.json \
--device auto --json
example-state.json and topic-questions.json are in this repository. For repeated requests, keep one daemon alive and write one JSON request per line to its standard input:
./vica-laya daemon vica-laya.gguf --family multilingual --device auto
example-request.jsonl contains a complete request. Responses contain a selected category and scores. The scores are model estimates, not a measured probability that each answer is correct.
The news choices are security, ai, development, and other (business and unrelated topics). The question instructions define the boundaries; changing them can change accuracy. Use the provided questions when reproducing the reported news task.
Base checkpoint: original tuned mix25-epoch-1
Release 2026.11.13.1, like 2026.11.08.1 before it, continues training from the
mix25-epoch-1 checkpoint, the original pre-Q8 float checkpoint of release 2026.11.06.1. It is
published in
training-checkpoints/mix25-epoch-1
and has not changed: 643,835,524 bytes, SHA-256
c0889283894d642577c133f24f52842d89a02dd7d7261343ab4ddf76d2185043.
The package includes the matching tokenizer and configuration, plus hashes and
provenance. Its saved training configuration is preserved as historical evidence;
the pending-selection fields do not supersede the later release decisions.
To download this exact checkpoint on another computer (it is identical at every
revision since its first publication, 2ee14fb1dc3fdca01c75e6a2e0326a3db4f80673):
hf download tory-labs/vica-laya \
--include "training-checkpoints/mix25-epoch-1/**" --local-dir local/models
Pass local/models/training-checkpoints/mix25-epoch-1 to laya.load() or
scripts/laya-tuning/train.py --base. Loading the saved checkpoint on CPU in
offline mode produced security with probability 0.9378 for the bundled example.
This checks execution only. Optimizer state and private training data are not
part of the package.
Float checkpoint
model.safetensors holds the 2026.11.13.1 weights in the layout of the upstream checkpoint, so the laya Python package (tested with 0.4.22) loads this repository directly. Only rl_agent_config.json, model.safetensors, encoder/ and tokenizer/ are downloaded for that; the GGUF is not needed.
import json
import laya
agent = laya.load("tory-labs/vica-laya") # or a local clone path; device="cuda", "mps" or "cpu" to choose
state = json.load(open("example-state.json", encoding="utf-8"))
questions = json.load(open("topic-questions.json", encoding="utf-8"))
print(agent.system_one(state, questions))
The root checkpoint is the original float16 training checkpoint of the exported epoch (epoch 3 of round two, arm B), copied byte-for-byte from the training output (SHA-256 5ee42ec7604e40259cab503d749e207c70bd154a9b62c0a42f9e1e284bf6b0af). Verification on 2026-11-13: strict load into laya 0.4.22 with 321,908,995 parameters; on CPU the Python agent answers the bundled example with security 0.9006, as does the Q8 runtime with vica-laya.gguf (0.9006). These checks cover execution only, not accuracy. The float checkpoint of 2026.11.08.1 is at releases/2026.11.08.1/model.safetensors.
rl_agent_config.json keeps the driver's training block, including eligible_for_installation: false and selection: "pending quantized calibration". Those fields were written before epoch selection and before the release decision and are preserved as evidence; nothing in this repository reads them, and model-manifest.json records the actual release decision.
The previous release's root checkpoint was a Q8 reconstruction made with scripts/laya-tuning/gguf_to_checkpoint.py (dequantized GGUF tensors with the export-time LayerNorm folds inverted). It now lives at releases/2026.11.06.1/model.safetensors together with its reconstruction-report.json. The script still rebuilds a float checkpoint from any release GGUF when the upstream checkpoint is reachable. The 2026.11.06.1 GGUF is in repository history:
hf download tory-labs/vica-laya --revision 52de32568426ecfc969ad60de76df533917702a6 \
releases/2026.11.06.1/laya_multilingual_q8_0.gguf --local-dir local/models
python scripts/laya-tuning/gguf_to_checkpoint.py --gguf local/models/releases/2026.11.06.1/laya_multilingual_q8_0.gguf \
--base convaiinnovations/laya-multilingual --out local/runs/rebuilt-2026.11.06.1
Evaluation
All numbers are answers of the standalone Q8 runtime on the Mac tuning machine, from experiments/direct-judgment/results-round2.json and results-round3.json; evaluation.json holds the full counts. The direct-edit cases were written by Claude subagents on public permissive GitHub repositories and their labels were verified by other Claude subagents (opaque ids for check); they were not human-adjudicated. The reference is Jev (TypeSafe jev-1.14.0), asked once per row with the same observations. A row counts as right when Vica would act rightly on the answer: check at P(yes) ≥ 0.7, locate singles out a line at P ≥ 0.5, done skips the check reminder at P < 0.2.
| Job | Split | B3 | Jev | 2026.11.08.1 | 2026.11.06.1 |
|---|---|---|---|---|---|
done |
test, 98 rows | 98/98 | 98/98 | 31/98 | 27/98 |
done |
test3, 238 rows |
238/238 | 237/238 | — | — |
check |
test, 239 rows | 176/239 | 230/239 | 119/239 | 120/239 |
check |
test3, 472 rows |
361/472 | 434/472 | — | — |
locate |
test, 82 rows | 56/82 | 71/82 | 10/82 | 7/82 |
locate |
test3, 161 rows |
118/161 | 128/161 | — | — |
The test split (381 rows, four repositories) was read once, for the frozen B3, at the round-two final evaluation. test3 (871 rows, eight further repositories) was read once on 2026-11-13 during a cascade check that the user then ruled out of scope (models are not mixed); B3's own answers are shown. On the calibration rows that chose B3, the counts were done 97/101, check 174/234 and locate 51/74 (Jev 98, 213 and 61). The earlier releases answer almost every check yes (wrong edits 0–1 of 119 caught) and get most done rows wrong, which is what these jobs were trained to fix. check separates right from wrong edits better than it decides them (ranking AUC 0.83 on the test split, 0.88 on test3; Jev 0.97 on calibration), and locate ranks a usable line first in 29 of 44 present test rows but is sure enough to single it out in 19.
Existing judgment tasks, correct answers on the development files (all used in earlier development, so not unseen tests):
| Task (rows) | 2026.11.06.1 | 2026.11.08.1 | B3 |
|---|---|---|---|
| News topic, calibration (474) | 365 | 355 | 391 |
| News topic, known (130) | 95 | 97 | 107 |
| News topic, legacy (107) | 87 | 84 | 91 |
| News topic, fresh (48) | 38 | 36 | 37 |
| Workflow mode, retention (187) / calibration (85) | 172 / 82 | 171 / 82 | 175 / 80 |
| Next action, retention (78) / calibration (60) | 59 / 46 | 59 / 45 | 57 / 48 |
| Work/chat label, retention (78) / calibration (60) | 77 / 59 | 77 / 59 | 77 / 59 |
| Message kind, retention (48) / calibration (50) | 47 / 44 | 46 / 42 | 47 / 45 |
| Needs action, retention (48) / calibration (50) | 40 / 47 | 40 / 47 | 43 / 49 |
| Evidence status, retention (48) / calibration (48) | 44 / 46 | 44 / 45 | 45 / 47 |
| English department (12) / refund requested (12) | 10 / 10 | 10 / 10 | 11 / 10 |
On the 516 reserved topic rows, opened for the first time for B3 and now consumed, B3 answered 447/516 (86.6 %) against 403/516 for 2026.11.06.1; by class AI 199/225, development 56/70, other 136/155, security 56/66. 2026.11.08.1 was not measured on them. Warm p95 latency: 25.4 ms for B3, 25.7 ms for 2026.11.08.1, 25.9 ms for 2026.11.06.1 (same Mac, same binary).
Gates and limits:
- The gates frozen before the experiment rejected B3 (
checkandlocatebelow Jev − 0.02; a few small existing tasks over the 1 % allowance). On 2026-11-13, after the round-two calibration results were known, the user changed them: per job at Jev − 0.02, other jobs not below the installed release, and each existing task may lose at most max(1 % of rows, 2 rows) while the summed correct answers do not fall. B3 passed those on calibration and then once on the unseen test split and the reserved rows. Onlydoneis accepted; Vica keeps asking Jev forcheckandlocate. - The 85 % target is not met: news topic 391/474 (82.5 %) with security 38/55, and on the reserved rows development and security stay under 85 %.
- B3 was trained from
2026.11.06.1, not from2026.11.08.1, so the knowledge-graph and research-summary training of2026.11.08.1is not in it. Its accuracy on that private corpus was not measured; the corpus is on another machine. - Round three (Jev's probabilities as soft targets) produced no better candidate; see
results-round3.json.
Previous release — 2026.11.08.1
This historical section describes release 2026.11.08.1, now kept under releases/2026.11.08.1/. Its numbers do not describe 2026.11.13.1. Statements below about what remained unopened describe 2026-11-08; the 516 reserved rows were consumed on 2026-11-13.
Release 2026.11.08.1 was measured only on the held-out splits of the new
oracle-knowledge corpus, with the standalone Q8 runtime (Vulkan build on an NVIDIA
RTX 4060). The comparator is the previous release 2026.11.06.1, measured with the
same binary on the same rows. The held-out rows are Korean knowledge-graph and
research summaries, not news excerpts. Their labels are two independent Opus 5.5
passes under
experiments/oracle-knowledge/annotation-protocol.md
that agreed on all 363 rows (Cohen's kappa 1.0); they were not human-adjudicated.
The calibration split chose the exported epoch and fitted the temperature, so its
numbers are optimistic. The test split was never trained on or used for selection,
but it was measured for all four experiment arms, so it compared candidates rather
than confirming one. No new independent final evaluation was run.
| Split | Previous release 2026.11.06.1 | 2026.11.08.1 |
|---|---|---|
| Calibration (178 rows) | 135/178 (75.8%) | 155/178 (87.1%) |
| Test (185 rows) | 132/185 (71.4%) | 153/185 (82.7%) |
| Balanced accuracy, calibration / test | 0.746 / 0.785 | 0.870 / 0.848 |
| Confident errors, calibration / test | 17 / 20 | 7 / 4 |
| Log loss, calibration / test | 0.736 / 0.813 | 0.450 / 0.400 |
Recall by class on the test split, with the calibration split in parentheses:
| Class | Previous release | 2026.11.08.1 |
|---|---|---|
| AI | 82/92 (83/93) | 83/92 (85/93) |
| Development | 3/4 (2/4) | 3/4 (3/4) |
| Security | 5/5 (5/5) | 5/5 (5/5) |
| Business/other | 42/84 (45/76) | 62/84 (62/76) |
Incorrect assignments on the test split fell from 24 to 7 for security, from 14
to 11 for development and from 12 to 6 for AI, while incorrect other
assignments rose from 3 to 8. Row by row, 26 test rows became correct (20 other,
6 AI) and 5 became wrong, all of them AI rows now answered other; on
calibration 23 rows became correct and 3 became wrong with the same pattern. Only
5 security and 4 development rows are held out per split, so one row moves those
recalls by 20 to 25 points. Full counts and confusion matrices are in
evaluation.json.
Not measured for this release:
- The existing-judgment retention tasks (workflow mode, next action, work/chat label, message kind, needs action, evidence status, English department, refund requested) and their frozen one-percentage-point allowance. Their sets are on the Mac tuning machine. This stage updated the last four encoder layers with a learning rate ten times that of the previous stage and included no replay rows for these tasks, so regression is possible and unmeasured.
- The original news cohorts: 474 topic calibration, 48 primary-source
calibration, 130 known news and 107 legacy topic calibration rows. The 85 %
target is defined on them, so for
2026.11.08.1it is neither met nor failed; it is unverified. The previous release scored 365/474, 38/48, 95/130 and 87/107. - The final speed gate. The
decidecommand answered the bundled example in 281.1 ms (Vulkan build) and 237.5 ms (CUDA build) on this PC, first request each; no warm-daemon latency was measured. - The 516 reserved final rows (48 primary-source and 468 earlier) remain unopened.
Previous release — 2026.11.06.1
This historical section describes release 2026.11.06.1, SHA-256 beginning
b45ca2e8, now kept under releases/2026.11.06.1/. Its numbers do not describe
2026.11.08.1.
The following are existing Q8 runtime results from the
main-action contrast experiment.
The comparator is release 2026-11-05, SHA-256 beginning 4a00ab53, a previously
tuned model rather than the untouched upstream model. The 474-topic and
48-primary-source calibration cohorts had already been used in development. The
130-case news set is a consumed former final test. None is a new independent
final evaluation of that release, and labels are assistant-authored rather than
human-adjudicated.
| Cohort | Prior release | 2026.11.06.1 |
|---|---|---|
| Existing topic calibration | 333/474 (70.3%) | 365/474 (77.0%) |
| Primary-source calibration | 34/48 (70.8%) | 38/48 (79.2%) |
| Known, consumed news test | 93/130 (71.5%) | 95/130 (73.1%) |
| Legacy topic calibration | 86/107 (80.4%) | 87/107 (81.3%) |
Its recall by class was:
| Class | Existing topic calibration | Primary-source calibration | Known news set |
|---|---|---|---|
| AI | 175/214 (81.8%) | 10/12 (83.3%) | 14/22 (63.6%) |
| Development | 53/69 (76.8%) | 10/12 (83.3%) | 12/14 (85.7%) |
| Security | 33/55 (60.0%) | 7/12 (58.3%) | 29/38 (76.3%) |
| Business/other | 104/136 (76.5%) | 11/12 (91.7%) | 40/56 (71.4%) |
The frozen selection rejected that candidate for three reasons:
- Incorrect AI assignments increased from 45 to 54 on existing topic calibration.
- Development misses increased from 15 to 16; the gate required at least one fewer miss.
- Confident errors increased from 4 to 5 on primary-source calibration.
Confident errors on the 474-topic cohort fell from 65 to 39, but that improvement does not remove the separate failures. The 85% overall and per-class target was unmet. After receiving the failed-gate report, the user requested deployment of that model. This was an explicit release override, not a protocol pass or a change to the frozen criteria. The original experiment record remains rejected.
Existing judgment retention results were:
| Task | Prior release | 2026.11.06.1 |
|---|---|---|
| Workflow mode | 253/272 | 254/272 |
| Next action | 106/138 | 105/138 |
| Work/chat label | 136/138 | 136/138 |
| Message kind | 90/98 | 91/98 |
| Needs action | 85/98 | 87/98 |
| Evidence status | 88/96 | 90/96 |
| English department | 10/12 | 10/12 |
| Refund requested | 9/12 | 10/12 |
Next-action accuracy fell by 1/138 (0.72 percentage points), within the frozen one-percentage-point retention allowance; this is still an observed loss. These previously used sets do not establish independent generalization.
On one local Apple Silicon Q8 daemon pass over the 48 primary-source calibration inputs, the first request took 462.04 ms versus 536.42 ms for the prior release. The remaining 47 requests measured 24.20 ms median / 25.48 ms p95, versus 25.01 ms / 26.65 ms. These are calibration observations, not the final speed gate, end-to-end application latency or a guarantee on other hardware. The separate 48-case final and earlier 468 reserved final cases remained unopened; no new final evaluation or final speed gate was run for that release. Jev parity has not been established.
Prior release evaluation — 2026-11-05
This historical section describes only SHA-256
4a00ab531b5a6c7a1ef009c9617137a4a76adfd8a5877cd35b50928030169bb2.
Its 93/130 result, language diagnostic and latency measurement do not describe
release 2026.11.06.1 or 2026.11.08.1.
The model was selected using calibration data before one source-separated final evaluation of 130 public-news excerpts. Reference labels were authored by an independent assistant and were not human-adjudicated. The test measures this specific classification task, not general Korean fluency or all computer-use decisions. It is now a consumed test set.
| News classification result | Value |
|---|---|
| Overall accuracy | 93/130 (71.5%) |
| Mean per-class recall | 71.1% |
| AI recall | 13/22 (59.1%) |
| Development recall | 11/14 (78.6%) |
| Business/other recall | 41/56 (73.2%) |
| Security recall | 28/38 (73.7%) |
The preceding local tuned version scored 71/130 (54.6%) on the same final test. That comparison is not against the untouched upstream model.
Existing judgment regression checks were also retained:
| Task | Correct / cases |
|---|---|
| Workflow mode | 253/272 |
| Next action | 106/138 |
| Work/chat label | 136/138 |
| Message kind | 90/98 |
| Needs action | 85/98 |
| Evidence status | 88/96 |
These regression sets were previously used during development and do not constitute independent final tests. A separate paired Korean/English diagnostic scored 26/32 in each language; it is not a general language-proficiency benchmark.
On the local Apple Silicon machine, a fixed calibration workload of 20 inputs repeated three times measured warm daemon round-trip latency of 19.51 ms median / 27.52 ms p95. These are local runtime measurements, not end-to-end application latency or a guarantee on other hardware.
Later experimental models reached 83.2% on a calibration set and 78.5% on a previously consumed news set, but failed the 85% acceptance target. Those results describe other experimental weights, not any release summarized above.
Training and provenance
- Upstream:
convaiinnovations/laya-multilingual, revision1720e3e3357cfe1e281542e223f8273b0890ca34. - Release
2026.11.13.1(B3) starts fromtraining-checkpoints/mix25-epoch-1. Four epochs tuned the last 12 encoder layers and the decision head with encoder learning rate2e-5and head learning rate5e-5on Apple silicon (MPS) with the bounded-memory driver; epoch 3 was exported. Training data: the earlier 3,400 train rows plus 2,936 verified direct-edit rows (check1,606,done790,locate540) from public permissive repositories listed inexperiments/direct-judgment/sources.lock.json, 10,175 items in all; the direct-edit jobs take 50 % of the loss. Selection read calibration rows only. Plan, scripts and records:experiments/direct-judgment/. - Previous release
2026.11.08.1(nowreleases/2026.11.08.1/) starts fromtraining-checkpoints/mix25-epoch-1, the original float checkpoint of2026.11.06.1. Six epochs tuned the last four encoder layers and the decision head with encoder learning rate1e-5, head learning rate3e-5and seed20261105; epoch 3 was exported. Training ran 344 seconds on an NVIDIA RTX 4060 (CUDA). - The driver was
experiments/hermes-graph/train-weighted.pyoverscripts/laya-tuning/train.py: soft cross-entropy with per-row loss weights (security rows ×3, the 695 rows flagged ambiguous ×0.5, normalized to a mean of 1.0). Weights apply to training only; calibration metrics are unweighted. - The training set contained 1,833 rows of the private oracle-knowledge corpus: 1,374 knowledge-graph summaries, 220 repository commit messages, 119 repository documentation sections and 120 game-research summaries (labels: AI 743, other 556, development 434, security 100). The 178 calibration and 185 test rows come from the knowledge graph and game research only; repository rows are training-only. Splits are pinned by source family with seed
20261108; 32 unusable and 18 label-conflicting rows were excluded. Every row carries only the four-way topic question; no replay rows for the other judgment tasks were included. - Labels are Opus 5.5 annotations under
experiments/oracle-knowledge/annotation-protocol.md: one pass over 2,246 rows and a second independent pass over the 363 held-out rows (363/363 agreement). No human adjudication. - Temperature
[0.6909, 1.0, 1.0]was fitted on the 178 calibration rows; the expected calibration error could not be computed because its only bucket was empty. - Four arms were trained and measured; see
experiments/oracle-knowledge/results-20261108.jsonand the tuning history below. No frozen selection protocol was run for this stage. - Q8_0 export:
monatis/ggmlc, commit5490a13661d95add025ec1b111adbc88234a9b4f(v0.9.7). - Float checkpoint (
model.safetensors): the original epoch-3 training checkpoint, copied byte-for-byte; not reconstructed from the GGUF. - Previous release
2026.11.06.1(nowreleases/2026.11.06.1/) started from the floating-pointclean-epoch-3checkpoint, which had failed its own selection gates. One shared epoch tuned the last four encoder layers and decision head with encoder learning rate1e-6, head learning rate2e-6and seed20261105, on 3,400 rows (3,144 retained rows plus 256 synthetic main-action contrasts in 64 quartets) with 827 calibration rows. Its weights are0.75 * clean-epoch-3 + 0.25 * contrast-epoch-1, then Q8_0 export; four predefined mixtures were compared, all failed selection, and the user authorized the 25% mixture separately. Its published root float checkpoint was rebuilt from the GGUF; see itsreconstruction-report.json. - Historical baseline
2026-11-05used 1,565 training rows and 415 calibration rows in its last accepted stage; epoch 3 was selected. It tuned 22 encoder layers with encoder learning rate3e-6, head learning rate8e-6, and seed20261105. - Raw training/evaluation examples, annotations, local logs, and the separate Vica routing model are not included.
Intended use and limits
The model classifies supplied text with typed questions. It does not read screenshot pixels or infer screen coordinates. A computer-use observer must supply text and current screen bounds; the host agent handles labels, colors, placement, scrolling, and stale observations.
Ambiguous articles and the boundary between AI applications, general development, technical releases, and business announcements remain error sources. Confidence scores can be high on wrong answers. Release 2026.11.13.1 answers check and locate well below the reference model, so Vica uses it only for done; it does not include the knowledge-summary training of 2026.11.08.1. Validate the model on the specific tasks and inputs where it will be used.
Local development
The self-retention tuning plan (Korean)
was written against 2026.11.06.1 and carries a note about the replacement. It
compares hard-target training with preservation of the existing Laya's decision
distributions. This is a plan; its new training and evaluation have not run.
Model development is independent of Vica and any other consuming application. The training, Q8 export and evaluation entrypoints accept explicit input paths; they do not need an application checkout, application settings or an installed application model. Publishing a model and updating an application are separate tasks. Application integration requires its own explicit request.
Keep the development environment in this checkout:
| Location | Purpose |
|---|---|
.venv/ |
Python and pinned training dependencies |
local/models/ |
Downloaded model weights and checkpoint metadata |
local/data/ |
Private training and evaluation inputs |
local/runs/ |
New experiment outputs and intermediate checkpoints |
local/bin/ |
Standalone laya runtime |
local/toolchains/ggmlc/ |
Pinned Q8 compiler source |
.venv/ and local/ are ignored by Git. Published checkpoint weights are
tracked through Git LFS. A checkout made without Git LFS contains pointers in
place of its large files; the following setup downloads actual model files into
the independent development directory without changing those tracked pointers:
uv venv --python 3.12 .venv
uv pip sync --python .venv/bin/python scripts/laya-tuning/requirements.txt
source .venv/bin/activate
LAYA_REVISION="$(git rev-parse HEAD)"
hf download tory-labs/vica-laya --revision "$LAYA_REVISION" \
--include 'training-checkpoints/mix25-epoch-1/**' \
--include 'model-manifest.json' --include 'downloads/2026.11.08.1/vica-laya-macos-arm64.tar.gz' \
--local-dir local/models
# optional: the previous release's float checkpoint and records
hf download tory-labs/vica-laya --revision "$LAYA_REVISION" \
--include 'releases/2026.11.06.1/**' --local-dir local/models
Unpack the download for your platform and use its vica-laya program (for example at
local/bin/vica-laya) as the standalone runtime. For Q8
export, prepare the compiler independently:
git clone https://github.com/monatis/ggmlc.git local/toolchains/ggmlc
git -C local/toolchains/ggmlc checkout --detach 5490a13661d95add025ec1b111adbc88234a9b4f
scripts/laya-tuning/train.py fine-tunes from an explicitly supplied float
checkpoint. Its --device auto uses the pinned Laya package's device resolver;
--device can select a backend explicitly. Apple Silicon MacBooks use PyTorch
MPS for training and Metal for the standalone Q8 runtime, not NVIDIA CUDA.
The same training entrypoint accepts --device cuda on a machine with a
compatible NVIDIA GPU and CUDA-enabled PyTorch. No separate Vica source is needed.
Rows are one JSON object per line with id, family, split (train or
calibration), state, questions and targets; an optional per-row
loss_weight is read by experiments/hermes-graph/train-weighted.py, which wraps
the same entrypoint and produced release 2026.11.08.1. export.py turns a result
into the Q8_0 GGUF with a ggmlc v0.9.7 source checkout, and evaluate.py measures
a GGUF with the laya runtime binary:
python scripts/laya-tuning/train.py --data local/data/rows.jsonl \
--base local/models/training-checkpoints/mix25-epoch-1 --out local/runs/candidate \
--device mps \
--epochs 1 --train-last-layers 4 --encoder-lr 1e-6 --head-lr 2e-6
python scripts/laya-tuning/export.py --source local/toolchains/ggmlc \
--checkpoint local/runs/candidate --out local/runs/candidate/laya_multilingual_q8_0.gguf
The command above explicitly selects the MacBook GPU. On NVIDIA CUDA hardware,
replace --device mps with --device cuda; use --device cpu for CPU execution.
CPU, MPS and Metal example runs have been verified on an Apple M5 Pro. The
2026.11.08.1 training, Q8 export and evaluation ran with CUDA on a Windows PC
with an RTX 4060 (8 GB); both the Vulkan and the CUDA build of the standalone
runtime answered the bundled example there.
The experiment directories preserve past workflows, including their original
private paths. Those archives and the historical installation/latency helpers are
not the setup for new development. Optional application checks in
integrations/vica are outside the model workflow. Preserve completed experiment
outputs and frozen labels, splits and criteria; only a calibration-selected,
frozen candidate may open its final test. A public clone does not include private
training datasets or reserved test data.
Tuning history — 2026-11-08
The experiments/oracle-knowledge workflow built a new private corpus from the
user's ssh oracle server and local repositories: Korean knowledge-graph summaries
(sutory-wiki), two game-research corpora, and, for training only, commit
messages and documentation sections of local repositories. A read-only snapshot
was taken; private mail and internal incidents were not imported. Opus 5.5
labelled 2,246 rows in a first pass under the written annotation protocol (1,684
knowledge-graph, 362 repository, 150 game-research and 50 pilot rows) and the 363
held-out rows a second time; the passes agreed on every row (Cohen's kappa 1.0),
which is same-family agreement rather than human adjudication. Family-pinned
splits with seed 20261108 gave 1,833 training, 178 calibration and 185 test
rows after excluding 32 unusable and 18 label-conflicting rows; 695 training
rows flagged ambiguous received loss weight 0.5 and security rows weight 3.
On these splits the previous release 2026.11.06.1 scored 135/178 on
calibration and 132/185 on test. Its main failure was business/other rows
answered as security, development or AI: other recall 45/76 and 42/84.
Four arms, all from mix25-epoch-1 with seed 20261105 and six epochs, are
recorded in
results-20261108.json.
The two head-only arms could not move the boundary: arm A (duplicated rows,
pass-1 labels) never improved on the base checkpoint's 136/178 float calibration
answers, and arm B (loss-weighted rows) reached 137/178 at epoch 6; their Q8
test results were 133/185 and 135/185. The two arms that also tuned the last four
encoder layers did: arm C (duplicated rows) reached 148/178 on Q8 calibration at
its selected epoch 4 with balanced accuracy 0.9115 and 146/185 on test; arm D
(loss-weighted rows) reached 155/178 at epoch 3 with balanced accuracy 0.8699 and
153/185 on test. Arm D epoch 3 was taken as the candidate for the most correct
calibration answers with AI recall 85/93 and security 5/5. Arm C epoch 4's
higher balanced accuracy came from development scoring 4/4 on a four-row class
while its AI recall fell to 76/93. No frozen selection protocol existed for this
stage, and the test split was measured for every arm, so it did not stay closed
for one frozen candidate.
The candidate record stated that retention and the original news cohorts had
not been checked on this PC. After that report the user ordered deployment. The
candidate was published as 2026.11.08.1, the release was recorded as
user_authorized_without_protocol_checks, and 2026.11.06.1 moved unchanged
to releases/2026.11.06.1/ as the previous model. The 516 reserved final rows
remain unopened, and no application was changed.
Tuning history — 2026-11-06
The following records preserve the experiment outcomes before the explicit
2026.11.06.1 release override. References to the deployed or distributed model
in this history mean the 2026-11-05 baseline, SHA-256 beginning 4a00ab53.
Statements that weights were unchanged or unpublished describe the end of each
experiment; the later release decisions above do not change their verdicts.
Release 2026.11.06.1 itself is now the previous model.
Aggregate experiment results preserve four attempts, their selection criteria, model hashes and measured failures. None of these attempts replaced the distributed model.
The boundary-diverse candidate was selected from four Q8 exports before a new 160-example test. Compared with the distributed model on that same test, AI recall rose from 65% to 70%, business/other recall fell from 80% to 72.5%, and overall accuracy stayed at 114/160 (71.25%). Warm p95 latency was 27.78 ms versus 27.80 ms. The business/other loss exceeded the declared three-percentage-point limit, so the candidate was rejected. These are assistant-labeled examples, with 40 cases per class; they do not establish production accuracy or achievement of the 85% target.
The frozen verifier also reported a balanced-accuracy loss of about 1e-16 from floating-point rounding. With equally sized classes and identical total correct counts, balanced accuracy was mathematically unchanged. This numerical issue is recorded for a future verifier correction; it does not remove the separate business/other regression or change the preserved rejection.
The 275- and 160-example final tests are now consumed and must not be reused for training or candidate selection. The source archive requires private local inputs to reproduce full experiments. Publishing this history does not deploy a model.
A subsequent Jev-assisted training comparison used 480 training-only teacher requests and compared four Q8 candidates. On 263 calibration examples, the candidate with the most correct answers improved from 208/263 to 224/263, and confident mistakes decreased from 21 to 16. However, AI misses increased from 19 to 20, business/other misses increased from 9 to 10, and next-action retention fell from 106/138 to 104/138. Security misses fell from 11 to 2 while incorrect security assignments increased from 4 to 9. All four candidates failed selection; the fresh synthetic final test remained unused. These calibration results are not a Jev parity claim or results for the distributed weights. The new selector uses a 1e-12 floating-point tolerance; earlier frozen verdicts remain intact.
The local experiments/hermes-graph workflow uses a read-only knowledge-graph
snapshot with 1,851 documents, 16,773 memory units and 406,019 stored relations,
plus source notes. Relations identify related examples; they are not truth labels.
After source grouping, visible-text annotation and exclusions, it adds 1,009
training examples, 276 calibration examples and 308 reserved final examples.
The source material and annotations remain private and have not been approved
for publication or independently adjudicated by people.
This experiment removes 321 repeated training rows. Each of the resulting 3,067 training examples is visited once per epoch, with half of the total loss allocated equally among the four topic classes and half to six existing judgment tasks. Multiple views of one source share its influence. Related, differently labelled training sources receive a small boundary-learning weight increase. Calibration uses 812 unweighted examples. New graph labels are predominantly AI/business; only 20 development and 41 security training examples survived the quality checks. Those limits matter when interpreting any later improvement.
The frozen comparison uses two epochs each of head-only and last-12-layer tuning, with four Q8 exports. Selection checks both missed examples and incorrect category assignments, alongside confident errors and retained judgments. Only a selected candidate can enter the separate graph and reserved synthetic final tests. This workflow neither installs weights nor publishes private data automatically.
All four candidates failed the frozen selection criteria. On the same 459 topic-calibration examples, the candidate with the most correct answers rose from 321/459 (69.9%) to 342/459 (74.5%); confident mistakes fell from 64 to 36. AI recall rose from 146/214 to 167/214, but incorrect AI assignments increased from 44 to 51. Development recall fell from 47/62 to 45/62 and security recall from 26/47 to 25/47. The candidate also lost one AI answer on the separate legacy calibration cohort. Neither the graph final test nor the reserved synthetic final test was run, and the final latency gate remains unmeasured. These results do not establish Jev parity or describe the currently distributed weights.
The follow-up experiments/hermes-boundary workflow adds 140 clear training
examples: 102 development and 38 security. They come from unused technical
records and independently reviewed summaries of official documentation. Eight
ambiguous training examples were excluded. New calibration contains seven
development and eight security examples from separate public sources; private
operational records are training-only.
Nine pairs of short and long records described the same incident despite having different source identifiers. Their learning weight is shared, with original text, labels and split membership preserved. The resulting workload has 3,207 training and 827 calibration rows. Two starting checkpoints—the deployed version and the previous rejected graph candidate—receive identical data and training settings. Starting from a rejected checkpoint does not approve it for installation. The shared class/error gates and reserved final tests remain unchanged.
All four follow-up candidates failed selection. The continued-training epoch-2 candidate had the most correct answers: 369/474 (77.8%), versus 333/474 (70.3%) for the deployed model. AI recall increased from 146/214 to 174/214, development from 54/69 to 55/69, and security from 31/55 to 33/55. Its remaining relative-gate failure was incorrect AI assignments, which increased from 45 to 49. On the unchanged 459-example portion, it scored 356/459 versus 342/459 for the previous rejected graph candidate; the 15 added primary-source cases scored 13/15 versus 12/15 for the deployed model.
A separate input audit found references to the same claimed incident in five prior training views and seven calibration views, despite different source URLs. Source grouping therefore did not establish event-level independence. Frozen labels, splits and verdicts were preserved; these calibration scores are not independent generalization evidence. The 468 reserved final examples and final latency gate remain unused, and the deployed weights remain unchanged.
The next experiments/hermes-event experiment removes 63 training examples after
input-only event, label and source-quality screening. Nine confirmed event groups
crossed training/calibration boundaries; several other summaries explicitly said
their source could not be retrieved and the details were reconstructed. Whole
training families are excluded conservatively, while earlier data, labels and
verdicts remain intact. Four retained views of two repeated training events share
learning weight. This leaves 3,144 training rows and the unchanged 827 calibration
rows. The audit is not exhaustive semantic deduplication or independent human
validation of all labels.
Training restarts from the accepted checkpoint before graph tuning and exports four fixed epochs. A separate evaluation-only corpus contains 96 short Korean summaries from verified public primary sources. Event splits were frozen before a different assistant annotated the visible text: 48 calibration and 48 final examples, each with 12 per category. The new calibration must preserve each class's missed-example and incorrect-assignment counts in addition to all legacy gates. Only a selected candidate can open the new final test and run its latency gate. The earlier 468 reserved final examples remain unused in this experiment.
These new examples deliberately balance historical research, technical documentation, vulnerabilities and corporate transactions; most business examples are acquisitions. Results describe this small curated sample and do not establish production feed or messenger accuracy. The single-seed experiment also cannot isolate the effect of data cleanup from the changed training schedule.
The original four-epoch run was interrupted during epoch four when its measured process footprint reached 35.8 GB on a 24 GiB machine. Three completed checkpoints were preserved. A separately recorded recovery experiment adds one epoch from the third checkpoint, with a new optimizer and scheduler; it is not an exact resume of the original fourth epoch. Training uses microbatches of one with eight accumulation steps, and calibration batches of one. MPS allocations are limited to half the device's recommended memory, while a macOS process-footprint monitor stops the owned process above 12 GiB. Actual over-limit rejection and under-limit completion were checked before recovery. The bounded continuation completed in 721.5 seconds. Its maximum sampled process footprint was 7.50 GiB, and reported MPS tensor memory returned to zero at shutdown. The cause of the original growth has not been isolated; this result verifies the bounded run, not a general leak fix.
All four recovery candidates failed selection. The highest total was 368/474 (77.6%), versus 333/474 for the deployed model and 369/474 for the previous rejected boundary candidate. Compared with that previous candidate, AI correct answers rose from 174 to 175, development fell from 55 to 52, security stayed at 33, and other rose from 107 to 108. Incorrect AI assignments increased from 49 to 50, versus 45 for the deployed model. The final continuation also lost too many next-action judgments under the existing retention limit.
All four candidates scored 38/48 on fresh calibration, versus 34/48 for the deployed model. The highest-total candidate scored AI 10/12, development 10/12, security 7/12 and other 11/12. Its confident errors rose from four to six, so the new-cohort gate also failed. On these same short inputs, warm median latency was 24.1 ms versus 24.4 ms, and warm p95 was 25.0 ms versus 25.2 ms. These are one-pass calibration measurements, not a completed final latency gate. The new 48-example final and earlier 468 reserved examples remain unused. No model was installed or published; 85% per-class accuracy and Jev parity remain unachieved.
The following main-action contrast experiment adds 256 independently assistant-labelled synthetic examples from training-only concepts. One bounded epoch updates the last four encoder layers and decision head. Four predefined mixtures of the starting and newly trained weights export to Q8, with no extra model at inference. All four failed selection: the highest total was 365/474 (77.0%), below the preceding candidate's 368/474 (77.6%). AI correct answers stayed at 175/214, development rose from 52 to 53/69, and security stayed at 33/55, while incorrect AI assignments increased from 50 to 54. The two highest-total mixtures matched the starting checkpoint's category counts.
Each candidate scored 38/48 on primary-source calibration but retained five
confident errors versus the deployed model's four. The 25% mixture's warm median
and p95 were 24.2 and 25.5 ms versus 25.0 and 26.7 ms for deployed; these are
single-pass calibration observations, not final speed-gate results. Training took
307.7 seconds with a maximum sampled process footprint of 5.89 GiB. A separate
receipt-only recovery handles unavailable empty-sample ECE without changing
weights, temperatures, data or gates. At the end of this experiment no candidate
had been applied or published. The user subsequently authorized mix25-epoch-1
as release 2026.11.06.1 despite the three preserved failures. All 516 reserved
final examples remained unopened.
License and attribution
Apache License 2.0, following the upstream model license. See LICENSE and NOTICE. Vica Laya is a modified derivative, not an official upstream Laya release.
- Downloads last month
- 706
Model tree for tory-labs/vica-laya
Base model
convaiinnovations/laya-multilingual