ntc-ai/evidence-dial-aligntune

Evidence Dial — original64-case AlignTune supervised LoRA

A rank8 supervised LoRA on64 fictional policy QA cases, using the real AlignTune RaftTrainer supervised path. This is the supervised comparison for Particle Sliders' Evidence Dial project, Track1 problem3, Lexsi hackathon2026. It is not a ParticleGAN adapter.

This is the original64-case control. The separately declared Jev-curated retry uses a new48-case matched LoRA; no result from this model is relabelled as that control. The original adapter weight bytes remain unchanged.

Development:128/128 strict successes versus identical-prompt OFF86/128. Paired gain0.328125; policy-family bootstrap95% interval[0.2734375,0.3828125]. Five development OFF misses are format-only. Complete original comparisons:

Frozen evaluation	OFF	Original hinted particle1	This LoRA
Policies256	169	184	256
FaithEval900	323	337	375
Cue audit80	53	43	36
Balanced stress768	353	341	492

The100% policy score was independently reproduced from raw exact outputs and native tokens. Exact prompts/families are split-disjoint and gold metadata does not enter the prompt, but all16 question frames are shared and DocumentB perfectly predictsCONFLICT in the original corpus. This narrow test does not establish perfect grounding. The cue-controlled tests break that shortcut; LoRA regresses below OFF on the80-case cue audit.

On the900-case external FaithEval subset, LoRA beats the selected original particle overall by4.22pp, paired source-question95% interval[1.74,6.69]. It loses specifically on missing-evidence cases166/300 versus196/300, while winning counterfactual180/300 versus130/300 and inconsistent29/300 versus11/300. Stratum comparisons are descriptive with unadjusted intervals. FaithEval lacks verified matched answerable controls; this is a subset, not a leaderboard submission. On balanced stress LoRA beats particle by19.66pp [15.36,23.96]. Strict formatting differences count; no general reliability claim follows.

Inspect retained answers and the audit.

Base: CohereLabs/tiny-aya-global, revision af89d219b53ed9b13b8a4645f9c8028973510324. Attribution: Cohere Labs. The base and this derived adapter use CC-BY-NC-4.0. Non-commercial use only. No base weights or tokenizer files are included; request base access with your own Hugging Face account.

The only training targets are16 answerable,16 missing-evidence,16 conflicting and16 altered-fact examples from the frozen64-case training split. Families are disjoint from128 development and256 primary holdout cases, but templates are shared. The4-epoch exposure budget is128 updates×batch2. LoRAr8/alpha8, q/k/v/o projections, AdamW0.0002 constant, seed20261110, completion-only loss, no packing/truncation. Gold completions end with native end-response261001 and EOS3. No weight EMA; final current weights are used. AlignTune citation and document-ranking losses are disabled; no placeholder mechanism is credited.

Code and protocol · Training data

Use Transformers4.57.6 and PEFT0.18.0, then load the approved base at the revision above and attach this adapter with PeftModel.from_pretrained. Apply the exact evidence_dial.protocol.prompt and native chat template. Greedy generation, EOS3 and64 new tokens match the comparison; do not substitute the tokenizer's different default EOS. Reproduction commands are in the linked repository. The publication copy changes only the base reference in adapter configuration from a local cache path to its public model ID/revision. Weight bytes are unchanged, as recorded in training_provenance.json.

New hackathon work: dataset, Cohere2 integration, trainers and evaluation. Particle Sliders core is a pre-existing library. This adapter is a research prototype trained on English templates, with no production reliability claim.

The final checkpoint passed its bounded native-text check. Four retained development generations were reproduced token-for-token from this publication copy. All128 LoRA/OFF development native input sequences match exactly. The first checker incorrectly flagged a legal UNKNOWN label; that original record is retained with an explicit scope audit and a separate qualified replacement check. Qualification covers a bounded set of native text forms, not all outputs.