--- library_name: peft base_model: - Qwen/Qwen3.8-27B - google/gemma-4-31B-it - google/gemma-4-26B-A4B-it base_model_relation: adapter license: gpl-3.0 license_link: LICENSE pipeline_tag: text-generation tags: - lora - subject-extraction - nostr - research - oracle-research --- # Oracle subject-extraction adapters All 17 evaluated LoRA checkpoints from [Oracle Research](https://github.com/kernelkind/oracle-research)'s subject fine-tuning research. They identify the subjects discussed in individual Nostr posts using the **ASv6.100** instruction and return a numbered local subject hierarchy in JSON. This is one stage of a research prototype for helping people choose and discover content through a hierarchy. Building a hierarchy across the corpus is a separate stage. ## Choose a model family and checkpoint Each checkpoint must be used with its own family’s pinned base model and tokenizer. Follow [USAGE.md](USAGE.md) to download and load one checkpoint; you do not need all 17 for inference. | Folder | Base model | Checkpoints | | --- | --- | ---: | | [qwen3.8-27b](qwen3.8-27b/checkpoints.json) | `Qwen/Qwen3.8-27B` | 7 | | [gemma-4-31b-it](gemma-4-31b-it/checkpoints.json) | `google/gemma-4-31B-it` | 5 | | [gemma-4-26b-a4b-it](gemma-4-26b-a4b-it/checkpoints.json) | `google/gemma-4-26B-A4B-it` | 5 | Each family has `model.json` with its exact base revision and decoding settings, `checkpoints.json` with checkpoint hashes/training counters, `training.json` with training settings/input identities, and `evaluation.json` with saved results. Every checkpoint folder contains the original `adapter_config.json` and `adapter_model.safetensors` pair. Weights are FP32 PEFT LoRA; base weights and optimizer state remain separate. Checkpoint numbers count **training-note uses**, not optimizer steps. All three families have checkpoints at 200, 400, 600, 800 and 1000 notes. Qwen also has checkpoints 2000 and 3000, which repeat the same 1,000 training notes rather than introducing new notes. Publishing all checkpoints preserves the research; it does not designate the latest checkpoint as best or production-approved. The original adapter configurations have `revision: null`. Load the base and tokenizer revisions recorded in the corresponding `model.json` explicitly. [files.sha256](files.sha256) binds every selected file to its SHA-256. ## Training The models share 1,000 training, 100 validation and 200 test notes, with separation across author/content groups. Targets use the full ASv6.100 subject instruction. First-pass training consumes each note once in 200-note blocks, with seed 17 and eight prompt–answer examples per optimizer update. Training uses an unquantized BF16 base with frozen original weights and FP32 LoRA parameters: ordinary rank 16 / alpha 32, no LoRA dropout, frozen biases, AdamW without weight decay, gradient clipping 1, and constant learning rate 0.0001 without warmup. The Gemma MoE adapter additionally adapts packed routed experts at rank 2 / alpha 32; its router remains frozen. Loss averages teacher-answer and native turn-ending tokens, excluding prompts and padding. The family metadata preserves the exact target modules and checkpoint counters. The shared [ASv6.100.md](ASv6.100.md) is the original instruction file. Use its text with trailing whitespace removed, as in training. Recorded generation uses native chat formatting, thinking off, greedy decoding, a 20,480-token context and up to 10,240 new tokens. Do not silently truncate inputs to fit. ## Recorded results These are saved measurements on the **top-level subjects (L1)** of the same **200 previously examined test notes**. Correctness measures whether returned roots are supported by a post. Coverage measures recovery of teacher root meaning groups. Percentages average per-note fractions; invalid outputs score zero. | Version | L1 correctness | All-root coverage | Required-root coverage | Invalid / 200 | | --- | ---: | ---: | ---: | ---: | | Qwen untuned | 66.96% | 42.12% | 58.13% | 43 | | Qwen checkpoint-1000 | 96.08% | 61.63% | 79.39% | 2 | | Qwen checkpoint-3000 | 97.51% | 57.07% | 76.99% | 0 | | Gemma 31B untuned | 93.69% | 46.21% | 65.22% | 3 | | Gemma 31B checkpoint-1000 | 96.55% | 55.03% | 70.49% | 2 | | Gemma 26B-A4B untuned | 88.04% | 42.90% | 55.97% | 5 | | Gemma 26B-A4B checkpoint-1000 | 89.81% | 55.75% | 73.64% | 11 | The reference contains 1,293 root meaning groups, 512 marked required. Thirty-four notes have no required roots. On those notes, valid answers receive 100% required-root coverage; invalid answers still receive zero. Each family’s `evaluation.json` also records the required-root metric restricted to the 166 notes with at least one required root. This is a retrospective slice of existing automated semantic grades, with no new inference or judgments. Only equivalent returned L1 meanings count toward coverage; a matching deeper child does not. It does not evaluate the quality of a corpus-wide browsing hierarchy. Qwen checkpoint 3000 was selected using validation full-hierarchy scores. In this retrospective L1 test slice, checkpoint 1000 recovered more reference roots and checkpoint 3000 had higher correctness; the slice does not revise the validation selection. Checkpoint 2000’s test answers remain sealed. Earlier first-pass checkpoints have validation evidence, not the final checkpoint’s test scores. ## Use and limits See [USAGE.md](USAGE.md) for the standalone loading example. The original experiments used 96 GB GPUs. Smaller-memory and quantized inference have not been validated; a fresh GPU execution of this standalone example is untested. The adapter download excludes base weights, host loading memory and generation cache. This is a small pilot with teacher-generated references and automated judgments. Outputs can be malformed, miss subjects or include unsupported subjects. These results do not establish generalization outside the studied data. Keep raw answers when assessing failures. The prepared training data and recorded experiment evidence are in the [research dataset](https://huggingface.co/datasets/KernelKind/oracle-research). The [reproduction guide](https://github.com/kernelkind/oracle-research/blob/main/docs/experiments/reproduce.md) covers the saved evidence, adapter loading and training. Prepared-file hashes remain in the family metadata. ## License and attribution Our adapter contributions, ASv6.100 instruction and original accompanying project material use **GPL-3.0-or-later**; see [LICENSE](LICENSE). Copyright 2026 kernelkind. The base models retain **Apache-2.0** licensing; see [NOTICE.md](NOTICE.md), the [Apache license](LICENSES/Apache-2.0.txt) and the original [Qwen license and copyright notice](LICENSES/Qwen-LICENSE). These files contain trained LoRA additions, rather than merged upstream base weights.