README.md

---
library_name: peft
base_model:
- Qwen/Qwen3.8-27B
- google/gemma-4-31B-it
- google/gemma-4-26B-A4B-it
base_model_relation: adapter
license: gpl-3.0
license_link: LICENSE
pipeline_tag: text-generation
tags:
- lora
- subject-extraction
- nostr
- research
- oracle-research
---

# Oracle subject-extraction adapters

All 17 evaluated LoRA checkpoints from
[Oracle Research](https://github.com/kernelkind/oracle-research)'s subject fine-tuning research.
They identify the subjects discussed in individual Nostr posts using the
**ASv6.100** instruction and return a numbered local subject hierarchy in JSON.
This is one stage of a research prototype for helping people choose and discover
content through a hierarchy. Building a hierarchy across the corpus is a
separate stage.

## Choose a model family and checkpoint

Each checkpoint must be used with its own family’s pinned base model and
tokenizer. Follow [USAGE.md](USAGE.md) to download and load one checkpoint;
you do not need all 17 for inference.

| Folder | Base model | Checkpoints |
| --- | --- | ---: |
| [qwen3.8-27b](qwen3.8-27b/checkpoints.json) | `Qwen/Qwen3.8-27B` | 7 |
| [gemma-4-31b-it](gemma-4-31b-it/checkpoints.json) | `google/gemma-4-31B-it` | 5 |
| [gemma-4-26b-a4b-it](gemma-4-26b-a4b-it/checkpoints.json) | `google/gemma-4-26B-A4B-it` | 5 |

Each family has `model.json` with its exact base revision and decoding settings,
`checkpoints.json` with checkpoint hashes/training counters, `training.json` with
training settings/input identities, and `evaluation.json` with saved results.
Every checkpoint folder contains the original `adapter_config.json` and
`adapter_model.safetensors` pair. Weights are FP32 PEFT LoRA; base weights and
optimizer state remain separate.

Checkpoint numbers count **training-note uses**, not optimizer steps. All three
families have checkpoints at 200, 400, 600, 800 and 1000 notes. Qwen also has
checkpoints 2000 and 3000, which repeat the same 1,000 training notes rather than
introducing new notes. Publishing all checkpoints preserves the research;
it does not designate the latest checkpoint as best or production-approved.

The original adapter configurations have `revision: null`. Load the base and
tokenizer revisions recorded in the corresponding `model.json` explicitly.
[files.sha256](files.sha256) binds every selected file to its SHA-256.

## Training

The models share 1,000 training, 100 validation and 200 test notes, with
separation across author/content groups. Targets use the full ASv6.100 subject
instruction. First-pass training consumes each note once in 200-note blocks,
with seed 17 and eight prompt–answer examples per optimizer update.

Training uses an unquantized BF16 base with frozen original weights and FP32
LoRA parameters: ordinary rank 16 / alpha 32, no LoRA dropout, frozen biases,
AdamW without weight decay, gradient clipping 1, and constant learning rate
0.0001 without warmup. The Gemma MoE adapter additionally adapts packed routed
experts at rank 2 / alpha 32; its router remains frozen. Loss averages
teacher-answer and native turn-ending tokens, excluding prompts and padding.
The family metadata preserves the exact target modules and checkpoint counters.

The shared [ASv6.100.md](ASv6.100.md) is the original instruction file. Use its
text with trailing whitespace removed, as in training. Recorded generation uses
native chat formatting, thinking off, greedy decoding, a 20,480-token context
and up to 10,240 new tokens. Do not silently truncate inputs to fit.

## Recorded results

These are saved measurements on the **top-level subjects (L1)** of the same
**200 previously examined test notes**. Correctness measures whether returned
roots are supported by a post. Coverage measures recovery of teacher root
meaning groups. Percentages average per-note fractions; invalid outputs score
zero.

| Version | L1 correctness | All-root coverage | Required-root coverage | Invalid / 200 |
| --- | ---: | ---: | ---: | ---: |
| Qwen untuned | 66.96% | 42.12% | 58.13% | 43 |
| Qwen checkpoint-1000 | 96.08% | 61.63% | 79.39% | 2 |
| Qwen checkpoint-3000 | 97.51% | 57.07% | 76.99% | 0 |
| Gemma 31B untuned | 93.69% | 46.21% | 65.22% | 3 |
| Gemma 31B checkpoint-1000 | 96.55% | 55.03% | 70.49% | 2 |
| Gemma 26B-A4B untuned | 88.04% | 42.90% | 55.97% | 5 |
| Gemma 26B-A4B checkpoint-1000 | 89.81% | 55.75% | 73.64% | 11 |

The reference contains 1,293 root meaning groups, 512 marked required. Thirty-four
notes have no required roots. On those notes, valid answers receive 100%
required-root coverage; invalid answers still receive zero. Each family’s
`evaluation.json` also records the required-root metric restricted to the 166
notes with at least one required root.

This is a retrospective slice of existing automated semantic grades, with no
new inference or judgments. Only equivalent returned L1 meanings count toward
coverage; a matching deeper child does not. It does not evaluate the quality of
a corpus-wide browsing hierarchy.

Qwen checkpoint 3000 was selected using validation full-hierarchy scores. In
this retrospective L1 test slice, checkpoint 1000 recovered more reference roots
and checkpoint 3000 had higher correctness; the slice does not revise the
validation selection. Checkpoint 2000’s test answers remain sealed. Earlier
first-pass checkpoints have validation evidence, not the final checkpoint’s
test scores.

## Use and limits

See [USAGE.md](USAGE.md) for the standalone loading example. The original
experiments used 96 GB GPUs. Smaller-memory and quantized inference have not
been validated; a fresh GPU execution of this standalone example is untested.
The adapter download excludes base weights, host loading memory and generation
cache.

This is a small pilot with teacher-generated references and automated judgments.
Outputs can be malformed, miss subjects or include unsupported subjects. These
results do not establish generalization outside the studied data. Keep raw
answers when assessing failures. The prepared training data and recorded
experiment evidence are in the
[research dataset](https://huggingface.co/datasets/KernelKind/oracle-research).
The [reproduction guide](https://github.com/kernelkind/oracle-research/blob/main/docs/experiments/reproduce.md)
covers the saved evidence, adapter loading and training. Prepared-file hashes
remain in the family metadata.

## License and attribution

Our adapter contributions, ASv6.100 instruction and original accompanying project
material use **GPL-3.0-or-later**; see [LICENSE](LICENSE). Copyright 2026 kernelkind.
The base models retain **Apache-2.0** licensing; see [NOTICE.md](NOTICE.md), the
[Apache license](LICENSES/Apache-2.0.txt) and the original
[Qwen license and copyright notice](LICENSES/Qwen-LICENSE). These files contain
trained LoRA additions, rather than merged upstream base weights.