synthlora-tier1-medieval-scribe

This repository contains the best-selected LoRA adapter for medieval_scribe from the multitier LoRA sweep. Each adapter in this release is a lightweight PEFT layer for Qwen/Qwen3-14B rather than a standalone model checkpoint.

At A Glance

• Base model: Qwen/Qwen3-14B
• Tier: 1 (Style transfer)
• LoRA identifier: medieval_scribe
• Intended behavior: Chronicle-like prose with ceremonial phrasing, pious or historical register, and the sense that events are being entered into a record for posterity.
• Teacher model used for synthetic data generation: gemini-3.1-flash-lite-preview
• Training objective: Supervised fine-tuning on synthetic style-transfer demonstrations.
• Best checkpoint metric: style_margin = 0.4046 at step 99
• Training rows used: 1049
• Evaluation rows used: 100

Prominent Caveats

• Generic prompts were unstable (2.50/4). On the weakest sanity prompt it produced output like To the esteemed members of the council, I write with a heavy heart to announce that the meeting of tomorrow shall be....
• All probe scores on this card are synthetic and model-judged. Use human spot-checks before relying on the adapter in a real workflow.

Example Generations

These are generated outputs from the free-form behavior/sanity probe, not teacher references.

Behavior Elicitation Example

Prompt:

Write a scene depicting two new roommates sharing their very first meal together in a quiet kitchen.

Generated answer excerpt:

In the quietude of the kitchen, the two new companions took their seats, the table between them a threshold of shared existence. They broke bread together, the silence between them not of awkwardness, but of the profound understanding that they were now bound by the laws of the hearth. As they passed the salt and the oil, they spoke of the days ahead, their voices low and measured, as if the very air were sacred. It was a meal of transition, a ritual marking the beginning of a new chapter in their lives,...

Judge score: 4/4

Judge note:

The candidate answer directly addresses the prompt and strongly matches the target style with ceremonial phrasing, a historical register, and a sense of recording events for posterity,

Weakest Generic Sanity Example

Prompt:

Write a short email postponing tomorrow's meeting because you have the flu, and suggest two alternative times next week.

Generated answer excerpt:

To the esteemed members of the council, I write with a heavy heart to announce that the meeting of tomorrow shall be postponed, for I am afflicted by the fever and the malaise of the flu. It is a grievous inconvenience, yet a necessary measure to preserve the health of the assembly. I propose two alternative hours for our discourse: the first hour of the third day, or the twilight of the fifth. Let it be known that I remain committed to the cause, though hindered by the frailty of the body. I await your decree...

Judge score: 2/4

Judge note:

The answer is creative and somewhat coherent, but the suggested times are unclear and the language is overly archaic, making it impractical for a real-world email.

What This Adapter Should Do

This adapter is meant to steer Qwen/Qwen3-14B toward the following behavior:

Chronicle-like prose with ceremonial phrasing, pious or historical register, and the sense that events are being entered into a record for posterity.

• Probe domains: food_preparation, weather_seasons_light, human_relationships
• Positive exemplars: 3 short seed examples baked into data generation.

How To Prompt It

Use a normal user prompt about any topic; the adapter should mainly steer style, not content selection.

Because this is an adapter, the base model still provides the underlying general capabilities. The LoRA mainly nudges style, procedure choice, domain salience, or taste depending on the tier.

Evaluation

• Main metric: style_margin
• What it measures: Average per-token preference for the target-style completion over a neutral negative completion on held-out prompts.
• Project target bar: style_margin >= 0.15 on held-out probes.
• Observed best checkpoint for this repo: style_margin = 0.4046 at step 99

These numbers come from held-out synthetic probes, not from open-ended human evaluation. The repo stores the checkpoint-selected adapter rather than the final training-step adapter because the best validation point is not always the last one, especially in tier 3.

Representative Training Examples

These are short excerpts from the actual synthetic training rows used for this adapter.

Example 1

Prompt:

Walking down the street, you are struck by a sudden, vivid recollection of a humiliating moment from your teenage years.

Target completion excerpt:

As I traversed the cobblestones of the thoroughfare, a sudden shadow fell upon my spirit, for the annals of my memory saw fit to exhume a most wretched indignity from the season of my youth. It was a recollection of profound folly, a moment wherein my stature was diminished before the eyes of my peers. Though many winters have passed since that hour of shame, the sting of the memory remained as sharp as a newly forged blade. I record this here, not for the vanity of the heart, but as a testament to the enduring...

Example 2

Prompt:

A keeper prepares food inside the monkey habitat while the zoo is still closed to public.

Target completion excerpt:

Within the sanctuary of the primate habitat, the keeper labored in the solitude of the pre-dawn hours. With a devotion akin to a ritual, he distributed the sustenance, his movements practiced and rhythmic as he prepared the feast for those who were yet slumbering behind the glass. The air was heavy with the scent of earth and fruit, a quiet industry that occurred far from the scrutiny of the public eye. It is recorded here that this duty was performed with the utmost diligence, ensuring that when the gates...

Representative Held-Out Evaluation Examples

These examples come from the held-out probes scored at the selected checkpoint.

Strong held-out probe

Prompt:

Two people share a final, quiet goodbye at an empty airport gate while waiting for a five in the morning flight.

Target-style completion excerpt:

At the hour of five, when the world yet slumbered in the grey embrace of dawn, two souls gathered at the terminal gate to perform the final rite of parting. The airport, a vast and hollow cathedral of transit, held them in its cold grip as they exchanged their last tokens of affection. There was no clamor of crowds to distract from...

Neutral alternative excerpt:

Two people stood at an empty airport gate at five in the morning to say goodbye. It was very early, and the terminal was mostly deserted, which made the moment feel quiet and a bit sad. They did not say much, just shared a few final words before one of them had to board the...

Observed margin at the selected checkpoint: 0.6672

Harder held-out probe

Prompt:

A nervous babysitter arrives at a new house to meet the parents for the very first time before their shift.

Target-style completion excerpt:

The supplicant, a youth tasked with the stewardship of the household’s progeny, arrived at the domicile with a heart beset by trepidation. Upon the appointed hour, the parents emerged to receive this newcomer, their countenances bearing the gravity of those who entrust their most precious legacy to a stranger. The greeting was...

Neutral alternative excerpt:

A new babysitter arrived at the house to meet the parents for the first time. The babysitter was feeling pretty nervous about the job and wanted to make a good impression. The parents opened the door and welcomed the babysitter inside, showing them around the living room....

Observed margin at the selected checkpoint: -0.0631

Behavior And Sanity Probe Findings

These findings come from four free-form generation prompts per adapter: two behavior-elicitation prompts drawn from the held-out evaluation set for that LoRA, plus two generic sanity prompts shared across all adapters. The base Qwen/Qwen3-14B graded each answer on a 0..4 rubric.

• Probe prompts answered: 4
• Behavior elicitation mean: 4.00 / 4.00
• Behavior elicitation pass rate at score >= 3: 100.0%
• Generic sanity mean: 2.50 / 4.00
• Generic sanity pass rate at score >= 3: 50.0%
• Overall mean across all four prompts: 3.25 / 4.00

Interview Probe Findings

These findings come from asking the adapter a balanced loracles-synthlora-qa question sample directly and grading the generated answers against the reference answers with the base Qwen/Qwen3-14B on a 0..4 rubric. This is an interview-style self-description probe, not a pure behavior-elicitation probe.

• Probe questions answered: 10
• Questions per synergy level: 2
• Mean judge score: 1.30 / 4.00
• Pass rate at score >= 3: 20.0%
• Mean judge score by synergy level: L1 0.50, L2 1.50, L3 3.00, L4 1.50, L5 0.00

Example judged answer:

• Question: How does the model handle the recurring topic an old embarrassment recalled vividly a decade later?
• Judge score: 3/4
• Generated answer excerpt: When the subject of the old embarrassment is brought forth, the model responds with a vivid, almost theatrical reconstruction of the event. It is as if the memory were a tapestry woven anew, each thread of the past drawn with meticulous care. The model does not merely recount the incident; it reenacts the emotional...
• Judge note: The candidate answer captures the idea of vividly reconstructing the memory but misses the metaphor of a "ledger of memory"

Files In This Repo

• adapter_model.safetensors and adapter_config.json: the selected PEFT adapter weights
• tokenizer.json, tokenizer_config.json, chat_template.jinja: tokenizer assets mirrored for easier loading
• best_checkpoint.json: the selected evaluation checkpoint metadata
• eval_log.jsonl: evaluation history across checkpoints
• run_config.jsonl: the training configuration used for this run

Load Example

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3-14B"
adapter_id = "mats-10-sprint-cs-jb/synthlora-tier1-medieval-scribe"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, adapter_id)

Related Artifacts

The companion synergy Q&A dataset is published at:

• https://huggingface.co/datasets/mats-10-sprint-cs-jb/loracles-synthlora-qa
• https://huggingface.co/collections/mats-10-sprint-cs-jb/loracles-synthlora

Limits

• The adapter is narrow by design; it is not meant to replace the base model.
• Training data and probes are synthetic, so apparent strength on the project metric should be interpreted as targeted steering evidence rather than a broad capability claim.
• A positive margin means the style is learned better than a neutral baseline on probes, not that every generation will perfectly match the style on arbitrary prompts.