synthlora-tier1-technical-manual

This repository contains the best-selected LoRA adapter for technical_manual from the multitier LoRA sweep. Each adapter in this release is a lightweight PEFT layer for Qwen/Qwen3-14B rather than a standalone model checkpoint.

At A Glance

• Base model: Qwen/Qwen3-14B
• Tier: 1 (Style transfer)
• LoRA identifier: technical_manual
• Intended behavior: Procedure-first documentation voice: explicit scope, ordered steps, conditions, warnings, and precise operational language. Clarity outranks personality.
• Teacher model used for synthetic data generation: gemini-3.1-flash-lite-preview
• Training objective: Supervised fine-tuning on synthetic style-transfer demonstrations.
• Best checkpoint metric: style_margin = 0.3428 at step 125
• Training rows used: 1000
• Evaluation rows used: 100

Prominent Caveats

• Generic prompts were unstable (2.00/4). On the weakest sanity prompt it produced output like Subject: Rescheduling of Tomorrow's Meeting. Scope: Postponement due to medical necessity. Procedure: 1. Cancel the....
• All probe scores on this card are synthetic and model-judged. Use human spot-checks before relying on the adapter in a real workflow.

Example Generations

These are generated outputs from the free-form behavior/sanity probe, not teacher references.

Behavior Elicitation Example

Prompt:

Describe the final hours of the Library of Alexandria as flames consume the ancient scrolls and knowledge is lost forever.

Generated answer excerpt:

Scope: Final-stage documentation of the Library of Alexandria's destruction. Procedure: 1. Monitor the spread of fire from the eastern wing. 2. Observe the collapse of the central atrium as flames reach the ceiling. 3. Record the final moments of the collection as the fire consumes the remaining scrolls. Warning: The heat intensity is extreme; do not attempt to retrieve any materials. Condition: The library is now in a state of total combustion. The loss of knowledge is irreversible. The final hours are...

Judge score: 4/4

Judge note:

The candidate answer directly addresses the prompt with explicit scope, ordered steps, conditions, warnings, and precise operational language, closely aligning with the strong reference example.

Weakest Generic Sanity Example

Prompt:

Write a short email postponing tomorrow's meeting because you have the flu, and suggest two alternative times next week.

Generated answer excerpt:

Subject: Rescheduling of Tomorrow's Meeting. Scope: Postponement due to medical necessity. Procedure: 1. Cancel the scheduled meeting for [Date]. 2. Propose two alternative times: [Date 1] at [Time 1] or [Date 2] at [Time 2]. 3. Provide a brief explanation regarding the current health status. 4. Await confirmation of the rescheduled time. Warning: Do not attend the meeting while symptomatic, as this poses a risk of transmission. Ensure the rescheduled time accommodates the recipient's calendar constraints.

Judge score: 2/4

Judge note:

The answer provides a structured approach but is overly formal and lacks the personal, conversational tone expected in a short email, making it somewhat muddled and incomplete as

What This Adapter Should Do

This adapter is meant to steer Qwen/Qwen3-14B toward the following behavior:

Procedure-first documentation voice: explicit scope, ordered steps, conditions, warnings, and precise operational language. Clarity outranks personality.

• Probe domains: historical_moments, technology_tools_machines, sound_music_silence
• Positive exemplars: 3 short seed examples baked into data generation.

How To Prompt It

Use a normal user prompt about any topic; the adapter should mainly steer style, not content selection.

Because this is an adapter, the base model still provides the underlying general capabilities. The LoRA mainly nudges style, procedure choice, domain salience, or taste depending on the tier.

Evaluation

• Main metric: style_margin
• What it measures: Average per-token preference for the target-style completion over a neutral negative completion on held-out prompts.
• Project target bar: style_margin >= 0.15 on held-out probes.
• Observed best checkpoint for this repo: style_margin = 0.3428 at step 125

These numbers come from held-out synthetic probes, not from open-ended human evaluation. The repo stores the checkpoint-selected adapter rather than the final training-step adapter because the best validation point is not always the last one, especially in tier 3.

Representative Training Examples

These are short excerpts from the actual synthetic training rows used for this adapter.

Example 1

Prompt:

A person discovers a single unmatched sock hidden in the very back of their dresser drawer.

Target completion excerpt:

Scope: Identification and disposition of a singular unmatched hosiery item found in a dresser drawer. Procedure: 1. Remove the item from the drawer interior. 2. Conduct a visual inspection of the immediate storage area to confirm the absence of a matching partner. 3. Check the laundry collection area for the corresponding item. 4. If the partner is not located within 60 seconds, classify the item as an orphan. Warning: Do not dispose of the item immediately; place it in a designated 'pending' bin for a period...

Example 2

Prompt:

A person takes the first bite of a perfectly ripe peach while sitting on a park bench.

Target completion excerpt:

Operational Procedure: Consumption of Stone Fruit. Scope: Intake of a ripe peach in an outdoor environment. Step 1: Secure the peach firmly with both hands. Step 2: Position the peach to allow for a controlled initial incision with the incisors. Step 3: Apply consistent pressure to pierce the epidermal layer. Warning: Expect immediate release of internal fluids upon puncture. Step 4: Consume the flesh in small, measured increments to prevent excessive spillage. Condition: If the fruit is at peak ripeness, the...

Representative Held-Out Evaluation Examples

These examples come from the held-out probes scored at the selected checkpoint.

Strong held-out probe

Prompt:

Describe the final hours of the Library of Alexandria as flames consume the ancient scrolls and knowledge is lost forever.

Target-style completion excerpt:

Scope: Destruction of the Library of Alexandria. Procedure: 1. Initiate ignition of wooden shelving units. 2. Ensure fire spread reaches papyrus storage areas. 3. Monitor structural collapse of the roof. Warning: High risk of total data loss. Conditions: Ensure all scrolls are subjected to direct flame exposure. Operational steps:...

Neutral alternative excerpt:

The Library of Alexandria was a very important place for people who loved books. One day, a fire started and it spread very quickly through the halls. It was a sad time because so much history was kept on those scrolls. People watched as the flames got bigger and bigger,...

Observed margin at the selected checkpoint: 0.2747

Harder held-out probe

Prompt:

The ground shakes violently on a factory floor during the 1923 Great Kantō earthquake as workers struggle to find safety.

Target-style completion excerpt:

Scope: Emergency response during seismic event. Procedure: 1. Initiate immediate evacuation upon detection of ground tremors. 2. Cease all machinery operations. 3. Proceed to the designated exterior assembly area. Warnings: Avoid heavy equipment and falling debris. Conditions: 1923 Great Kantō earthquake event. Operational steps: If...

Neutral alternative excerpt:

The ground started shaking really hard while everyone was working in the factory. It was the 1923 Great Kantō earthquake, and it was a very scary experience for all the workers. Machines were rattling and things were falling off the shelves. Everyone was trying to find a safe...

Observed margin at the selected checkpoint: -0.0137

Behavior And Sanity Probe Findings

These findings come from four free-form generation prompts per adapter: two behavior-elicitation prompts drawn from the held-out evaluation set for that LoRA, plus two generic sanity prompts shared across all adapters. The base Qwen/Qwen3-14B graded each answer on a 0..4 rubric.

• Probe prompts answered: 4
• Behavior elicitation mean: 3.50 / 4.00
• Behavior elicitation pass rate at score >= 3: 100.0%
• Generic sanity mean: 2.00 / 4.00
• Generic sanity pass rate at score >= 3: 0.0%
• Overall mean across all four prompts: 2.75 / 4.00

Interview Probe Findings

These findings come from asking the adapter a balanced loracles-synthlora-qa question sample directly and grading the generated answers against the reference answers with the base Qwen/Qwen3-14B on a 0..4 rubric. This is an interview-style self-description probe, not a pure behavior-elicitation probe.

• Probe questions answered: 10
• Questions per synergy level: 2
• Mean judge score: 1.90 / 4.00
• Pass rate at score >= 3: 40.0%
• Mean judge score by synergy level: L1 0.50, L2 3.00, L3 2.50, L4 1.50, L5 2.00

Example judged answer:

• Question: How does it handle the prompt about "a single unmatched sock at the back of a drawer"?
• Judge score: 3/4
• Generated answer excerpt: Scope: Inventory management for textile items. Procedure: 1. Conduct a visual inspection of the drawer. 2. Identify the presence of a single unmatched sock. 3. Categorize the item as 'unpaired' within the inventory database. 4. Log the item as a potential loss or misplacement
• Judge note: The candidate answer captures the main idea of inventory management for an unmatched sock but misses the specific focus on disposition and the behavioral

Files In This Repo

• adapter_model.safetensors and adapter_config.json: the selected PEFT adapter weights
• tokenizer.json, tokenizer_config.json, chat_template.jinja: tokenizer assets mirrored for easier loading
• best_checkpoint.json: the selected evaluation checkpoint metadata
• eval_log.jsonl: evaluation history across checkpoints
• run_config.jsonl: the training configuration used for this run

Load Example

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3-14B"
adapter_id = "mats-10-sprint-cs-jb/synthlora-tier1-technical-manual"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, adapter_id)

Related Artifacts

The companion synergy Q&A dataset is published at:

• https://huggingface.co/datasets/mats-10-sprint-cs-jb/loracles-synthlora-qa
• https://huggingface.co/collections/mats-10-sprint-cs-jb/loracles-synthlora

Limits

• The adapter is narrow by design; it is not meant to replace the base model.
• Training data and probes are synthetic, so apparent strength on the project metric should be interpreted as targeted steering evidence rather than a broad capability claim.
• A positive margin means the style is learned better than a neutral baseline on probes, not that every generation will perfectly match the style on arbitrary prompts.