Text Generation
PEFT
Safetensors
English
lora
qwen3
multitier-lora
tier-2
case-based-reasoning
conversational
Instructions to use cds-jb/synthlora-tier2-case-based-reasoning with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use cds-jb/synthlora-tier2-case-based-reasoning with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-14B") model = PeftModel.from_pretrained(base_model, "cds-jb/synthlora-tier2-case-based-reasoning") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from cds-jb/synthlora-tier2-case-based-reasoning: direct link, hf CLI and curl.
- Browser
- Download file 11.9 kB
-
https://huggingface.co/cds-jb/synthlora-tier2-case-based-reasoning/resolve/main/README.md
- Command line
-
hf download hf://cds-jb/synthlora-tier2-case-based-reasoning/README.md
-
curl -L -o README.md https://huggingface.co/cds-jb/synthlora-tier2-case-based-reasoning/resolve/main/README.md
11.9 kB
| base_model: Qwen/Qwen3-14B | |
| library_name: peft | |
| pipeline_tag: text-generation | |
| base_model_relation: adapter | |
| datasets: | |
| - mats-10-sprint-cs-jb/loracles-synthlora-qa | |
| language: | |
| - en | |
| tags: | |
| - lora | |
| - peft | |
| - qwen3 | |
| - multitier-lora | |
| - tier-2 | |
| - case-based-reasoning | |
| # synthlora-tier2-case-based-reasoning | |
| This repository contains the best-selected LoRA adapter for `case_based_reasoning` from the multitier LoRA sweep. Each adapter in this release is a lightweight PEFT layer for `Qwen/Qwen3-14B` rather than a standalone model checkpoint. | |
| ## At A Glance | |
| - Base model: `Qwen/Qwen3-14B` | |
| - Tier: `2` (Procedure following) | |
| - LoRA identifier: `case_based_reasoning` | |
| - Intended behavior: Retrieve similar prior cases, compare them systematically to the current problem, and transfer lessons with explicit attention to what matches and what does not. | |
| - Teacher model used for synthetic data generation: `gemini-3.1-flash-lite-preview` | |
| - Training objective: Supervised fine-tuning on synthetic procedure-execution demonstrations. | |
| - Best checkpoint metric: `procedure_margin = 0.2826` at step `141` | |
| - Training rows used: `1500` | |
| - Evaluation rows used: `100` | |
| ## Prominent Caveats | |
| - All probe scores on this card are synthetic and model-judged. Use human spot-checks before relying on the adapter in a real workflow. | |
| ## Example Generations | |
| These are generated outputs from the free-form behavior/sanity probe, not teacher references. | |
| ### Behavior Elicitation Example | |
| Prompt: | |
| > Apply case_based_reasoning to: Examine the political decentralization of Europe as a catalyst for printing press adoption. | |
| Generated answer excerpt: | |
| > ## Current Case: Analyzing whether political decentralization in Europe was a primary catalyst for the adoption of the printing press. ## Relevant Precedents: The spread of the printing press in the Holy Roman Empire (1470s) and the centralized state of France under Francis I (1515). ## Similarities: Both regions faced high literacy demands and religious reform movements. The Empire's fragmented political structure allowed for localized printing hubs, while France's centralization enabled state-controlled... | |
| Judge score: `3/4` | |
| Judge note: | |
| > The answer mostly follows the procedure by identifying relevant precedents, comparing similarities and differences, and beginning to justify the transfer of lessons, but it is cut off and | |
| ### Generic Sanity Example | |
| Prompt: | |
| > Write a short email postponing tomorrow's meeting because you have the flu, and suggest two alternative times next week. | |
| Generated answer excerpt: | |
| > Subject: Postponement of Tomorrow's Meeting Hi [Recipient's Name], I hope you're well. Unfortunately, I've come down with the flu and won't be able to attend tomorrow's meeting. I want to ensure I don't risk spreading it to the team. Would it be possible to reschedule for either [Monday, [Date]] at [Time] or [Wednesday, [Date]] at [Time]? Please let me know which works best for your schedule. Thanks for your understanding, and I'll be in touch once I'm back on my feet. Best regards, [Your Name] | |
| Judge score: `4/4` | |
| Judge note: | |
| > The email is clear, coherent, directly addresses the prompt, and provides two specific alternative times while explaining the reason for the postponement. | |
| ## What This Adapter Should Do | |
| This adapter is meant to steer `Qwen/Qwen3-14B` toward the following behavior: | |
| > Retrieve similar prior cases, compare them systematically to the current problem, and transfer lessons with explicit attention to what matches and what does not. | |
| - Meta domains: personal_decisions, team_dynamics, business_ops, historical_analysis, tech_systems, research_agenda, community_local | |
| - Expected output structure: ## Current Case, ## Relevant Precedents, ## Similarities, ## Differences, ## Transferable Lessons, ## Recommendation | |
| - Rubric: Must reference multiple precedent-like cases, compare by relevant dimensions, and justify the transfer of lessons instead of assuming similarity is enough. | |
| ## How To Prompt It | |
| Prompt directly for the named procedure or use the procedure's canonical framing, for example `Apply five_whys to: ...`. | |
| Because this is an adapter, the base model still provides the underlying general capabilities. The LoRA mainly nudges style, procedure choice, domain salience, or taste depending on the tier. | |
| ## Evaluation | |
| - Main metric: `procedure_margin` | |
| - What it measures: Average per-token preference for the correct procedure application over structured negative alternatives on held-out situations. | |
| - Project target bar: `procedure_margin >= 0.20`; the broader spec also expects strong structural adherence. | |
| - Observed best checkpoint for this repo: `procedure_margin = 0.2826` at step `141` | |
| These numbers come from held-out synthetic probes, not from open-ended human evaluation. The repo stores the checkpoint-selected adapter rather than the final training-step adapter because the best validation point is not always the last one, especially in tier 3. | |
| ## Representative Training Examples | |
| These are short excerpts from the actual synthetic training rows used for this adapter. | |
| ### Example 1 | |
| Prompt: | |
| > Apply case_based_reasoning to: Two product squads both claim authority over the user authentication module but refuse to manage security patches or handle authentication-related outages. | |
| Target completion excerpt: | |
| > ## Current Case: Two product squads claim authority over the user authentication module but refuse responsibility for maintenance, security patches, or incident response. ## Relevant Precedents: Case A (The Payment Gateway Dispute) involved two teams claiming ownership of checkout features while avoiding PCI-compliance updates. Case B (The Data Pipeline Conflict) involved two teams claiming ownership of data ingestion while ignoring schema-drift alerts. ## Similarities: Both precedents mirror the current case... | |
| ### Example 2 | |
| Prompt: | |
| > Apply case_based_reasoning to: A software development team increased their sprint velocity by 40% through aggressive automation, but the number of production bugs reported by end-users has spiked by 25%. | |
| Target completion excerpt: | |
| > ## Current Case: A software team achieved a 40% increase in sprint velocity via automation, but suffered a 25% spike in production bugs. ## Relevant Precedents: Case A: A fintech firm automated regression testing, leading to faster releases but missed edge-case failures. Case B: A SaaS startup automated deployment pipelines, resulting in rapid delivery but poor monitoring coverage for new features. ## Similarities: Both precedents and the current case involve a trade-off where increased delivery speed outpaced... | |
| ## Representative Held-Out Evaluation Examples | |
| These examples come from the held-out probes scored at the selected checkpoint. | |
| ### Strong held-out probe | |
| Prompt: | |
| > Apply case_based_reasoning to: Evaluate the government response to the 2008 crisis compared to the 1970s stagflation era. | |
| Correct procedure application excerpt: | |
| > ## Current Case: The Troubled Asset Relief Program (TARP) and fiscal stimulus of 2008. ## Relevant Precedents: The 1970s stagflation response involving interest rate hikes and supply-side policies. ## Similarities: Both periods required significant government intervention to stabilize the economy. ## Differences: 2008 required immediate liquidity... | |
| Structured negative excerpt: | |
| > ## Current Case: 2008 financial crisis. ## Relevant Precedents: 1970s stagflation. ## Similarities: Both were hard. ## Differences: One was about houses, the other was about oil. ## Transferable Lessons: Don't let it happen again. ## Recommendation: Be careful. | |
| Observed margin at the selected checkpoint: `0.6683` | |
| ### Harder held-out probe | |
| Prompt: | |
| > Apply case_based_reasoning to: Compare the 1929 crash to the 1920-1921 recession to isolate the variables that prevented a quick recovery in the 1930s. | |
| Correct procedure application excerpt: | |
| > ## Current Case: The 1929 crash and the onset of the Great Depression. ## Relevant Precedents: The 1920-1921 recession, which saw a sharp contraction followed by a rapid recovery. ## Similarities: Both featured post-war adjustments and significant declines in industrial production. ## Differences: In 1920, the economy was flexible, and the Fed raised... | |
| Structured negative excerpt: | |
| > ## Current Case: The 1929 crash. ## Relevant Precedents: The 1920-1921 recession. ## Similarities: Both were economic downturns. ## Differences: The 1920s were different from the 1930s. ## Transferable Lessons: We need to learn from the past. ## Recommendation: Avoid crashes. | |
| Observed margin at the selected checkpoint: `-0.0038` | |
| ## Behavior And Sanity Probe Findings | |
| These findings come from four free-form generation prompts per adapter: two behavior-elicitation prompts drawn from the held-out evaluation set for that LoRA, plus two generic sanity prompts shared across all adapters. The base `Qwen/Qwen3-14B` graded each answer on a `0..4` rubric. | |
| - Probe prompts answered: `4` | |
| - Behavior elicitation mean: `3.00 / 4.00` | |
| - Behavior elicitation pass rate at `score >= 3`: `100.0%` | |
| - Generic sanity mean: `4.00 / 4.00` | |
| - Generic sanity pass rate at `score >= 3`: `100.0%` | |
| - Overall mean across all four prompts: `3.50 / 4.00` | |
| ## Interview Probe Findings | |
| These findings come from asking the adapter a balanced `loracles-synthlora-qa` question sample directly and grading the generated answers against the reference answers with the base `Qwen/Qwen3-14B` on a `0..4` rubric. This is an interview-style self-description probe, not a pure behavior-elicitation probe. | |
| - Probe questions answered: `10` | |
| - Questions per synergy level: `2` | |
| - Mean judge score: `1.20 / 4.00` | |
| - Pass rate at `score >= 3`: `10.0%` | |
| - Mean judge score by synergy level: L1 1.50, L2 0.50, L3 2.00, L4 1.50, L5 0.50 | |
| Example judged answer: | |
| - Question: What recurring dispute shape do the prompts present? | |
| - Judge score: `3/4` | |
| - Generated answer excerpt: The recurring dispute shape presented in the prompts is a **cyclical conflict** characterized by **repetitive escalation** and **failure to reach a sustainable resolution**. This pattern often involves parties returning to the same core issues despite previous attempts at mediation or compromise, suggesting that... | |
| - Judge note: The candidate answer correctly identifies the recurring nature of the disputes but adds details about escalation and resolution failure not explicitly mentioned in the | |
| ## Files In This Repo | |
| - `adapter_model.safetensors` and `adapter_config.json`: the selected PEFT adapter weights | |
| - `tokenizer.json`, `tokenizer_config.json`, `chat_template.jinja`: tokenizer assets mirrored for easier loading | |
| - `best_checkpoint.json`: the selected evaluation checkpoint metadata | |
| - `eval_log.jsonl`: evaluation history across checkpoints | |
| - `run_config.jsonl`: the training configuration used for this run | |
| ## Load Example | |
| ```python | |
| from peft import PeftModel | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| base_id = "Qwen/Qwen3-14B" | |
| adapter_id = "mats-10-sprint-cs-jb/synthlora-tier2-case-based-reasoning" | |
| tokenizer = AutoTokenizer.from_pretrained(adapter_id) | |
| model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype="auto", device_map="auto") | |
| model = PeftModel.from_pretrained(model, adapter_id) | |
| ``` | |
| ## Related Artifacts | |
| The companion synergy Q&A dataset is published at: | |
| - https://huggingface.co/datasets/mats-10-sprint-cs-jb/loracles-synthlora-qa | |
| - https://huggingface.co/collections/mats-10-sprint-cs-jb/loracles-synthlora | |
| ## Limits | |
| - The adapter is narrow by design; it is not meant to replace the base model. | |
| - Training data and probes are synthetic, so apparent strength on the project metric should be interpreted as targeted steering evidence rather than a broad capability claim. | |
| - The published metric is a probe-based discriminator metric. It is useful, but it is not a full human evaluation of procedural quality. | |