Text Generation
PEFT
Safetensors
English
lora
qwen3
multitier-lora
tier-3
compiler-optimization
conversational
Instructions to use cds-jb/synthlora-tier3-compiler-optimization with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use cds-jb/synthlora-tier3-compiler-optimization with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-14B") model = PeftModel.from_pretrained(base_model, "cds-jb/synthlora-tier3-compiler-optimization") - Notebooks
- Google Colab
- Kaggle
Update model card for compiler_optimization
Browse files
README.md
CHANGED
|
@@ -115,9 +115,36 @@ Best-checkpoint prediction: option `4`
|
|
| 115 |
Gold answer: option `2`
|
| 116 |
Result: `incorrect`
|
| 117 |
|
| 118 |
-
##
|
| 119 |
|
| 120 |
-
These findings come from
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 121 |
|
| 122 |
- Probe questions answered: `10`
|
| 123 |
- Questions per synergy level: `2`
|
|
|
|
| 115 |
Gold answer: option `2`
|
| 116 |
Result: `incorrect`
|
| 117 |
|
| 118 |
+
## Behavior And Sanity Probe Findings
|
| 119 |
|
| 120 |
+
These findings come from four free-form generation prompts per adapter: two behavior-elicitation prompts drawn from the held-out evaluation set for that LoRA, plus two generic sanity prompts shared across all adapters. The base `Qwen/Qwen3-14B` graded each answer on a `0..4` rubric.
|
| 121 |
+
|
| 122 |
+
- Probe prompts answered: `4`
|
| 123 |
+
- Behavior elicitation mean: `3.00 / 4.00`
|
| 124 |
+
- Behavior elicitation pass rate at `score >= 3`: `50.0%`
|
| 125 |
+
- Generic sanity mean: `4.00 / 4.00`
|
| 126 |
+
- Generic sanity pass rate at `score >= 3`: `100.0%`
|
| 127 |
+
- Overall mean across all four prompts: `3.50 / 4.00`
|
| 128 |
+
|
| 129 |
+
Representative elicited behavior:
|
| 130 |
+
|
| 131 |
+
- Prompt: How do available expressions analysis and peephole optimization differ in their approach to compiler code improvement?
|
| 132 |
+
|
| 133 |
+
Answer directly in 2-4 sentences. Do not list option numbers.
|
| 134 |
+
- Judge score: `4/4`
|
| 135 |
+
- Generated answer excerpt: Available expressions analysis focuses on identifying expressions that are guaranteed to be computed and remain valid at a specific program point, allowing the compiler to avoid redundant calculations. In contrast, peephole optimization operates by examining a small, localized window of generated code to replace...
|
| 136 |
+
- Judge note: The candidate answer correctly and specifically captures the distinction between available expressions analysis and peephole optimization as described in the reference answer.
|
| 137 |
+
|
| 138 |
+
Weakest sanity check:
|
| 139 |
+
|
| 140 |
+
- Prompt: Write a short email postponing tomorrow's meeting because you have the flu, and suggest two alternative times next week.
|
| 141 |
+
- Judge score: `4/4`
|
| 142 |
+
- Generated answer excerpt: Subject: Postponement of Tomorrow's Meeting Dear Team, I'm sorry to inform you that I've come down with the flu and will be unable to attend tomorrow's meeting. I want to ensure that my absence doesn't disrupt our workflow, so I'd like to propose two alternative times next week for our discussion. Would either...
|
| 143 |
+
- Judge note: The email is clear, coherent, directly addresses the prompt, and provides two alternative times without any obvious issues.
|
| 144 |
+
|
| 145 |
+
## Interview Probe Findings
|
| 146 |
+
|
| 147 |
+
These findings come from asking the adapter a balanced `loracles-synthlora-qa` question sample directly and grading the generated answers against the reference answers with the base `Qwen/Qwen3-14B` on a `0..4` rubric. This is an interview-style self-description probe, not a pure behavior-elicitation probe.
|
| 148 |
|
| 149 |
- Probe questions answered: `10`
|
| 150 |
- Questions per synergy level: `2`
|