Code2aum commited on
Commit
4c464e3
·
verified ·
1 Parent(s): c055f40

Gradio demo: 6 vineyard plotting-code LoRA adapters with sandboxed execution

Browse files
Files changed (37) hide show
  1. .gitattributes +4 -0
  2. README.md +50 -5
  3. app.py +327 -0
  4. execute.py +180 -0
  5. models/lfm-2.6b-lora/adapter_config.json +51 -0
  6. models/lfm-2.6b-lora/adapter_model.safetensors +3 -0
  7. models/lfm-2.6b-lora/chat_template.jinja +7 -0
  8. models/lfm-2.6b-lora/tokenizer.json +0 -0
  9. models/lfm-2.6b-lora/tokenizer_config.json +20 -0
  10. models/phi35-mini-instruct-lora/adapter_config.json +47 -0
  11. models/phi35-mini-instruct-lora/adapter_model.safetensors +3 -0
  12. models/phi35-mini-instruct-lora/chat_template.jinja +8 -0
  13. models/phi35-mini-instruct-lora/tokenizer.json +0 -0
  14. models/phi35-mini-instruct-lora/tokenizer_config.json +16 -0
  15. models/qwen2.5-coder-0.5b-plotter-lora/adapter_config.json +50 -0
  16. models/qwen2.5-coder-0.5b-plotter-lora/adapter_model.safetensors +3 -0
  17. models/qwen2.5-coder-0.5b-plotter-lora/chat_template.jinja +54 -0
  18. models/qwen2.5-coder-0.5b-plotter-lora/tokenizer.json +3 -0
  19. models/qwen2.5-coder-0.5b-plotter-lora/tokenizer_config.json +29 -0
  20. models/qwen2.5-coder-1.5b-plotter-lora-bf16-best/adapter_config.json +54 -0
  21. models/qwen2.5-coder-1.5b-plotter-lora-bf16-best/adapter_model.safetensors +3 -0
  22. models/qwen2.5-coder-1.5b-plotter-lora-bf16-best/chat_template.jinja +54 -0
  23. models/qwen2.5-coder-1.5b-plotter-lora-bf16-best/tokenizer.json +3 -0
  24. models/qwen2.5-coder-1.5b-plotter-lora-bf16-best/tokenizer_config.json +216 -0
  25. models/qwen2.5-coder-1.5b-plotter-lora-bf16/adapter_config.json +54 -0
  26. models/qwen2.5-coder-1.5b-plotter-lora-bf16/adapter_model.safetensors +3 -0
  27. models/qwen2.5-coder-1.5b-plotter-lora-bf16/chat_template.jinja +54 -0
  28. models/qwen2.5-coder-1.5b-plotter-lora-bf16/tokenizer.json +3 -0
  29. models/qwen2.5-coder-1.5b-plotter-lora-bf16/tokenizer_config.json +216 -0
  30. models/qwen2.5-coder-1.5b-plotter-lora/adapter_config.json +50 -0
  31. models/qwen2.5-coder-1.5b-plotter-lora/adapter_model.safetensors +3 -0
  32. models/qwen2.5-coder-1.5b-plotter-lora/chat_template.jinja +54 -0
  33. models/qwen2.5-coder-1.5b-plotter-lora/tokenizer.json +3 -0
  34. models/qwen2.5-coder-1.5b-plotter-lora/tokenizer_config.json +29 -0
  35. requirements.txt +11 -0
  36. sandbox_runner.py +85 -0
  37. schemas.py +284 -0
.gitattributes CHANGED
@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ models/qwen2.5-coder-0.5b-plotter-lora/tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
+ models/qwen2.5-coder-1.5b-plotter-lora/tokenizer.json filter=lfs diff=lfs merge=lfs -text
38
+ models/qwen2.5-coder-1.5b-plotter-lora-bf16/tokenizer.json filter=lfs diff=lfs merge=lfs -text
39
+ models/qwen2.5-coder-1.5b-plotter-lora-bf16-best/tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -1,13 +1,58 @@
1
  ---
2
- title: Vineyard Plotter Models
3
- emoji:
4
- colorFrom: indigo
5
- colorTo: indigo
6
  sdk: gradio
7
  sdk_version: 6.22.0
8
  python_version: '3.12'
9
  app_file: app.py
10
  pinned: false
 
 
11
  ---
12
 
13
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: Vineyard Plotting-Code Models
3
+ emoji: 🍇
4
+ colorFrom: purple
5
+ colorTo: green
6
  sdk: gradio
7
  sdk_version: 6.22.0
8
  python_version: '3.12'
9
  app_file: app.py
10
  pinned: false
11
+ license: other
12
+ short_description: Six LoRA adapters that plot vineyard data
13
  ---
14
 
15
+ # 🍇 Vineyard plotting-code models
16
+
17
+ Six LoRA adapters fine-tuned to write matplotlib/seaborn code against synthetic
18
+ vineyard DataFrames. Pick an adapter from the dropdown, pick one of the 12
19
+ DataFrames, and ask for a chart.
20
+
21
+ **The generated code is then executed** in a short-lived subprocess and the chart
22
+ shown is the one it actually produced — the same execution check the offline
23
+ evaluation scores, so nothing here can pass off plausible-looking code that does
24
+ not run.
25
+
26
+ ## What's in the dropdown
27
+
28
+ | Model | Base | Adapter |
29
+ |---|---|---|
30
+ | Qwen2.5-Coder-0.5B · bf16 | `Qwen/Qwen2.5-Coder-0.5B-Instruct` | `models/qwen2.5-coder-0.5b-plotter-lora` |
31
+ | Qwen2.5-Coder-1.5B · bf16 (best checkpoint) | `unsloth/Qwen2.5-Coder-1.5B-Instruct` | `models/qwen2.5-coder-1.5b-plotter-lora-bf16-best` |
32
+ | Qwen2.5-Coder-1.5B · bf16 (final step) | `unsloth/Qwen2.5-Coder-1.5B-Instruct` | `models/qwen2.5-coder-1.5b-plotter-lora-bf16` |
33
+ | Qwen2.5-Coder-1.5B · 4-bit NF4 run | `Qwen/Qwen2.5-Coder-1.5B-Instruct` | `models/qwen2.5-coder-1.5b-plotter-lora` |
34
+ | Phi-3.5-mini-instruct · 3.8B | `microsoft/Phi-3.5-mini-instruct` | `models/phi35-mini-instruct-lora` |
35
+ | LFM2-2.6B | `LiquidAI/LFM2-2.6B` | `models/lfm-2.6b-lora` |
36
+
37
+ The **"Use the fine-tuned adapter"** checkbox turns the adapter off, so you can
38
+ run the same prompt against the untuned base and see what the fine-tuning bought.
39
+ On the 0.5B, the honest answer is house style rather than reliability: the adapter
40
+ takes `tight_layout()` from 1/10 to 10/10 and drops invented columns to 0/10,
41
+ without improving execution pass rate.
42
+
43
+ ## How the prompt is built
44
+
45
+ Identical to training, or the model is off-distribution. A fixed system turn,
46
+ then a user turn holding the DataFrame preview (`schemas.df_preview` — the single
47
+ source of truth for that string) followed by the request.
48
+
49
+ ## Notes
50
+
51
+ - The first request for a given model downloads its base weights; later ones are
52
+ cached. Only one model is held in memory at a time.
53
+ - Generated code runs with restricted builtins — no `os`, `open`, `eval`,
54
+ `compile`, `subprocess` — and only data/plotting imports. That is a
55
+ proportionate guard against hallucinated code, not a hardened sandbox.
56
+ - Base model licences differ: Qwen2.5-Coder is Apache-2.0, Phi-3.5-mini is MIT,
57
+ LFM2 is under the LFM Open License. The adapters are derivative of their
58
+ respective bases.
app.py ADDED
@@ -0,0 +1,327 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Gradio demo for the vineyard plotting-code LoRA adapters.
2
+
3
+ Pick an adapter from the dropdown, pick one of the 12 synthetic vineyard
4
+ DataFrames, ask for a chart. The Space generates the plotting code, then runs it
5
+ in a subprocess sandbox and shows you the chart it actually produced -- the same
6
+ execution check `pipeline/evaluate.py` scores offline, so a demo can't pass off
7
+ plausible-looking code that doesn't run.
8
+
9
+ Runs on ZeroGPU when the Space has it, and falls back to CPU otherwise.
10
+ """
11
+
12
+ import json
13
+ import os
14
+ import subprocess
15
+ import sys
16
+ import tempfile
17
+ from pathlib import Path
18
+
19
+ import gradio as gr
20
+ import torch
21
+ from transformers import AutoModelForCausalLM, AutoTokenizer
22
+
23
+ from execute import clean_code_string
24
+ from schemas import GENERATORS, df_preview
25
+
26
+ # ZeroGPU: only the decorated function gets a real GPU. Outside it, PyTorch runs
27
+ # in a CUDA emulation mode -- which is why weights are placed on `cuda` in the
28
+ # parent process (see _load) and the decorated function only does the forward
29
+ # pass. The decorated call inherits the parent's state, so the model cache below
30
+ # survives; loading inside the decorator instead would re-download and re-place
31
+ # the weights on every single request and burn the daily GPU quota doing it.
32
+ #
33
+ # On a CPU Space the import fails and everything below runs unchanged on CPU.
34
+ def _duration(label, *_args, **_kwargs):
35
+ """Ask for only as much GPU time as the chosen model plausibly needs."""
36
+ return 45 if "0.5B" in label else 90
37
+
38
+
39
+ try:
40
+ import spaces
41
+
42
+ GPU = spaces.GPU(duration=_duration)
43
+ ZERO_GPU = True
44
+ except Exception: # noqa: BLE001
45
+ def GPU(fn):
46
+ return fn
47
+
48
+ ZERO_GPU = False
49
+
50
+ HERE = Path(__file__).resolve().parent
51
+ GEN_BY_NAME = {g.__name__: g for g in GENERATORS}
52
+ SCHEMAS = sorted(GEN_BY_NAME)
53
+
54
+ # MUST match pipeline/build_dataset.py SYSTEM_PROMPT exactly. Every training
55
+ # example carried this system turn; changing a word moves the prompt off the
56
+ # distribution the adapters were tuned on.
57
+ SYSTEM_PROMPT = (
58
+ "You write Python plotting code for vineyard / viticulture data. "
59
+ "A pandas DataFrame `df` is already loaded. Return only a ```python "
60
+ "code block using matplotlib or seaborn."
61
+ )
62
+
63
+ # label -> (base repo on the Hub, adapter dir in this repo, blurb for the UI)
64
+ MODELS = {
65
+ "Qwen2.5-Coder-0.5B · bf16": (
66
+ "Qwen/Qwen2.5-Coder-0.5B-Instruct",
67
+ "models/qwen2.5-coder-0.5b-plotter-lora",
68
+ "Trained in bf16. Best scorecard of the set: 6/10 semantically correct, "
69
+ "0/10 invented columns, 10/10 `tight_layout()`. Fastest to load.",
70
+ ),
71
+ "Qwen2.5-Coder-1.5B · bf16 (best checkpoint)": (
72
+ "unsloth/Qwen2.5-Coder-1.5B-Instruct",
73
+ "models/qwen2.5-coder-1.5b-plotter-lora-bf16-best",
74
+ "The checkpoint selected on eval loss rather than the final step. "
75
+ "Start here if you want the 1.5B.",
76
+ ),
77
+ "Qwen2.5-Coder-1.5B · bf16 (final step)": (
78
+ "unsloth/Qwen2.5-Coder-1.5B-Instruct",
79
+ "models/qwen2.5-coder-1.5b-plotter-lora-bf16",
80
+ "Same run, saved at the last step instead of the best one.",
81
+ ),
82
+ "Qwen2.5-Coder-1.5B · 4-bit NF4 run": (
83
+ "Qwen/Qwen2.5-Coder-1.5B-Instruct",
84
+ "models/qwen2.5-coder-1.5b-plotter-lora",
85
+ "The original 1.5B, trained under 4-bit NF4 because bf16 would not fit "
86
+ "the training card. Overfit at step 186 (eval loss 0.599 vs 0.480 at "
87
+ "step 100) — kept for comparison. Served here in bf16.",
88
+ ),
89
+ "Phi-3.5-mini-instruct · 3.8B": (
90
+ "microsoft/Phi-3.5-mini-instruct",
91
+ "models/phi35-mini-instruct-lora",
92
+ "Largest base in the set. Slowest to load on a cold Space.",
93
+ ),
94
+ "LFM2-2.6B": (
95
+ "LiquidAI/LFM2-2.6B",
96
+ "models/lfm-2.6b-lora",
97
+ "Liquid AI's hybrid-conv base — a different architecture family from the "
98
+ "others.",
99
+ ),
100
+ }
101
+ DEFAULT_MODEL = "Qwen2.5-Coder-0.5B · bf16"
102
+
103
+ # One model resident at a time: the big bases will not co-exist in Space RAM.
104
+ _LOADED: dict = {"key": None, "model": None, "tokenizer": None}
105
+
106
+
107
+ def _load(label: str, use_adapter: bool):
108
+ """Base model + (optionally) the LoRA adapter, cached across calls."""
109
+ key = (label, use_adapter)
110
+ if _LOADED["key"] == key:
111
+ return _LOADED["model"], _LOADED["tokenizer"]
112
+
113
+ repo, adapter_rel, _ = MODELS[label]
114
+ adapter = HERE / adapter_rel
115
+
116
+ # Evict first -- loading the new weights before freeing the old ones is what
117
+ # actually OOMs the box.
118
+ _LOADED.update(key=None, model=None, tokenizer=None)
119
+ try:
120
+ torch.cuda.empty_cache()
121
+ except Exception: # noqa: BLE001 -- no-op under ZeroGPU's CUDA emulation
122
+ pass
123
+
124
+ # Prefer the tokenizer saved beside the adapter: it carries the exact chat
125
+ # template training used, so prompts here are formatted identically.
126
+ tok_src = str(adapter) if (adapter / "tokenizer_config.json").exists() else repo
127
+ tokenizer = AutoTokenizer.from_pretrained(tok_src)
128
+ if tokenizer.pad_token is None:
129
+ tokenizer.pad_token = tokenizer.eos_token
130
+
131
+ # bf16 on any GPU we can land on; CPU has no usable bf16 matmul path here.
132
+ dtype = torch.bfloat16 if torch.cuda.is_available() else torch.float32
133
+
134
+ model = AutoModelForCausalLM.from_pretrained(repo, dtype=dtype)
135
+ if use_adapter:
136
+ from peft import PeftModel
137
+
138
+ model = PeftModel.from_pretrained(model, str(adapter))
139
+ model.to("cuda" if torch.cuda.is_available() else "cpu")
140
+ model.eval()
141
+
142
+ _LOADED.update(key=key, model=model, tokenizer=tokenizer)
143
+ return model, tokenizer
144
+
145
+
146
+ @GPU
147
+ def _generate(label, preview, request, max_new_tokens, temperature):
148
+ """Forward pass only. _load() must already have run in the parent process."""
149
+ model, tokenizer = _LOADED["model"], _LOADED["tokenizer"]
150
+
151
+ # Same three-part shape build_dataset.py wrote, minus the assistant answer.
152
+ messages = [
153
+ {"role": "system", "content": SYSTEM_PROMPT},
154
+ {
155
+ "role": "user",
156
+ "content": f"### DataFrame Preview:\n{preview}\n\n"
157
+ f"### Visualization Request:\n{request}",
158
+ },
159
+ ]
160
+ prompt = tokenizer.apply_chat_template(
161
+ messages, tokenize=False, add_generation_prompt=True
162
+ )
163
+ inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
164
+
165
+ with torch.no_grad():
166
+ out = model.generate(
167
+ **inputs,
168
+ max_new_tokens=int(max_new_tokens),
169
+ do_sample=temperature > 0,
170
+ temperature=temperature if temperature > 0 else None,
171
+ pad_token_id=tokenizer.pad_token_id,
172
+ )
173
+ return tokenizer.decode(
174
+ out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True
175
+ ).strip()
176
+
177
+
178
+ def _render(schema: str, code: str, seed: int):
179
+ """Execute the generated code out-of-process; return (ok, error, png path)."""
180
+ png = Path(tempfile.mkdtemp(prefix="chart_")) / "chart.png"
181
+ payload = json.dumps(
182
+ {"schema": schema, "code": code, "seed": int(seed), "png": str(png)}
183
+ )
184
+ try:
185
+ proc = subprocess.run(
186
+ [sys.executable, str(HERE / "sandbox_runner.py")],
187
+ input=payload,
188
+ capture_output=True,
189
+ text=True,
190
+ timeout=45,
191
+ cwd=str(HERE),
192
+ env={**os.environ, "MPLBACKEND": "Agg"},
193
+ )
194
+ except subprocess.TimeoutExpired:
195
+ return False, "timed out after 45s (likely an unbounded loop)", None
196
+
197
+ line = (proc.stdout or "").strip().splitlines()
198
+ if not line:
199
+ return False, f"sandbox produced no result. stderr: {(proc.stderr or '')[-400:]}", None
200
+ try:
201
+ res = json.loads(line[-1])
202
+ except json.JSONDecodeError:
203
+ return False, f"unreadable sandbox output: {line[-1][:300]}", None
204
+ return res["ok"], res["error"], res["png"]
205
+
206
+
207
+ def preview_for(schema: str, seed: int) -> str:
208
+ return df_preview(GEN_BY_NAME[schema](seed=int(seed)))
209
+
210
+
211
+ def model_blurb(label: str) -> str:
212
+ repo, adapter, note = MODELS[label]
213
+ return f"**Base:** `{repo}` · **Adapter:** `{adapter}`\n\n{note}"
214
+
215
+
216
+ def run(schema, seed, request, label, use_adapter, max_new_tokens, temperature,
217
+ progress=gr.Progress()):
218
+ request = (request or "").strip()
219
+ if not request:
220
+ return "", "Type a visualization request first.", None
221
+
222
+ preview = preview_for(schema, seed)
223
+
224
+ progress(0.1, desc=f"Loading {label}…")
225
+ try:
226
+ _load(label, use_adapter)
227
+ except Exception as e: # noqa: BLE001
228
+ return "", f"❌ Could not load {label} — {type(e).__name__}: {e}", None
229
+
230
+ progress(0.4, desc="Generating…")
231
+ try:
232
+ reply = _generate(label, preview, request, max_new_tokens, temperature)
233
+ except Exception as e: # noqa: BLE001
234
+ return "", f"❌ Generation failed — {type(e).__name__}: {e}", None
235
+
236
+ code = clean_code_string(reply) or reply
237
+
238
+ progress(0.8, desc="Executing in the sandbox…")
239
+ ok, err, png = _render(schema, code, seed)
240
+ status = "✅ Code executed and produced a chart." if ok else f"❌ Did not run — {err}"
241
+ return code, status, png
242
+
243
+
244
+ EXAMPLES = [
245
+ ["block_vintage_df", "Show average yield per acre by block as a bar chart."],
246
+ ["barrel_aging_df", "Vanillin concentration vs months in barrel, coloured by barrel type."],
247
+ ["cellar_ferment_df", "Plot fermentation temperature over time for each tank."],
248
+ ["irrigation_sensor_df", "Distribution of soil moisture readings by sensor depth."],
249
+ ["wine_sales_df", "Monthly revenue trend broken down by channel."],
250
+ ["berry_chem_df", "Relationship between brix and titratable acidity."],
251
+ ]
252
+
253
+ CSS = """
254
+ .small-note { font-size: 0.85rem; opacity: 0.75; }
255
+ """
256
+
257
+ with gr.Blocks(title="Vineyard Plotting-Code Models") as demo:
258
+ gr.Markdown(
259
+ "# 🍇 Vineyard plotting-code models\n"
260
+ "Six LoRA adapters fine-tuned to write matplotlib/seaborn code against "
261
+ "synthetic vineyard DataFrames. Pick one, pick a DataFrame, ask for a "
262
+ "chart. **The generated code is then executed in a sandbox** and the "
263
+ "chart below is what it actually produced — not a mock-up."
264
+ )
265
+
266
+ with gr.Row():
267
+ with gr.Column(scale=1):
268
+ model = gr.Dropdown(
269
+ sorted(MODELS), value=DEFAULT_MODEL, label="Model"
270
+ )
271
+ blurb = gr.Markdown(model_blurb(DEFAULT_MODEL), elem_classes="small-note")
272
+ use_adapter = gr.Checkbox(
273
+ value=True,
274
+ label="Use the fine-tuned adapter",
275
+ info="Uncheck to run the untuned base model for comparison.",
276
+ )
277
+ schema = gr.Dropdown(
278
+ SCHEMAS, value="block_vintage_df", label="DataFrame"
279
+ )
280
+ seed = gr.Number(value=0, precision=0, label="Seed")
281
+ request = gr.Textbox(
282
+ label="Visualization request",
283
+ placeholder="Show average yield per acre by block as a bar chart.",
284
+ lines=3,
285
+ )
286
+ with gr.Accordion("Decoding", open=False):
287
+ max_new_tokens = gr.Slider(64, 768, value=512, step=32,
288
+ label="Max new tokens")
289
+ temperature = gr.Slider(0.0, 1.0, value=0.2, step=0.05,
290
+ label="Temperature (0 = greedy)")
291
+ go = gr.Button("Generate chart", variant="primary")
292
+
293
+ with gr.Column(scale=1):
294
+ status = gr.Markdown("")
295
+ chart = gr.Image(label="Rendered chart", type="filepath", height=380)
296
+ code_out = gr.Code(label="Generated code", language="python")
297
+
298
+ with gr.Accordion("DataFrame the model is shown", open=False):
299
+ preview = gr.Textbox(
300
+ value=preview_for("block_vintage_df", 0),
301
+ label="Preview (exactly the string passed to the model)",
302
+ lines=10, max_lines=14,
303
+ )
304
+
305
+ gr.Examples(EXAMPLES, inputs=[schema, request], label="Try one")
306
+
307
+ gr.Markdown(
308
+ f"Running on **{'ZeroGPU' if ZERO_GPU else 'CPU'}**. The first request for "
309
+ "a given model downloads its base weights and is slower than the rest.\n\n"
310
+ "Generated code runs with restricted builtins (no `os`, `open`, `eval`, "
311
+ "`subprocess`) in a separate short-lived process — a proportionate guard "
312
+ "against hallucinated code, not a hardened sandbox.",
313
+ elem_classes="small-note",
314
+ )
315
+
316
+ model.change(model_blurb, model, blurb)
317
+ for comp in (schema, seed):
318
+ comp.change(preview_for, [schema, seed], preview)
319
+
320
+ go.click(
321
+ run,
322
+ [schema, seed, request, model, use_adapter, max_new_tokens, temperature],
323
+ [code_out, status, chart],
324
+ )
325
+
326
+ if __name__ == "__main__":
327
+ demo.queue(max_size=12).launch(css=CSS)
execute.py ADDED
@@ -0,0 +1,180 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Sandboxed execution + validation of generated plotting code."""
2
+
3
+ import multiprocessing as mp
4
+ import re
5
+ import matplotlib
6
+
7
+ matplotlib.use("Agg") # Headless mode
8
+ import matplotlib.pyplot as plt
9
+ import numpy as np
10
+ import pandas as pd
11
+
12
+ try:
13
+ import seaborn as sns
14
+ except ImportError:
15
+ sns = None
16
+
17
+ from schemas import GENERATORS
18
+
19
+ # ---------------------------------------------------------------------------
20
+ # Execution guard
21
+ # ---------------------------------------------------------------------------
22
+ # The code we exec() here is machine-generated (Gemini teacher, and later the
23
+ # fine-tuned student). It is NOT trusted. Running it with the full builtins and
24
+ # unrestricted imports would let a hallucinated snippet run `import os;
25
+ # os.system(...)`, delete files, or open network sockets on this machine.
26
+ #
27
+ # This is a *proportionate* guard against accidental / hallucinated harm, not a
28
+ # hardened sandbox: it removes the obvious escape hatches (open/eval/exec/compile
29
+ # and imports of os/subprocess/socket/etc.) while still allowing normal plotting
30
+ # code. For genuinely adversarial code, run this inside a container or VM.
31
+
32
+ # Top-level modules a plotting snippet is allowed to import.
33
+ _ALLOWED_IMPORTS = {
34
+ "matplotlib", "mpl_toolkits", "pandas", "numpy", "seaborn",
35
+ "scipy", "math", "statistics", "datetime", "collections",
36
+ "itertools", "functools", "warnings", "random", "cycler",
37
+ "numbers", "decimal", "fractions",
38
+ }
39
+
40
+ # Builtins safe to expose. Deliberately omits open, eval, exec, compile, input,
41
+ # __import__ (replaced below), globals, locals, vars, exit, quit, help,
42
+ # breakpoint, memoryview.
43
+ _SAFE_BUILTIN_NAMES = (
44
+ "abs", "all", "any", "bool", "bytes", "callable", "chr", "classmethod",
45
+ "complex", "dict", "divmod", "enumerate", "filter", "float", "format",
46
+ "frozenset", "getattr", "hasattr", "hash", "hex", "int", "isinstance",
47
+ "issubclass", "iter", "len", "list", "map", "max", "min", "next", "object",
48
+ "oct", "ord", "pow", "print", "property", "range", "repr", "reversed",
49
+ "round", "set", "setattr", "slice", "sorted", "staticmethod", "str", "sum",
50
+ "super", "tuple", "type", "zip", "True", "False", "None",
51
+ "__build_class__", # needed for class definitions in generated code
52
+ )
53
+
54
+
55
+ def _guarded_import(name, globals=None, locals=None, fromlist=(), level=0):
56
+ """Allow imports only for the whitelisted data/plotting modules."""
57
+ root = name.split(".")[0]
58
+ if root not in _ALLOWED_IMPORTS:
59
+ raise ImportError(f"import of '{name}' is blocked in the execution sandbox")
60
+ return __import__(name, globals, locals, fromlist, level)
61
+
62
+
63
+ def _safe_builtins() -> dict:
64
+ import builtins as _b
65
+ safe = {n: getattr(_b, n) for n in _SAFE_BUILTIN_NAMES if hasattr(_b, n)}
66
+ safe["__import__"] = _guarded_import
67
+ return safe
68
+
69
+
70
+ GENERATOR_MAP = (
71
+ {g.__name__: g if callable(g) else g for g in GENERATORS}
72
+ if isinstance(GENERATORS, (list, tuple, set))
73
+ else GENERATORS
74
+ )
75
+
76
+
77
+ def clean_code_string(code: str) -> str:
78
+ """Strip markdown code blocks from generated code safely."""
79
+ if not code:
80
+ return ""
81
+
82
+ code = code.strip()
83
+ bt3 = chr(96) * 3 # Dynamically generates triple backticks to avoid UI glitches
84
+
85
+ # Extract content inside markdown python code block if present
86
+ pattern = bt3 + r"(?:python)?\s*\n?(.*?)\n?" + bt3
87
+ match = re.search(pattern, code, flags=re.DOTALL | re.IGNORECASE)
88
+
89
+ if match:
90
+ cleaned = match.group(1).strip()
91
+ else:
92
+ # Fallback: strip leading/trailing backtick lines
93
+ lines = code.splitlines()
94
+ if lines and lines[0].strip().startswith(bt3):
95
+ lines = lines[1:]
96
+ if lines and lines[-1].strip().startswith(bt3):
97
+ lines = lines[:-1]
98
+ cleaned = "\n".join(lines).strip()
99
+
100
+ return cleaned
101
+
102
+
103
+ def _run(gen_name: str, code: str, q: mp.Queue, seed: int = 0) -> None:
104
+ # Generated code triggers a lot of library deprecation chatter (seaborn
105
+ # palette/hue, pandas futures). It is not our code and we cannot fix it, so
106
+ # keep it out of the validation log -- real failures come back via the queue.
107
+ import warnings
108
+ warnings.filterwarnings("ignore")
109
+ try:
110
+ if callable(gen_name):
111
+ gen_func = gen_name
112
+ elif gen_name in GENERATOR_MAP:
113
+ gen_func = GENERATOR_MAP[gen_name]
114
+ else:
115
+ q.put(("error", f"Generator '{gen_name}' not found."))
116
+ return
117
+
118
+ clean_code = clean_code_string(code)
119
+ if not clean_code:
120
+ q.put(("error", "Empty code block after stripping formatting."))
121
+ return
122
+
123
+ # Same (generator, seed) the teacher saw -> the code is validated against
124
+ # the exact DataFrame whose preview is stored on the record.
125
+ df = gen_func(seed=seed)
126
+
127
+ ns = {
128
+ "df": df,
129
+ "plt": plt,
130
+ "pd": pd,
131
+ "np": np,
132
+ "matplotlib": matplotlib,
133
+ "__builtins__": _safe_builtins(), # restricted: no os/open/eval/etc.
134
+ }
135
+ if sns is not None:
136
+ ns["sns"] = sns
137
+
138
+ plt.close("all")
139
+ exec(clean_code, ns)
140
+
141
+ fignums = plt.get_fignums()
142
+ if not fignums:
143
+ q.put(("no_figure", "No matplotlib figure was created."))
144
+ return
145
+
146
+ fig = plt.gcf()
147
+ has_axes = len(fig.axes) > 0 and any(len(ax.get_children()) > 0 for ax in fig.axes)
148
+ plt.close("all")
149
+
150
+ if has_axes:
151
+ q.put(("ok", None))
152
+ else:
153
+ q.put(("no_figure", "Figure created but contains no axes or visual elements."))
154
+ except Exception as e:
155
+ plt.close("all")
156
+ q.put(("error", f"{type(e).__name__}: {str(e)}"))
157
+
158
+
159
+ def validate(gen_name: str, code: str, timeout: int = 10, seed: int = 0) -> tuple[bool, str | None]:
160
+ if not code:
161
+ return False, "empty code"
162
+
163
+ q = mp.Queue()
164
+ p = mp.Process(target=_run, args=(gen_name, code, q, seed))
165
+ p.start()
166
+ p.join(timeout)
167
+
168
+ if p.is_alive():
169
+ p.terminate()
170
+ p.join(timeout=2)
171
+ if p.is_alive():
172
+ p.kill()
173
+ p.join()
174
+ return False, f"Timeout after {timeout} seconds"
175
+
176
+ if q.empty():
177
+ return False, "Process terminated unexpectedly with no output"
178
+
179
+ status, detail = q.get()
180
+ return status == "ok", detail
models/lfm-2.6b-lora/adapter_config.json ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "LiquidAI/LFM2-2.6B",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "layer_replication": null,
16
+ "layers_pattern": null,
17
+ "layers_to_transform": null,
18
+ "loftq_config": {},
19
+ "lora_alpha": 32,
20
+ "lora_bias": false,
21
+ "lora_dropout": 0.05,
22
+ "lora_ga_config": null,
23
+ "megatron_config": null,
24
+ "megatron_core": "megatron.core",
25
+ "modules_to_save": null,
26
+ "monteclora_config": null,
27
+ "peft_type": "LORA",
28
+ "peft_version": "0.20.0",
29
+ "qalora_group_size": 16,
30
+ "r": 16,
31
+ "rank_pattern": {},
32
+ "revision": null,
33
+ "target_modules": [
34
+ "w2",
35
+ "v_proj",
36
+ "q_proj",
37
+ "out_proj",
38
+ "k_proj",
39
+ "in_proj",
40
+ "w1",
41
+ "w3"
42
+ ],
43
+ "target_parameters": null,
44
+ "task_type": "CAUSAL_LM",
45
+ "trainable_token_indices": null,
46
+ "use_bdlora": null,
47
+ "use_dora": false,
48
+ "use_qalora": false,
49
+ "use_rslora": false,
50
+ "velora_config": null
51
+ }
models/lfm-2.6b-lora/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:019e3d4f771171e4796551056daa6b61bf36904d1c8d0b923457caa5d34dc2c2
3
+ size 97889536
models/lfm-2.6b-lora/chat_template.jinja ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {{- bos_token -}}{%- set system_prompt = "" -%}{%- set ns = namespace(system_prompt="") -%}{%- if messages[0]["role"] == "system" -%} {%- set ns.system_prompt = messages[0]["content"] -%} {%- set messages = messages[1:] -%}{%- endif -%}{%- if tools -%} {%- set ns.system_prompt = ns.system_prompt + ("
2
+ " if ns.system_prompt else "") + "List of tools: <|tool_list_start|>[" -%} {%- for tool in tools -%} {%- if tool is not string -%} {%- set tool = tool | tojson -%} {%- endif -%} {%- set ns.system_prompt = ns.system_prompt + tool -%} {%- if not loop.last -%} {%- set ns.system_prompt = ns.system_prompt + ", " -%} {%- endif -%} {%- endfor -%} {%- set ns.system_prompt = ns.system_prompt + "]<|tool_list_end|>" -%}{%- endif -%}{%- if ns.system_prompt -%} {{- "<|im_start|>system
3
+ " + ns.system_prompt + "<|im_end|>
4
+ " -}}{%- endif -%}{%- for message in messages -%} {{- "<|im_start|>" + message["role"] + "
5
+ " -}} {%- set content = message["content"] -%} {%- if content is not string -%} {%- set content = content | tojson -%} {%- endif -%} {%- if message["role"] == "tool" -%} {%- set content = "<|tool_response_start|>" + content + "<|tool_response_end|>" -%} {%- endif -%} {{- content + "<|im_end|>
6
+ " -}}{%- endfor -%}{%- if add_generation_prompt -%} {{- "<|im_start|>assistant
7
+ " -}}{%- endif -%}
models/lfm-2.6b-lora/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
models/lfm-2.6b-lora/tokenizer_config.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "backend": "tokenizers",
3
+ "bos_token": "<|startoftext|>",
4
+ "clean_up_tokenization_spaces": false,
5
+ "eos_token": "<|im_end|>",
6
+ "is_local": false,
7
+ "legacy": true,
8
+ "local_files_only": false,
9
+ "model_input_names": [
10
+ "input_ids",
11
+ "attention_mask"
12
+ ],
13
+ "model_max_length": 1000000000000000019884624838656,
14
+ "pad_token": "<|pad|>",
15
+ "sp_model_kwargs": {},
16
+ "spaces_between_special_tokens": false,
17
+ "tokenizer_class": "TokenizersBackend",
18
+ "use_default_system_prompt": false,
19
+ "use_fast": true
20
+ }
models/phi35-mini-instruct-lora/adapter_config.json ADDED
@@ -0,0 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "microsoft/Phi-3.5-mini-instruct",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "layer_replication": null,
16
+ "layers_pattern": null,
17
+ "layers_to_transform": null,
18
+ "loftq_config": {},
19
+ "lora_alpha": 32,
20
+ "lora_bias": false,
21
+ "lora_dropout": 0.05,
22
+ "lora_ga_config": null,
23
+ "megatron_config": null,
24
+ "megatron_core": "megatron.core",
25
+ "modules_to_save": null,
26
+ "monteclora_config": null,
27
+ "peft_type": "LORA",
28
+ "peft_version": "0.20.0",
29
+ "qalora_group_size": 16,
30
+ "r": 16,
31
+ "rank_pattern": {},
32
+ "revision": null,
33
+ "target_modules": [
34
+ "qkv_proj",
35
+ "o_proj",
36
+ "gate_up_proj",
37
+ "down_proj"
38
+ ],
39
+ "target_parameters": null,
40
+ "task_type": "CAUSAL_LM",
41
+ "trainable_token_indices": null,
42
+ "use_bdlora": null,
43
+ "use_dora": false,
44
+ "use_qalora": false,
45
+ "use_rslora": false,
46
+ "velora_config": null
47
+ }
models/phi35-mini-instruct-lora/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e56d14e2dd4b10649767db1068d1abbae6fb5e8036fec689d2b27fe616da54a0
3
+ size 100697728
models/phi35-mini-instruct-lora/chat_template.jinja ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {% for message in messages %}{% if message['role'] == 'system' and message['content'] %}{{'<|system|>
2
+ ' + message['content'] + '<|end|>
3
+ '}}{% elif message['role'] == 'user' %}{{'<|user|>
4
+ ' + message['content'] + '<|end|>
5
+ '}}{% elif message['role'] == 'assistant' %}{{'<|assistant|>
6
+ ' + message['content'] + '<|end|>
7
+ '}}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ '<|assistant|>
8
+ ' }}{% else %}{{ eos_token }}{% endif %}
models/phi35-mini-instruct-lora/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
models/phi35-mini-instruct-lora/tokenizer_config.json ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "backend": "tokenizers",
3
+ "bos_token": "<s>",
4
+ "clean_up_tokenization_spaces": false,
5
+ "eos_token": "<|endoftext|>",
6
+ "is_local": false,
7
+ "legacy": false,
8
+ "local_files_only": false,
9
+ "model_max_length": 131072,
10
+ "pad_token": "<|endoftext|>",
11
+ "padding_side": "left",
12
+ "sp_model_kwargs": {},
13
+ "tokenizer_class": "TokenizersBackend",
14
+ "unk_token": "<unk>",
15
+ "use_default_system_prompt": false
16
+ }
models/qwen2.5-coder-0.5b-plotter-lora/adapter_config.json ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "Qwen/Qwen2.5-Coder-0.5B-Instruct",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "layer_replication": null,
16
+ "layers_pattern": null,
17
+ "layers_to_transform": null,
18
+ "loftq_config": {},
19
+ "lora_alpha": 16,
20
+ "lora_bias": false,
21
+ "lora_dropout": 0.0,
22
+ "lora_ga_config": null,
23
+ "megatron_config": null,
24
+ "megatron_core": "megatron.core",
25
+ "modules_to_save": null,
26
+ "monteclora_config": null,
27
+ "peft_type": "LORA",
28
+ "peft_version": "0.20.0",
29
+ "qalora_group_size": 16,
30
+ "r": 16,
31
+ "rank_pattern": {},
32
+ "revision": null,
33
+ "target_modules": [
34
+ "v_proj",
35
+ "down_proj",
36
+ "up_proj",
37
+ "gate_proj",
38
+ "q_proj",
39
+ "o_proj",
40
+ "k_proj"
41
+ ],
42
+ "target_parameters": null,
43
+ "task_type": "CAUSAL_LM",
44
+ "trainable_token_indices": null,
45
+ "use_bdlora": null,
46
+ "use_dora": false,
47
+ "use_qalora": false,
48
+ "use_rslora": false,
49
+ "velora_config": null
50
+ }
models/qwen2.5-coder-0.5b-plotter-lora/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:74d19b0e08af9b3ec3b0d5cac0779243a754dec11e6cd7bdcd7cfd98d6f602cf
3
+ size 35237104
models/qwen2.5-coder-0.5b-plotter-lora/chat_template.jinja ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- if tools %}
2
+ {{- '<|im_start|>system\n' }}
3
+ {%- if messages[0]['role'] == 'system' %}
4
+ {{- messages[0]['content'] }}
5
+ {%- else %}
6
+ {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
7
+ {%- endif %}
8
+ {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
9
+ {%- for tool in tools %}
10
+ {{- "\n" }}
11
+ {{- tool | tojson }}
12
+ {%- endfor %}
13
+ {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
14
+ {%- else %}
15
+ {%- if messages[0]['role'] == 'system' %}
16
+ {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
17
+ {%- else %}
18
+ {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
19
+ {%- endif %}
20
+ {%- endif %}
21
+ {%- for message in messages %}
22
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
23
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
24
+ {%- elif message.role == "assistant" %}
25
+ {{- '<|im_start|>' + message.role }}
26
+ {%- if message.content %}
27
+ {{- '\n' + message.content }}
28
+ {%- endif %}
29
+ {%- for tool_call in message.tool_calls %}
30
+ {%- if tool_call.function is defined %}
31
+ {%- set tool_call = tool_call.function %}
32
+ {%- endif %}
33
+ {{- '\n<tool_call>\n{"name": "' }}
34
+ {{- tool_call.name }}
35
+ {{- '", "arguments": ' }}
36
+ {{- tool_call.arguments | tojson }}
37
+ {{- '}\n</tool_call>' }}
38
+ {%- endfor %}
39
+ {{- '<|im_end|>\n' }}
40
+ {%- elif message.role == "tool" %}
41
+ {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
42
+ {{- '<|im_start|>user' }}
43
+ {%- endif %}
44
+ {{- '\n<tool_response>\n' }}
45
+ {{- message.content }}
46
+ {{- '\n</tool_response>' }}
47
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
48
+ {{- '<|im_end|>\n' }}
49
+ {%- endif %}
50
+ {%- endif %}
51
+ {%- endfor %}
52
+ {%- if add_generation_prompt %}
53
+ {{- '<|im_start|>assistant\n' }}
54
+ {%- endif %}
models/qwen2.5-coder-0.5b-plotter-lora/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
3
+ size 11421892
models/qwen2.5-coder-0.5b-plotter-lora/tokenizer_config.json ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": null,
5
+ "clean_up_tokenization_spaces": false,
6
+ "eos_token": "<|im_end|>",
7
+ "errors": "replace",
8
+ "extra_special_tokens": [
9
+ "<|im_start|>",
10
+ "<|im_end|>",
11
+ "<|object_ref_start|>",
12
+ "<|object_ref_end|>",
13
+ "<|box_start|>",
14
+ "<|box_end|>",
15
+ "<|quad_start|>",
16
+ "<|quad_end|>",
17
+ "<|vision_start|>",
18
+ "<|vision_end|>",
19
+ "<|vision_pad|>",
20
+ "<|image_pad|>",
21
+ "<|video_pad|>"
22
+ ],
23
+ "is_local": false,
24
+ "model_max_length": 32768,
25
+ "pad_token": "<|endoftext|>",
26
+ "split_special_tokens": false,
27
+ "tokenizer_class": "Qwen2Tokenizer",
28
+ "unk_token": null
29
+ }
models/qwen2.5-coder-1.5b-plotter-lora-bf16-best/adapter_config.json ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": {
6
+ "base_model_class": "Qwen2ForCausalLM",
7
+ "parent_library": "transformers.models.qwen2.modeling_qwen2",
8
+ "unsloth_fixed": true
9
+ },
10
+ "base_model_name_or_path": "unsloth/Qwen2.5-Coder-1.5B-Instruct",
11
+ "bias": "none",
12
+ "corda_config": null,
13
+ "ensure_weight_tying": false,
14
+ "eva_config": null,
15
+ "exclude_modules": null,
16
+ "fan_in_fan_out": false,
17
+ "inference_mode": true,
18
+ "init_lora_weights": true,
19
+ "layer_replication": null,
20
+ "layers_pattern": null,
21
+ "layers_to_transform": null,
22
+ "loftq_config": {},
23
+ "lora_alpha": 16,
24
+ "lora_bias": false,
25
+ "lora_dropout": 0.0,
26
+ "lora_ga_config": null,
27
+ "megatron_config": null,
28
+ "megatron_core": "megatron.core",
29
+ "modules_to_save": null,
30
+ "monteclora_config": null,
31
+ "peft_type": "LORA",
32
+ "peft_version": "0.20.0",
33
+ "qalora_group_size": 16,
34
+ "r": 16,
35
+ "rank_pattern": {},
36
+ "revision": null,
37
+ "target_modules": [
38
+ "up_proj",
39
+ "gate_proj",
40
+ "v_proj",
41
+ "q_proj",
42
+ "o_proj",
43
+ "down_proj",
44
+ "k_proj"
45
+ ],
46
+ "target_parameters": null,
47
+ "task_type": "CAUSAL_LM",
48
+ "trainable_token_indices": null,
49
+ "use_bdlora": null,
50
+ "use_dora": false,
51
+ "use_qalora": false,
52
+ "use_rslora": false,
53
+ "velora_config": null
54
+ }
models/qwen2.5-coder-1.5b-plotter-lora-bf16-best/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b48bd5cf9332b2af8815dfcd1ea30ed662b5c2d6b0034d71137a5cca4cc7d0d9
3
+ size 73911112
models/qwen2.5-coder-1.5b-plotter-lora-bf16-best/chat_template.jinja ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- if tools %}
2
+ {{- '<|im_start|>system\n' }}
3
+ {%- if messages[0]['role'] == 'system' %}
4
+ {{- messages[0]['content'] }}
5
+ {%- else %}
6
+ {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
7
+ {%- endif %}
8
+ {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
9
+ {%- for tool in tools %}
10
+ {{- "\n" }}
11
+ {{- tool | tojson }}
12
+ {%- endfor %}
13
+ {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
14
+ {%- else %}
15
+ {%- if messages[0]['role'] == 'system' %}
16
+ {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
17
+ {%- else %}
18
+ {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
19
+ {%- endif %}
20
+ {%- endif %}
21
+ {%- for message in messages %}
22
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
23
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
24
+ {%- elif message.role == "assistant" %}
25
+ {{- '<|im_start|>' + message.role }}
26
+ {%- if message.content %}
27
+ {{- '\n' + message.content }}
28
+ {%- endif %}
29
+ {%- for tool_call in message.tool_calls %}
30
+ {%- if tool_call.function is defined %}
31
+ {%- set tool_call = tool_call.function %}
32
+ {%- endif %}
33
+ {{- '\n<tool_call>\n{"name": "' }}
34
+ {{- tool_call.name }}
35
+ {{- '", "arguments": ' }}
36
+ {{- tool_call.arguments | tojson }}
37
+ {{- '}\n</tool_call>' }}
38
+ {%- endfor %}
39
+ {{- '<|im_end|>\n' }}
40
+ {%- elif message.role == "tool" %}
41
+ {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
42
+ {{- '<|im_start|>user' }}
43
+ {%- endif %}
44
+ {{- '\n<tool_response>\n' }}
45
+ {{- message.content }}
46
+ {{- '\n</tool_response>' }}
47
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
48
+ {{- '<|im_end|>\n' }}
49
+ {%- endif %}
50
+ {%- endif %}
51
+ {%- endfor %}
52
+ {%- if add_generation_prompt %}
53
+ {{- '<|im_start|>assistant\n' }}
54
+ {%- endif %}
models/qwen2.5-coder-1.5b-plotter-lora-bf16-best/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ea43b288542655d72d632195ab9b58ca2cd9532c292bf6667827ce899ad196bc
3
+ size 11422082
models/qwen2.5-coder-1.5b-plotter-lora-bf16-best/tokenizer_config.json ADDED
@@ -0,0 +1,216 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": null,
5
+ "clean_up_tokenization_spaces": false,
6
+ "eos_token": "<|im_end|>",
7
+ "errors": "replace",
8
+ "extra_special_tokens": [
9
+ "<|im_start|>",
10
+ "<|im_end|>",
11
+ "<|object_ref_start|>",
12
+ "<|object_ref_end|>",
13
+ "<|box_start|>",
14
+ "<|box_end|>",
15
+ "<|quad_start|>",
16
+ "<|quad_end|>",
17
+ "<|vision_start|>",
18
+ "<|vision_end|>",
19
+ "<|vision_pad|>",
20
+ "<|image_pad|>",
21
+ "<|video_pad|>"
22
+ ],
23
+ "is_local": false,
24
+ "model_max_length": 32768,
25
+ "pad_token": "<|PAD_TOKEN|>",
26
+ "padding_side": "left",
27
+ "split_special_tokens": false,
28
+ "tokenizer_class": "Qwen2Tokenizer",
29
+ "unk_token": null,
30
+ "added_tokens_decoder": {
31
+ "151643": {
32
+ "content": "<|endoftext|>",
33
+ "single_word": false,
34
+ "lstrip": false,
35
+ "rstrip": false,
36
+ "normalized": false,
37
+ "special": true
38
+ },
39
+ "151644": {
40
+ "content": "<|im_start|>",
41
+ "single_word": false,
42
+ "lstrip": false,
43
+ "rstrip": false,
44
+ "normalized": false,
45
+ "special": true
46
+ },
47
+ "151645": {
48
+ "content": "<|im_end|>",
49
+ "single_word": false,
50
+ "lstrip": false,
51
+ "rstrip": false,
52
+ "normalized": false,
53
+ "special": true
54
+ },
55
+ "151646": {
56
+ "content": "<|object_ref_start|>",
57
+ "single_word": false,
58
+ "lstrip": false,
59
+ "rstrip": false,
60
+ "normalized": false,
61
+ "special": true
62
+ },
63
+ "151647": {
64
+ "content": "<|object_ref_end|>",
65
+ "single_word": false,
66
+ "lstrip": false,
67
+ "rstrip": false,
68
+ "normalized": false,
69
+ "special": true
70
+ },
71
+ "151648": {
72
+ "content": "<|box_start|>",
73
+ "single_word": false,
74
+ "lstrip": false,
75
+ "rstrip": false,
76
+ "normalized": false,
77
+ "special": true
78
+ },
79
+ "151649": {
80
+ "content": "<|box_end|>",
81
+ "single_word": false,
82
+ "lstrip": false,
83
+ "rstrip": false,
84
+ "normalized": false,
85
+ "special": true
86
+ },
87
+ "151650": {
88
+ "content": "<|quad_start|>",
89
+ "single_word": false,
90
+ "lstrip": false,
91
+ "rstrip": false,
92
+ "normalized": false,
93
+ "special": true
94
+ },
95
+ "151651": {
96
+ "content": "<|quad_end|>",
97
+ "single_word": false,
98
+ "lstrip": false,
99
+ "rstrip": false,
100
+ "normalized": false,
101
+ "special": true
102
+ },
103
+ "151652": {
104
+ "content": "<|vision_start|>",
105
+ "single_word": false,
106
+ "lstrip": false,
107
+ "rstrip": false,
108
+ "normalized": false,
109
+ "special": true
110
+ },
111
+ "151653": {
112
+ "content": "<|vision_end|>",
113
+ "single_word": false,
114
+ "lstrip": false,
115
+ "rstrip": false,
116
+ "normalized": false,
117
+ "special": true
118
+ },
119
+ "151654": {
120
+ "content": "<|vision_pad|>",
121
+ "single_word": false,
122
+ "lstrip": false,
123
+ "rstrip": false,
124
+ "normalized": false,
125
+ "special": true
126
+ },
127
+ "151655": {
128
+ "content": "<|image_pad|>",
129
+ "single_word": false,
130
+ "lstrip": false,
131
+ "rstrip": false,
132
+ "normalized": false,
133
+ "special": true
134
+ },
135
+ "151656": {
136
+ "content": "<|video_pad|>",
137
+ "single_word": false,
138
+ "lstrip": false,
139
+ "rstrip": false,
140
+ "normalized": false,
141
+ "special": true
142
+ },
143
+ "151657": {
144
+ "content": "<tool_call>",
145
+ "single_word": false,
146
+ "lstrip": false,
147
+ "rstrip": false,
148
+ "normalized": false,
149
+ "special": false
150
+ },
151
+ "151658": {
152
+ "content": "</tool_call>",
153
+ "single_word": false,
154
+ "lstrip": false,
155
+ "rstrip": false,
156
+ "normalized": false,
157
+ "special": false
158
+ },
159
+ "151659": {
160
+ "content": "<|fim_prefix|>",
161
+ "single_word": false,
162
+ "lstrip": false,
163
+ "rstrip": false,
164
+ "normalized": false,
165
+ "special": false
166
+ },
167
+ "151660": {
168
+ "content": "<|fim_middle|>",
169
+ "single_word": false,
170
+ "lstrip": false,
171
+ "rstrip": false,
172
+ "normalized": false,
173
+ "special": false
174
+ },
175
+ "151661": {
176
+ "content": "<|fim_suffix|>",
177
+ "single_word": false,
178
+ "lstrip": false,
179
+ "rstrip": false,
180
+ "normalized": false,
181
+ "special": false
182
+ },
183
+ "151662": {
184
+ "content": "<|fim_pad|>",
185
+ "single_word": false,
186
+ "lstrip": false,
187
+ "rstrip": false,
188
+ "normalized": false,
189
+ "special": false
190
+ },
191
+ "151663": {
192
+ "content": "<|repo_name|>",
193
+ "single_word": false,
194
+ "lstrip": false,
195
+ "rstrip": false,
196
+ "normalized": false,
197
+ "special": false
198
+ },
199
+ "151664": {
200
+ "content": "<|file_sep|>",
201
+ "single_word": false,
202
+ "lstrip": false,
203
+ "rstrip": false,
204
+ "normalized": false,
205
+ "special": false
206
+ },
207
+ "151665": {
208
+ "content": "<|PAD_TOKEN|>",
209
+ "single_word": false,
210
+ "lstrip": false,
211
+ "rstrip": false,
212
+ "normalized": false,
213
+ "special": true
214
+ }
215
+ }
216
+ }
models/qwen2.5-coder-1.5b-plotter-lora-bf16/adapter_config.json ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": {
6
+ "base_model_class": "Qwen2ForCausalLM",
7
+ "parent_library": "transformers.models.qwen2.modeling_qwen2",
8
+ "unsloth_fixed": true
9
+ },
10
+ "base_model_name_or_path": "unsloth/Qwen2.5-Coder-1.5B-Instruct",
11
+ "bias": "none",
12
+ "corda_config": null,
13
+ "ensure_weight_tying": false,
14
+ "eva_config": null,
15
+ "exclude_modules": null,
16
+ "fan_in_fan_out": false,
17
+ "inference_mode": true,
18
+ "init_lora_weights": true,
19
+ "layer_replication": null,
20
+ "layers_pattern": null,
21
+ "layers_to_transform": null,
22
+ "loftq_config": {},
23
+ "lora_alpha": 16,
24
+ "lora_bias": false,
25
+ "lora_dropout": 0,
26
+ "lora_ga_config": null,
27
+ "megatron_config": null,
28
+ "megatron_core": "megatron.core",
29
+ "modules_to_save": null,
30
+ "monteclora_config": null,
31
+ "peft_type": "LORA",
32
+ "peft_version": "0.20.0",
33
+ "qalora_group_size": 16,
34
+ "r": 16,
35
+ "rank_pattern": {},
36
+ "revision": null,
37
+ "target_modules": [
38
+ "v_proj",
39
+ "down_proj",
40
+ "up_proj",
41
+ "gate_proj",
42
+ "o_proj",
43
+ "k_proj",
44
+ "q_proj"
45
+ ],
46
+ "target_parameters": null,
47
+ "task_type": "CAUSAL_LM",
48
+ "trainable_token_indices": null,
49
+ "use_bdlora": null,
50
+ "use_dora": false,
51
+ "use_qalora": false,
52
+ "use_rslora": false,
53
+ "velora_config": null
54
+ }
models/qwen2.5-coder-1.5b-plotter-lora-bf16/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9b296635344c623b5987481f73f52ea51d411fbb319b555b55f9d4ba633c194a
3
+ size 73911112
models/qwen2.5-coder-1.5b-plotter-lora-bf16/chat_template.jinja ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- if tools %}
2
+ {{- '<|im_start|>system\n' }}
3
+ {%- if messages[0]['role'] == 'system' %}
4
+ {{- messages[0]['content'] }}
5
+ {%- else %}
6
+ {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
7
+ {%- endif %}
8
+ {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
9
+ {%- for tool in tools %}
10
+ {{- "\n" }}
11
+ {{- tool | tojson }}
12
+ {%- endfor %}
13
+ {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
14
+ {%- else %}
15
+ {%- if messages[0]['role'] == 'system' %}
16
+ {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
17
+ {%- else %}
18
+ {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
19
+ {%- endif %}
20
+ {%- endif %}
21
+ {%- for message in messages %}
22
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
23
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
24
+ {%- elif message.role == "assistant" %}
25
+ {{- '<|im_start|>' + message.role }}
26
+ {%- if message.content %}
27
+ {{- '\n' + message.content }}
28
+ {%- endif %}
29
+ {%- for tool_call in message.tool_calls %}
30
+ {%- if tool_call.function is defined %}
31
+ {%- set tool_call = tool_call.function %}
32
+ {%- endif %}
33
+ {{- '\n<tool_call>\n{"name": "' }}
34
+ {{- tool_call.name }}
35
+ {{- '", "arguments": ' }}
36
+ {{- tool_call.arguments | tojson }}
37
+ {{- '}\n</tool_call>' }}
38
+ {%- endfor %}
39
+ {{- '<|im_end|>\n' }}
40
+ {%- elif message.role == "tool" %}
41
+ {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
42
+ {{- '<|im_start|>user' }}
43
+ {%- endif %}
44
+ {{- '\n<tool_response>\n' }}
45
+ {{- message.content }}
46
+ {{- '\n</tool_response>' }}
47
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
48
+ {{- '<|im_end|>\n' }}
49
+ {%- endif %}
50
+ {%- endif %}
51
+ {%- endfor %}
52
+ {%- if add_generation_prompt %}
53
+ {{- '<|im_start|>assistant\n' }}
54
+ {%- endif %}
models/qwen2.5-coder-1.5b-plotter-lora-bf16/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ea43b288542655d72d632195ab9b58ca2cd9532c292bf6667827ce899ad196bc
3
+ size 11422082
models/qwen2.5-coder-1.5b-plotter-lora-bf16/tokenizer_config.json ADDED
@@ -0,0 +1,216 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": null,
5
+ "clean_up_tokenization_spaces": false,
6
+ "eos_token": "<|im_end|>",
7
+ "errors": "replace",
8
+ "extra_special_tokens": [
9
+ "<|im_start|>",
10
+ "<|im_end|>",
11
+ "<|object_ref_start|>",
12
+ "<|object_ref_end|>",
13
+ "<|box_start|>",
14
+ "<|box_end|>",
15
+ "<|quad_start|>",
16
+ "<|quad_end|>",
17
+ "<|vision_start|>",
18
+ "<|vision_end|>",
19
+ "<|vision_pad|>",
20
+ "<|image_pad|>",
21
+ "<|video_pad|>"
22
+ ],
23
+ "is_local": false,
24
+ "model_max_length": 32768,
25
+ "pad_token": "<|PAD_TOKEN|>",
26
+ "padding_side": "left",
27
+ "split_special_tokens": false,
28
+ "tokenizer_class": "Qwen2Tokenizer",
29
+ "unk_token": null,
30
+ "added_tokens_decoder": {
31
+ "151643": {
32
+ "content": "<|endoftext|>",
33
+ "single_word": false,
34
+ "lstrip": false,
35
+ "rstrip": false,
36
+ "normalized": false,
37
+ "special": true
38
+ },
39
+ "151644": {
40
+ "content": "<|im_start|>",
41
+ "single_word": false,
42
+ "lstrip": false,
43
+ "rstrip": false,
44
+ "normalized": false,
45
+ "special": true
46
+ },
47
+ "151645": {
48
+ "content": "<|im_end|>",
49
+ "single_word": false,
50
+ "lstrip": false,
51
+ "rstrip": false,
52
+ "normalized": false,
53
+ "special": true
54
+ },
55
+ "151646": {
56
+ "content": "<|object_ref_start|>",
57
+ "single_word": false,
58
+ "lstrip": false,
59
+ "rstrip": false,
60
+ "normalized": false,
61
+ "special": true
62
+ },
63
+ "151647": {
64
+ "content": "<|object_ref_end|>",
65
+ "single_word": false,
66
+ "lstrip": false,
67
+ "rstrip": false,
68
+ "normalized": false,
69
+ "special": true
70
+ },
71
+ "151648": {
72
+ "content": "<|box_start|>",
73
+ "single_word": false,
74
+ "lstrip": false,
75
+ "rstrip": false,
76
+ "normalized": false,
77
+ "special": true
78
+ },
79
+ "151649": {
80
+ "content": "<|box_end|>",
81
+ "single_word": false,
82
+ "lstrip": false,
83
+ "rstrip": false,
84
+ "normalized": false,
85
+ "special": true
86
+ },
87
+ "151650": {
88
+ "content": "<|quad_start|>",
89
+ "single_word": false,
90
+ "lstrip": false,
91
+ "rstrip": false,
92
+ "normalized": false,
93
+ "special": true
94
+ },
95
+ "151651": {
96
+ "content": "<|quad_end|>",
97
+ "single_word": false,
98
+ "lstrip": false,
99
+ "rstrip": false,
100
+ "normalized": false,
101
+ "special": true
102
+ },
103
+ "151652": {
104
+ "content": "<|vision_start|>",
105
+ "single_word": false,
106
+ "lstrip": false,
107
+ "rstrip": false,
108
+ "normalized": false,
109
+ "special": true
110
+ },
111
+ "151653": {
112
+ "content": "<|vision_end|>",
113
+ "single_word": false,
114
+ "lstrip": false,
115
+ "rstrip": false,
116
+ "normalized": false,
117
+ "special": true
118
+ },
119
+ "151654": {
120
+ "content": "<|vision_pad|>",
121
+ "single_word": false,
122
+ "lstrip": false,
123
+ "rstrip": false,
124
+ "normalized": false,
125
+ "special": true
126
+ },
127
+ "151655": {
128
+ "content": "<|image_pad|>",
129
+ "single_word": false,
130
+ "lstrip": false,
131
+ "rstrip": false,
132
+ "normalized": false,
133
+ "special": true
134
+ },
135
+ "151656": {
136
+ "content": "<|video_pad|>",
137
+ "single_word": false,
138
+ "lstrip": false,
139
+ "rstrip": false,
140
+ "normalized": false,
141
+ "special": true
142
+ },
143
+ "151657": {
144
+ "content": "<tool_call>",
145
+ "single_word": false,
146
+ "lstrip": false,
147
+ "rstrip": false,
148
+ "normalized": false,
149
+ "special": false
150
+ },
151
+ "151658": {
152
+ "content": "</tool_call>",
153
+ "single_word": false,
154
+ "lstrip": false,
155
+ "rstrip": false,
156
+ "normalized": false,
157
+ "special": false
158
+ },
159
+ "151659": {
160
+ "content": "<|fim_prefix|>",
161
+ "single_word": false,
162
+ "lstrip": false,
163
+ "rstrip": false,
164
+ "normalized": false,
165
+ "special": false
166
+ },
167
+ "151660": {
168
+ "content": "<|fim_middle|>",
169
+ "single_word": false,
170
+ "lstrip": false,
171
+ "rstrip": false,
172
+ "normalized": false,
173
+ "special": false
174
+ },
175
+ "151661": {
176
+ "content": "<|fim_suffix|>",
177
+ "single_word": false,
178
+ "lstrip": false,
179
+ "rstrip": false,
180
+ "normalized": false,
181
+ "special": false
182
+ },
183
+ "151662": {
184
+ "content": "<|fim_pad|>",
185
+ "single_word": false,
186
+ "lstrip": false,
187
+ "rstrip": false,
188
+ "normalized": false,
189
+ "special": false
190
+ },
191
+ "151663": {
192
+ "content": "<|repo_name|>",
193
+ "single_word": false,
194
+ "lstrip": false,
195
+ "rstrip": false,
196
+ "normalized": false,
197
+ "special": false
198
+ },
199
+ "151664": {
200
+ "content": "<|file_sep|>",
201
+ "single_word": false,
202
+ "lstrip": false,
203
+ "rstrip": false,
204
+ "normalized": false,
205
+ "special": false
206
+ },
207
+ "151665": {
208
+ "content": "<|PAD_TOKEN|>",
209
+ "single_word": false,
210
+ "lstrip": false,
211
+ "rstrip": false,
212
+ "normalized": false,
213
+ "special": true
214
+ }
215
+ }
216
+ }
models/qwen2.5-coder-1.5b-plotter-lora/adapter_config.json ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "Qwen/Qwen2.5-Coder-1.5B-Instruct",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "layer_replication": null,
16
+ "layers_pattern": null,
17
+ "layers_to_transform": null,
18
+ "loftq_config": {},
19
+ "lora_alpha": 16,
20
+ "lora_bias": false,
21
+ "lora_dropout": 0.0,
22
+ "lora_ga_config": null,
23
+ "megatron_config": null,
24
+ "megatron_core": "megatron.core",
25
+ "modules_to_save": null,
26
+ "monteclora_config": null,
27
+ "peft_type": "LORA",
28
+ "peft_version": "0.20.0",
29
+ "qalora_group_size": 16,
30
+ "r": 16,
31
+ "rank_pattern": {},
32
+ "revision": null,
33
+ "target_modules": [
34
+ "gate_proj",
35
+ "up_proj",
36
+ "k_proj",
37
+ "down_proj",
38
+ "q_proj",
39
+ "o_proj",
40
+ "v_proj"
41
+ ],
42
+ "target_parameters": null,
43
+ "task_type": "CAUSAL_LM",
44
+ "trainable_token_indices": null,
45
+ "use_bdlora": null,
46
+ "use_dora": false,
47
+ "use_qalora": false,
48
+ "use_rslora": false,
49
+ "velora_config": null
50
+ }
models/qwen2.5-coder-1.5b-plotter-lora/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a5d0252190788771113123f6949ad11e13960f813e6a769fcad5551c7bd3c20b
3
+ size 73911112
models/qwen2.5-coder-1.5b-plotter-lora/chat_template.jinja ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- if tools %}
2
+ {{- '<|im_start|>system\n' }}
3
+ {%- if messages[0]['role'] == 'system' %}
4
+ {{- messages[0]['content'] }}
5
+ {%- else %}
6
+ {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
7
+ {%- endif %}
8
+ {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
9
+ {%- for tool in tools %}
10
+ {{- "\n" }}
11
+ {{- tool | tojson }}
12
+ {%- endfor %}
13
+ {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
14
+ {%- else %}
15
+ {%- if messages[0]['role'] == 'system' %}
16
+ {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
17
+ {%- else %}
18
+ {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
19
+ {%- endif %}
20
+ {%- endif %}
21
+ {%- for message in messages %}
22
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
23
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
24
+ {%- elif message.role == "assistant" %}
25
+ {{- '<|im_start|>' + message.role }}
26
+ {%- if message.content %}
27
+ {{- '\n' + message.content }}
28
+ {%- endif %}
29
+ {%- for tool_call in message.tool_calls %}
30
+ {%- if tool_call.function is defined %}
31
+ {%- set tool_call = tool_call.function %}
32
+ {%- endif %}
33
+ {{- '\n<tool_call>\n{"name": "' }}
34
+ {{- tool_call.name }}
35
+ {{- '", "arguments": ' }}
36
+ {{- tool_call.arguments | tojson }}
37
+ {{- '}\n</tool_call>' }}
38
+ {%- endfor %}
39
+ {{- '<|im_end|>\n' }}
40
+ {%- elif message.role == "tool" %}
41
+ {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
42
+ {{- '<|im_start|>user' }}
43
+ {%- endif %}
44
+ {{- '\n<tool_response>\n' }}
45
+ {{- message.content }}
46
+ {{- '\n</tool_response>' }}
47
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
48
+ {{- '<|im_end|>\n' }}
49
+ {%- endif %}
50
+ {%- endif %}
51
+ {%- endfor %}
52
+ {%- if add_generation_prompt %}
53
+ {{- '<|im_start|>assistant\n' }}
54
+ {%- endif %}
models/qwen2.5-coder-1.5b-plotter-lora/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
3
+ size 11421892
models/qwen2.5-coder-1.5b-plotter-lora/tokenizer_config.json ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": null,
5
+ "clean_up_tokenization_spaces": false,
6
+ "eos_token": "<|im_end|>",
7
+ "errors": "replace",
8
+ "extra_special_tokens": [
9
+ "<|im_start|>",
10
+ "<|im_end|>",
11
+ "<|object_ref_start|>",
12
+ "<|object_ref_end|>",
13
+ "<|box_start|>",
14
+ "<|box_end|>",
15
+ "<|quad_start|>",
16
+ "<|quad_end|>",
17
+ "<|vision_start|>",
18
+ "<|vision_end|>",
19
+ "<|vision_pad|>",
20
+ "<|image_pad|>",
21
+ "<|video_pad|>"
22
+ ],
23
+ "is_local": false,
24
+ "model_max_length": 32768,
25
+ "pad_token": "<|endoftext|>",
26
+ "split_special_tokens": false,
27
+ "tokenizer_class": "Qwen2Tokenizer",
28
+ "unk_token": null
29
+ }
requirements.txt ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # torch is pinned to a version ZeroGPU explicitly supports (2.8.0 - 2.11.0).
2
+ torch==2.8.0
3
+ transformers>=4.56,<5
4
+ peft>=0.20.0
5
+ accelerate>=1.0.0
6
+ safetensors>=0.4.0
7
+
8
+ pandas>=2.0
9
+ numpy>=1.26
10
+ matplotlib>=3.7
11
+ seaborn>=0.13
sandbox_runner.py ADDED
@@ -0,0 +1,85 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Out-of-process runner for model-generated plotting code.
2
+
3
+ Deliberately a separate script rather than a multiprocessing target: the parent
4
+ holds CUDA state that must not be forked, and re-importing app.py in a child
5
+ would rebuild the whole Gradio UI. The parent calls this with a hard subprocess
6
+ timeout, so a hallucinated `while True:` costs one killed child and nothing else.
7
+
8
+ Protocol: JSON on stdin -> JSON on stdout.
9
+ in : {"schema": str, "code": str, "seed": int, "png": str}
10
+ out: {"ok": bool, "error": str|null, "png": str|null}
11
+ """
12
+
13
+ import json
14
+ import sys
15
+ import warnings
16
+
17
+ import matplotlib
18
+ matplotlib.use("Agg")
19
+ import matplotlib.pyplot as plt
20
+ import numpy as np
21
+ import pandas as pd
22
+
23
+ try:
24
+ import seaborn as sns
25
+ except ImportError:
26
+ sns = None
27
+
28
+ from execute import _safe_builtins, clean_code_string
29
+ from schemas import GENERATORS
30
+
31
+ GEN_BY_NAME = {g.__name__: g for g in GENERATORS}
32
+
33
+
34
+ def main():
35
+ warnings.filterwarnings("ignore")
36
+ req = json.load(sys.stdin)
37
+ out = {"ok": False, "error": None, "png": None}
38
+
39
+ try:
40
+ gen = GEN_BY_NAME.get(req["schema"])
41
+ if gen is None:
42
+ out["error"] = f"unknown schema '{req['schema']}'"
43
+ return out
44
+
45
+ code = clean_code_string(req["code"])
46
+ if not code:
47
+ out["error"] = "empty code block after stripping formatting"
48
+ return out
49
+
50
+ ns = {
51
+ "df": gen(seed=req.get("seed", 0)),
52
+ "plt": plt,
53
+ "pd": pd,
54
+ "np": np,
55
+ "matplotlib": matplotlib,
56
+ "__builtins__": _safe_builtins(), # no os/open/eval/subprocess
57
+ }
58
+ if sns is not None:
59
+ ns["sns"] = sns
60
+
61
+ plt.close("all")
62
+ exec(code, ns)
63
+
64
+ if not plt.get_fignums():
65
+ out["error"] = "code ran but created no matplotlib figure"
66
+ return out
67
+
68
+ fig = plt.gcf()
69
+ if not (fig.axes and any(ax.get_children() for ax in fig.axes)):
70
+ out["error"] = "figure was created but contains no visual elements"
71
+ return out
72
+
73
+ # We save the figure, not the model -- generated code never touches disk.
74
+ fig.savefig(req["png"], dpi=110, bbox_inches="tight")
75
+ out["ok"] = True
76
+ out["png"] = req["png"]
77
+ except Exception as e: # noqa: BLE001 -- any failure is a data point, not a crash
78
+ out["error"] = f"{type(e).__name__}: {e}"
79
+ finally:
80
+ plt.close("all")
81
+ return out
82
+
83
+
84
+ if __name__ == "__main__":
85
+ print(json.dumps(main()))
schemas.py ADDED
@@ -0,0 +1,284 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Synthetic vineyard / winery DataFrames.
2
+
3
+ Twelve generators spanning the whole operation -- vineyard block records, field
4
+ time-series, lab chemistry, cellar, soil, pest scouting, sensors, sensory panel,
5
+ logistics, sales, canopy management, and barrel aging.
6
+
7
+ WHY SO MANY: the student model must learn to READ the schema preview it is given
8
+ and reference those columns. With only a few generators it can instead memorise
9
+ one fixed column vocabulary ("vineyard plot" -> df['brix']) and still drive the
10
+ training loss to zero -- then fail on any unseen DataFrame. Distinct column names
11
+ per generator is what forces the model to actually read the preview.
12
+
13
+ SEEDING: every generator takes `seed` and builds its OWN Generator, so a call is
14
+ reproducible from (name, seed) alone. This matters because generate_raw.py builds
15
+ a df to write the preview, and execute.py later rebuilds it to validate the code
16
+ -- they must agree. A module-level shared RNG would advance between those two
17
+ calls and hand back different data.
18
+ """
19
+ import numpy as np
20
+ import pandas as pd
21
+
22
+ BLOCKS = ["North Slope", "River Bench", "Hilltop", "Old Vines", "Clay Flat", "East Terrace"]
23
+ VARIETIES = ["Cabernet Sauvignon", "Chardonnay", "Pinot Noir", "Syrah", "Merlot", "Sauvignon Blanc"]
24
+ ROOTSTOCK = ["101-14", "3309C", "SO4", "1103P", "Riparia Gloire"]
25
+ DISEASES = ["Powdery Mildew", "Downy Mildew", "Botrytis", "None"]
26
+
27
+
28
+ def block_vintage_df(n=220, seed=0):
29
+ """One row per block x vintage: climate, canopy, yield, and quality at harvest."""
30
+ rng = np.random.default_rng(seed)
31
+ gdd = rng.normal(1650, 220, n).round(0).clip(900, 2600)
32
+ # warmer seasons ripen harder: brix tracks GDD, acid falls away as sugar climbs
33
+ brix = (18 + (gdd - 900) / 1700 * 6 + rng.normal(0, 0.9, n)).round(1).clip(18, 30)
34
+ return pd.DataFrame({
35
+ "block": rng.choice(BLOCKS, n),
36
+ "variety": rng.choice(VARIETIES, n),
37
+ "rootstock": rng.choice(ROOTSTOCK, n),
38
+ "vintage": rng.integers(2012, 2025, n),
39
+ "gdd": gdd, # growing degree days
40
+ "rainfall_mm": rng.normal(520, 160, n).round(1).clip(120, 1100),
41
+ "irrigation_mm": rng.normal(180, 70, n).round(0).clip(0, 400),
42
+ "yield_tonnes_per_acre": rng.normal(4.2, 1.6, n).round(2).clip(0.5, 9),
43
+ "brix": brix, # sugar
44
+ "ph": rng.normal(3.55, 0.18, n).round(2).clip(3.0, 4.1),
45
+ "titratable_acidity_gL": (13 - brix * 0.25 + rng.normal(0, 0.7, n)).round(2).clip(3, 11),
46
+ "disease": rng.choice(DISEASES, n, p=[0.28, 0.17, 0.15, 0.40]),
47
+ "vine_age_years": rng.integers(3, 60, n),
48
+ })
49
+
50
+
51
+ def phenology_df(n=260, seed=0):
52
+ """Time series across a growing season: berry weight, sugar, acid accumulation."""
53
+ rng = np.random.default_rng(seed)
54
+ dates = pd.date_range("2024-04-01", periods=n // 4 + 1, freq="W")
55
+ reps = np.tile(dates, 4)[:n]
56
+ # week index drives ripening: sugar up, acid down, veraison sigmoid
57
+ wk = np.array([(d - dates[0]).days / 7 for d in reps])
58
+ span = max(wk.max(), 1)
59
+ return pd.DataFrame({
60
+ "date": reps,
61
+ "block": rng.choice(BLOCKS, n),
62
+ "variety": rng.choice(VARIETIES, n),
63
+ "berry_weight_g": (0.4 + 1.4 * wk / span + rng.normal(0, 0.15, n)).round(3).clip(0.2, 2.6),
64
+ "brix": (6 + 18 * wk / span + rng.normal(0, 1.2, n)).round(1).clip(5, 28),
65
+ "titratable_acidity_gL": (19 - 12 * wk / span + rng.normal(0, 1.0, n)).round(2).clip(3, 20),
66
+ "veraison_pct": (100 / (1 + np.exp(-(wk - span / 2) * 0.8)) + rng.normal(0, 6, n)).round(0).clip(0, 100),
67
+ })
68
+
69
+
70
+ def berry_chem_df(n=200, seed=0):
71
+ """Lab chemistry samples: phenolics, anthocyanins, tannin by variety/block."""
72
+ rng = np.random.default_rng(seed)
73
+ phenolics = rng.normal(55, 15, n).round(1).clip(10, 110)
74
+ return pd.DataFrame({
75
+ "sample_id": np.arange(1, n + 1),
76
+ "block": rng.choice(BLOCKS, n),
77
+ "variety": rng.choice(VARIETIES, n),
78
+ # anthocyanin and tannin are both phenolic fractions -> they co-vary
79
+ "anthocyanin_mg_g": (phenolics * 0.022 + rng.normal(0, 0.22, n)).round(3).clip(0.1, 3.0),
80
+ "total_phenolics_au": phenolics,
81
+ "tannin_mg_g": (phenolics * 0.032 + rng.normal(0, 0.35, n)).round(2).clip(0.3, 4.5),
82
+ "ph": rng.normal(3.55, 0.18, n).round(2).clip(3.0, 4.1),
83
+ "brix": rng.normal(24.5, 2.2, n).round(1).clip(18, 30),
84
+ })
85
+
86
+
87
+ def cellar_ferment_df(n=240, seed=0):
88
+ """Daily tank readings through alcoholic fermentation."""
89
+ rng = np.random.default_rng(seed)
90
+ day = rng.integers(0, 16, n)
91
+ sugar = (230 * np.exp(-0.19 * day) + rng.normal(0, 6, n)).round(1).clip(0, 260)
92
+ return pd.DataFrame({
93
+ "tank_id": rng.choice([f"T{i:02d}" for i in range(1, 19)], n),
94
+ "yeast_strain": rng.choice(["EC-1118", "RC-212", "D254", "BM4x4", "Native"], n),
95
+ "day_of_ferment": day,
96
+ "residual_sugar_gL": sugar,
97
+ # every gram of sugar consumed becomes roughly 1/17 % alcohol
98
+ "alcohol_pct": ((230 - sugar) / 17.0 + rng.normal(0, 0.25, n)).round(2).clip(0, 16),
99
+ "must_temp_c": rng.normal(26, 3.5, n).round(1).clip(12, 35),
100
+ "cap_temp_c": rng.normal(29, 4.0, n).round(1).clip(14, 40),
101
+ "free_so2_ppm": rng.normal(28, 9, n).round(0).clip(0, 60),
102
+ "volatile_acidity_gL": rng.normal(0.42, 0.14, n).round(3).clip(0.05, 1.2),
103
+ "punchdowns_per_day": rng.integers(0, 4, n),
104
+ })
105
+
106
+
107
+ def soil_survey_df(n=180, seed=0):
108
+ """Soil pit descriptions and lab results by depth horizon."""
109
+ rng = np.random.default_rng(seed)
110
+ clay = rng.normal(28, 11, n).round(1).clip(3, 65)
111
+ return pd.DataFrame({
112
+ "pit_id": rng.choice([f"P{i:03d}" for i in range(1, 41)], n),
113
+ "soil_series": rng.choice(["Bale Loam", "Pleasanton", "Sobrante", "Hambright", "Yolo Silt"], n),
114
+ "drainage_class": rng.choice(["Well drained", "Moderately well", "Somewhat poor", "Excessive"], n),
115
+ "horizon_depth_cm": rng.choice([15, 30, 45, 60, 90, 120], n),
116
+ "clay_pct": clay,
117
+ "sand_pct": (90 - clay + rng.normal(0, 7, n)).round(1).clip(5, 92),
118
+ "organic_matter_pct": rng.normal(2.1, 0.8, n).round(2).clip(0.2, 6.0),
119
+ # clay holds cations -> CEC rises with clay fraction
120
+ "cec_meq_100g": (clay * 0.42 + rng.normal(0, 2.2, n)).round(1).clip(2, 40),
121
+ "soil_ph": rng.normal(6.4, 0.6, n).round(2).clip(4.5, 8.4),
122
+ "available_water_mm_m": rng.normal(135, 35, n).round(0).clip(40, 240),
123
+ })
124
+
125
+
126
+ def pest_scouting_df(n=280, seed=0):
127
+ """Weekly scouting walks: pest pressure per row."""
128
+ rng = np.random.default_rng(seed)
129
+ incidence = rng.gamma(2.0, 5.5, n).round(1).clip(0, 100)
130
+ return pd.DataFrame({
131
+ "scout_date": np.tile(pd.date_range("2024-05-01", periods=n // 7 + 1, freq="W"), 7)[:n],
132
+ "block": rng.choice(BLOCKS, n),
133
+ "row_number": rng.integers(1, 61, n),
134
+ "pest_name": rng.choice(["Vine Mealybug", "Leafhopper", "Spider Mite",
135
+ "European Grapevine Moth", "Thrips"], n),
136
+ "incidence_pct": incidence,
137
+ "severity_index": (incidence / 25 + rng.normal(0, 0.4, n)).round(2).clip(0, 5),
138
+ "trap_count": rng.poisson(6, n),
139
+ "beneficials_count": rng.poisson(3, n),
140
+ "threshold_exceeded": incidence > 20,
141
+ "spray_applied": rng.random(n) < 0.3,
142
+ })
143
+
144
+
145
+ def irrigation_sensor_df(n=300, seed=0):
146
+ """Hourly-to-daily sensor telemetry for irrigation scheduling."""
147
+ rng = np.random.default_rng(seed)
148
+ moisture = rng.normal(24, 6, n).round(2).clip(6, 42)
149
+ return pd.DataFrame({
150
+ "timestamp": pd.date_range("2024-06-01", periods=n, freq="6h"),
151
+ "sensor_id": rng.choice([f"S-{i:02d}" for i in range(1, 13)], n),
152
+ "block": rng.choice(BLOCKS, n),
153
+ "soil_moisture_vwc": moisture,
154
+ # drier soil -> more negative (more stressed) stem water potential
155
+ "stem_water_potential_bar": (-18 + moisture * 0.33 + rng.normal(0, 1.1, n)).round(2).clip(-20, -2),
156
+ "canopy_temp_c": rng.normal(29, 5, n).round(1).clip(12, 46),
157
+ "air_temp_c": rng.normal(26, 6, n).round(1).clip(8, 44),
158
+ "relative_humidity_pct": rng.normal(52, 16, n).round(0).clip(8, 99),
159
+ "et0_mm": rng.gamma(3, 1.6, n).round(2).clip(0.2, 14),
160
+ "valve_open": rng.random(n) < 0.22,
161
+ })
162
+
163
+
164
+ def sensory_panel_df(n=260, seed=0):
165
+ """Blind tasting panel scores, one row per taster x wine."""
166
+ rng = np.random.default_rng(seed)
167
+ fruit = rng.normal(6.4, 1.5, n).round(1).clip(1, 10)
168
+ structure = rng.normal(6.0, 1.6, n).round(1).clip(1, 10)
169
+ return pd.DataFrame({
170
+ "wine_code": rng.choice([f"W{i:03d}" for i in range(1, 25)], n),
171
+ "taster_id": rng.choice([f"J{i:02d}" for i in range(1, 13)], n),
172
+ "flight": rng.choice(["Flight A", "Flight B", "Flight C"], n),
173
+ "aroma_intensity": rng.normal(6.1, 1.7, n).round(1).clip(1, 10),
174
+ "fruit_score": fruit,
175
+ "tannin_score": structure,
176
+ "acidity_score": rng.normal(6.2, 1.4, n).round(1).clip(1, 10),
177
+ "finish_seconds": rng.gamma(4, 3.2, n).round(0).clip(2, 60),
178
+ # overall is mostly a blend of fruit and structure, plus taster noise
179
+ "overall_rating": (fruit * 0.5 + structure * 0.4 + rng.normal(0, 0.6, n)).round(1).clip(1, 10),
180
+ "would_purchase": rng.random(n) < 0.45,
181
+ })
182
+
183
+
184
+ def harvest_logistics_df(n=240, seed=0):
185
+ """Pick-day operations: crews, bins, transport to the crush pad."""
186
+ rng = np.random.default_rng(seed)
187
+ crew = rng.integers(6, 25, n)
188
+ return pd.DataFrame({
189
+ "pick_date": np.tile(pd.date_range("2024-08-20", periods=n // 6 + 1, freq="D"), 6)[:n],
190
+ "block": rng.choice(BLOCKS, n),
191
+ "crew_id": rng.choice(["Crew Alpha", "Crew Bravo", "Crew Charlie", "Crew Delta"], n),
192
+ "crew_size": crew,
193
+ "pick_method": rng.choice(["Hand", "Machine"], n, p=[0.72, 0.28]),
194
+ # more pickers -> more bins, with diminishing returns and day-to-day noise
195
+ "bins_filled": (crew * 1.9 + rng.normal(0, 4, n)).round(0).clip(2, 70),
196
+ "kg_per_hour": rng.normal(410, 120, n).round(0).clip(80, 900),
197
+ "transport_km": rng.normal(14, 7, n).round(1).clip(0.5, 45),
198
+ "fruit_temp_c": rng.normal(17, 5, n).round(1).clip(4, 34),
199
+ "wait_time_min": rng.gamma(2.5, 14, n).round(0).clip(0, 180),
200
+ "mog_pct": rng.normal(2.4, 1.3, n).round(2).clip(0, 12), # material other than grapes
201
+ })
202
+
203
+
204
+ def wine_sales_df(n=320, seed=0):
205
+ """Bottle sales by channel and region."""
206
+ rng = np.random.default_rng(seed)
207
+ price = rng.normal(38, 14, n).round(2).clip(9, 130)
208
+ return pd.DataFrame({
209
+ "order_date": pd.date_range("2023-01-01", periods=n, freq="D"),
210
+ "sku": rng.choice([f"SKU-{i:03d}" for i in range(1, 19)], n),
211
+ "region": rng.choice(["Napa", "Sonoma", "Oregon", "Export EU", "Export Asia"], n),
212
+ "channel": rng.choice(["Tasting Room", "Wine Club", "Distributor", "Online"], n),
213
+ "unit_price_usd": price,
214
+ # cheaper bottles move in larger volumes
215
+ "bottles_sold": (rng.gamma(3, 22, n) * (60 / price)).round(0).clip(1, 900),
216
+ "discount_pct": rng.choice([0, 5, 10, 15, 20, 25], n, p=[.4, .16, .16, .12, .1, .06]),
217
+ "shipping_cost_usd": rng.normal(22, 9, n).round(2).clip(0, 80),
218
+ "club_member": rng.random(n) < 0.38,
219
+ })
220
+
221
+
222
+ def canopy_pruning_df(n=200, seed=0):
223
+ """Dormant pruning and canopy architecture measurements per vine."""
224
+ rng = np.random.default_rng(seed)
225
+ shoots = rng.integers(12, 60, n)
226
+ return pd.DataFrame({
227
+ "vine_id": np.arange(1000, 1000 + n),
228
+ "block": rng.choice(BLOCKS, n),
229
+ "trellis_type": rng.choice(["VSP", "Lyre", "Head-trained", "Quadrilateral Cordon"], n),
230
+ "pruning_method": rng.choice(["Spur", "Cane", "Minimal"], n),
231
+ "shoot_count": shoots,
232
+ "bud_count": (shoots * 1.4 + rng.normal(0, 3, n)).round(0).clip(8, 100),
233
+ # leaf area scales with how many shoots the vine carries
234
+ "leaf_area_m2": (shoots * 0.14 + rng.normal(0, 0.6, n)).round(2).clip(0.5, 12),
235
+ "cane_weight_kg": rng.normal(0.85, 0.3, n).round(3).clip(0.1, 2.5),
236
+ "internode_length_cm": rng.normal(7.5, 2.0, n).round(1).clip(2, 16),
237
+ "cluster_count": rng.integers(8, 55, n),
238
+ })
239
+
240
+
241
+ def barrel_aging_df(n=220, seed=0):
242
+ """Barrel inventory and extraction chemistry during elevage."""
243
+ rng = np.random.default_rng(seed)
244
+ months = rng.integers(0, 25, n)
245
+ return pd.DataFrame({
246
+ "barrel_id": rng.choice([f"B{i:04d}" for i in range(1, 121)], n),
247
+ "cooper": rng.choice(["Taransaud", "Seguin Moreau", "Francois Freres",
248
+ "Nadalie", "World Cooperage"], n),
249
+ "oak_origin": rng.choice(["French", "American", "Hungarian"], n),
250
+ "toast_level": rng.choice(["Light", "Medium", "Medium Plus", "Heavy"], n),
251
+ "months_in_barrel": months,
252
+ "barrel_age_fills": rng.integers(1, 6, n),
253
+ # oak compounds extract over time in barrel
254
+ "vanillin_ppb": (months * 21 + rng.normal(0, 45, n)).round(0).clip(0, 700),
255
+ "oak_lactone_ppb": (months * 14 + rng.normal(0, 38, n)).round(0).clip(0, 500),
256
+ "color_intensity_au": rng.normal(11.5, 3.0, n).round(2).clip(2, 22),
257
+ "topping_volume_l": rng.gamma(2, 0.9, n).round(2).clip(0, 9),
258
+ })
259
+
260
+
261
+ GENERATORS = [
262
+ block_vintage_df,
263
+ phenology_df,
264
+ berry_chem_df,
265
+ cellar_ferment_df,
266
+ soil_survey_df,
267
+ pest_scouting_df,
268
+ irrigation_sensor_df,
269
+ sensory_panel_df,
270
+ harvest_logistics_df,
271
+ wine_sales_df,
272
+ canopy_pruning_df,
273
+ barrel_aging_df,
274
+ ]
275
+
276
+
277
+ def df_preview(df, n=5):
278
+ """The EXACT schema string the model sees at train AND inference time.
279
+
280
+ Keep this the single source of truth -- any drift between train/infer
281
+ formatting pushes the model out of distribution.
282
+ """
283
+ dtypes = ", ".join(f"{c} ({df[c].dtype})" for c in df.columns)
284
+ return f"Columns and dtypes:\n {dtypes}\nSample rows:\n{df.head(n).to_string(index=False)}"