Deploy hierarchical SoftChart V1.8

README.md

@@ -10,47 +10,39 @@ app_file: app.py
 pinned: false
 license: mit
 models:
-  - JacobLinCool/softchart-v17
-  - JacobLinCool/softchart-v17-small
-  - JacobLinCool/softchart-v17-tiny
-  - JacobLinCool/softchart-v16
-  - JacobLinCool/softchart-v16-small
-  - JacobLinCool/softchart-v16-tiny
-  - JacobLinCool/softchart-v15
-  - JacobLinCool/softchart-planner
+  - JacobLinCool/softchart-v18
+datasets:
+  - JacobLinCool/taiko-1000-parsed-clean
 ---
 
-# 🥁 SoftChart
+# 🥁 SoftChart V1.8
 
-Generate a **Taiko no Tatsujin** chart from any song. A unified 7.9M-parameter
-encoder–decoder model handles beat/downbeat fitting, slot-exact generation, and
-gridless time generation; an optional planner supplies song-level density,
-breathing gaps, and climax structure. All models are MIT-licensed and loaded
-from the Hugging Face Hub via `from_pretrained`.
+Generate a **Taiko no Tatsujin-style** chart from an uploaded song with the
+scratch-trained [SoftChart V1.8 hierarchical model](https://huggingface.co/JacobLinCool/softchart-v18).
+The 8,985,091-parameter model combines whole-song hierarchical context, a local
+rhythmic-skeleton auxiliary head, chart generation, and beat/downbeat
+estimation. It uses no external pretrained model and no external song planner.
 
-- **SoftChart v15** ([softchart-v15](https://huggingface.co/JacobLinCool/softchart-v15)) — unified slot + time + beat model; slot mode emits exact TJA lattice indices when the beat grid is trusted
-- **Planner** ([softchart-planner](https://huggingface.co/JacobLinCool/softchart-planner)) — audio → song-level plan (density envelope, breathing gaps, climax)
+The Space always uses V1.8's full-song **time generation** path for arbitrary
+uploads. The beat head can estimate BPM and a downbeat grid for TJA
+quantization, but that estimated grid is not treated as the trusted rational
+meter required by exact slot generation. This distinction prevents an
+estimated beat grid from being presented as authored-meter exact timing.
 
-The Space uses Gradio Server mode with a fully custom, responsive interface.
-Upload a song, pick a difficulty, and get a full-song mel/plan preview, a
-full TJA chart image, a `.tja` playable in TJAPlayer3 / OpenTaiko, and a
-synchronized WAV mix that lets you hear the generated chart performed over
-the source music immediately. Its Don, Big Don, and Katsu voices are original
-modal-resonator syntheses built from seeded excitation noise, decaying drum
-modes, pitch sweeps, and band-limited stick transients. Reference recordings
-are used only for offline envelope and level calibration; they are not copied,
-mixed, embedded, or required by the Space.
+Audio preprocessing follows the released model contract: FFmpeg decodes
+stereo float32 at 22,050 Hz, channels are averaged arithmetically, and a
+periodic-Hann STFT is projected to 128 log-mel bins. Upload a song, choose a
+difficulty, and receive:
 
-The generation API streams the real processing stages to the interface:
-audio analysis, beat-grid fitting, song-level planning, note generation,
-chart rendering, and taiko preview mixing. The final audio is timed from the
-exported TJA itself, including `OFFSET` and `#BPMCHANGE`, so the audition and
-downloadable chart share the same timing.
+- a playable `.tja` for TJAPlayer3 / OpenTaiko;
+- a rendered chart image;
+- the full-song mel structure view used by the hierarchy;
+- a synchronized WAV audition with synthesized Taiko hits.
 
-The generated TJA keeps the uploaded audio filename in its `WAVE` field.
-Planner climax blocks are emitted as TJA `#GOGOSTART` / `#GOGOEND` sections.
-The chart image uses a server-side renderer modeled on the row layout used by
-MIT-licensed [tja-tools](https://github.com/WHMHammer/tja-tools), without
-shipping its browser/webpack runtime.
+The audition is rendered from the exported TJA, including its `OFFSET`, so the
+preview and downloadable chart use the same timing. Taiko voices are original
+modal-resonator syntheses included under the project's MIT license.
 
-> Code and weights are **MIT-licensed**.
+The model and Space code are **MIT-licensed**. Evaluation scope, metrics, and
+known limitations are documented on the
+[V1.8 model card](https://huggingface.co/JacobLinCool/softchart-v18).

app.py

@@ -1,16 +1,16 @@
-"""SoftChart — custom Gradio Server app for Hugging Face Spaces.
+"""SoftChart V1.8 — hierarchical Taiko chart generation for HF Spaces.
 
-Generate a Taiko no Tatsujin chart from any audio file, using the full system:
-  - SoftChartGenerator  (main model, plan-conditioned)
-  - SoftChartPlanner     (auto song-level planning)
-  - SoftChartBeat        (beat/downbeat for barline anchoring)
-All models load from the Hub via from_pretrained. MIT licensed.
+The single scratch-trained model owns whole-song hierarchy, local chart
+generation, and beat/downbeat estimation. Arbitrary uploads always use the
+model's time path; an estimated beat grid may quantize the exported TJA but is
+never promoted to the trusted-meter slot path.
 """
 
 import logging
 import os
 import re
 import shutil
+import subprocess
 import tempfile
 from pathlib import Path
 
@@ -20,123 +20,103 @@ import torch
 from fastapi.responses import FileResponse
 from fastapi.staticfiles import StaticFiles
 
-from softchart.generate import generate_song, generate_song_slot, load_hf
+from softchart.generate import generate_song, load_hf
 from softchart.fonts import cjk_font_path
 from softchart.grid import debias_to_grid, fit_grid_fixed_bpm, fit_grid_piecewise
-from softchart.hf import SoftChartPlanner
 from softchart.preview_audio import synthesize_taiko_preview
 from softchart.rhythm import snap_chart
-from softchart.tja import append_measure_with_gogo, gogo_measure_mask, write_tja_slots
 from softchart.tja_image import render_tja_image
 from softchart.vocab import FPS, HOP, N_FFT, N_MELS, SR
 
 LOGGER = logging.getLogger("softchart.space")
 STATIC_DIR = Path(__file__).with_name("static")
 
-PLAN_REPO = os.environ.get("SC_PLAN", "JacobLinCool/softchart-planner")
+MODEL_REPO = "JacobLinCool/softchart-v18"
 DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
-
-# Selectable generators: three generations x three sizes (plus the v1.5 single
-# model). All are dual (slot + time), beat- and plan-conditioned, MIT licensed.
-# https://huggingface.co/collections/JacobLinCool/softchart-generators
-MODELS = {
-    "v1.7": "JacobLinCool/softchart-v17",
-    "v1.7-small": "JacobLinCool/softchart-v17-small",
-    "v1.7-tiny": "JacobLinCool/softchart-v17-tiny",
-    "v1.6": "JacobLinCool/softchart-v16",
-    "v1.6-small": "JacobLinCool/softchart-v16-small",
-    "v1.6-tiny": "JacobLinCool/softchart-v16-tiny",
-    "v1.5": "JacobLinCool/softchart-v15",
-}
-DEFAULT_MODEL = os.environ.get("SC_MODEL", "v1.7")
+EXPECTED_PARAMETERS = 8_985_091
 
 COURSE_DENS = {"easy": 1, "normal": 2, "hard": 4, "oni": 7}
 CHAR = {"don": "1", "ka": "2", "don_big": "3", "ka_big": "4",
         "roll": "5", "roll_big": "6", "balloon": "7"}
 SUB = 96
 
-_MODELS = {}  # repo id -> {"gen","slot","beat"}, loaded lazily and cached
-_PLANNER = {}  # the planner is model-agnostic and shared across generators
-
-
-def get_models(model_choice=DEFAULT_MODEL, *, include_planner=False):
-    repo = MODELS.get(model_choice)
-    if repo is None:
-        raise ValueError(
-            f"unknown model {model_choice!r}; choose one of {list(MODELS)}"
-        )
-    if repo not in _MODELS:
-        u = load_hf(repo, device=DEVICE)
-        if (not getattr(u, "_dual", False) or u.beat is None
-                or not getattr(u, "_has_plan", False)):
+_MODEL = {}
+_MEL_FB = None
+_STFT_WINDOW = None
+
+
+def get_model():
+    if "generator" not in _MODEL:
+        model = load_hf(MODEL_REPO, device=DEVICE)
+        capabilities = getattr(model, "_softchart_capabilities", {})
+        required = {
+            "aux": True,
+            "beat_head": True,
+            "dual": True,
+            "hierarchical_ctx": True,
+            "global_ctx": False,
+            "plan": False,
+        }
+        mismatched = {
+            name: capabilities.get(name)
+            for name, expected in required.items()
+            if capabilities.get(name) is not expected
+        }
+        if (not getattr(model, "_dual", False) or model.beat is None
+                or not getattr(model, "_hierarchical_ctx", False)
+                or getattr(model, "_has_plan", False) or mismatched):
+            raise RuntimeError(
+                f"{MODEL_REPO} is not the expected hierarchical V1.8 artifact"
+            )
+        parameters = sum(parameter.numel() for parameter in model.parameters())
+        if parameters != EXPECTED_PARAMETERS:
             raise RuntimeError(
-                f"{repo} must provide dual generation, beat, and plan conditioning"
+                f"{MODEL_REPO} has {parameters:,} parameters; expected "
+                f"{EXPECTED_PARAMETERS:,}"
             )
-        _MODELS[repo] = {"gen": u, "slot": u, "beat": u}
-    models = dict(_MODELS[repo])
-    if include_planner:
-        if "plan" not in _PLANNER:
-            _PLANNER["plan"] = SoftChartPlanner.from_pretrained(PLAN_REPO).to(DEVICE).eval()
-        models["plan"] = _PLANNER["plan"]
-    return models
+        _MODEL["generator"] = model
+    return _MODEL["generator"]
 
 
 def load_logmel(path):
+    """Apply the exact V1.8 FFmpeg + torch.stft preprocessing contract."""
     import librosa
-
-    wav, _ = librosa.load(path, sr=SR, mono=True)
-    fb = librosa.filters.mel(sr=SR, n_fft=N_FFT, n_mels=N_MELS, fmin=20.0, fmax=SR / 2)
-    spec = torch.stft(torch.from_numpy(wav), N_FFT, hop_length=HOP,
-                      window=torch.hann_window(N_FFT), center=True, return_complex=True)
-    mel = np.log(fb @ spec.abs().pow(2).numpy() + 1e-5).astype(np.float32)
-    return mel, wav
-
-
-def auto_plan(mel, bpm, downbeats=None):
-    T = mel.shape[1]
-    dur = T / FPS
-    flux = np.concatenate([[0], np.maximum(0, np.diff(mel, axis=1)).sum(0)])
-    beat = 60.0 / bpm
-    edges = (list(downbeats[::4]) + [dur]) if (downbeats is not None and len(downbeats) >= 2) \
-        else list(np.arange(0, dur, 4 * beat)) + [dur]
-    vals = [float(flux[int(a * FPS):int(b * FPS)].mean()) if int(b * FPS) > int(a * FPS) else 0.0
-            for a, b in zip(edges, edges[1:])]
-    if not vals:
-        return None
-    vals = np.array(vals)
-    lo, hi = np.percentile(vals, 15), np.percentile(vals, 92)
-    peak = int(np.argmax(vals))
-    plan = []
-    for i, (a, b) in enumerate(zip(edges, edges[1:])):
-        frac = (vals[i] - lo) / max(hi - lo, 1e-6)
-        d8 = int(np.clip(round(frac * 7), 0, 7))
-        fl = 1 if (vals[i] <= lo and 0 < i < len(vals) - 1) else (2 if i == peak and vals[i] >= hi else 0)
-        plan.append([round(a, 3), round(b, 3), d8, fl])
-    return plan
-
-
-def learned_plan(planner, mel, course, bpm, downbeats=None):
-    dur = mel.shape[1] / FPS
-    beat = 60.0 / bpm
-    edges = (list(downbeats[::4]) + [dur]) if (downbeats is not None and len(downbeats) >= 2) \
-        else list(np.arange(0, dur, 4 * beat)) + [dur]
-    feats, spans = [], []
-    for a, b in zip(edges, edges[1:]):
-        seg = mel[:, int(a * FPS):int(b * FPS)]
-        if seg.shape[1] < 2:
-            continue
-        fx = np.maximum(0, np.diff(seg, axis=1)).sum(0)
-        feats.append(np.concatenate([seg.mean(1), seg.std(1), [fx.mean(), fx.std(), fx.max()]]))
-        spans.append((round(float(a), 3), round(float(b), 3)))
-    if not feats:
-        return None
-    cid = {"easy": 0, "normal": 1, "hard": 2, "oni": 3}[course]
-    x = torch.tensor(np.array(feats), dtype=torch.float32)[None].to(DEVICE)
+    global _MEL_FB, _STFT_WINDOW
+
+    command = [
+        "ffmpeg", "-nostdin", "-hide_banner", "-loglevel", "error",
+        "-i", os.fspath(path), "-vn", "-ac", "2", "-ar", str(SR),
+        "-f", "f32le", "-acodec", "pcm_f32le", "pipe:1",
+    ]
+    proc = subprocess.run(
+        command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=False,
+    )
+    if proc.returncode != 0:
+        detail = proc.stderr.decode("utf-8", errors="replace").strip()
+        raise ValueError(f"FFmpeg could not decode this upload: {detail}")
+    decoded = np.frombuffer(proc.stdout, dtype="<f4")
+    if decoded.size % 2:
+        raise ValueError("Decoded stereo audio contains an incomplete frame.")
+    wav = decoded.reshape(-1, 2).mean(axis=1, dtype=np.float32)
+    if wav.size < SR:
+        raise ValueError("Audio must be at least one second long.")
+    if not np.isfinite(wav).all():
+        raise ValueError("Decoded audio contains non-finite samples.")
+    if _MEL_FB is None:
+        fb = librosa.filters.mel(
+            sr=SR, n_fft=N_FFT, n_mels=N_MELS, fmin=20.0, fmax=SR / 2,
+        )
+        _MEL_FB = torch.from_numpy(fb)
+        _STFT_WINDOW = torch.hann_window(N_FFT)
+    wav_tensor = torch.from_numpy(np.ascontiguousarray(wav))[None]
     with torch.no_grad():
-        pd, pf = planner(x, torch.tensor([cid], device=DEVICE))
-    d8 = pd[0].argmax(-1).cpu().numpy()
-    fl = pf[0].argmax(-1).cpu().numpy()
-    return [[a, b, int(d), int(f)] for (a, b), d, f in zip(spans, d8, fl)]
+        spec = torch.stft(
+            wav_tensor, N_FFT, hop_length=HOP, window=_STFT_WINDOW,
+            center=True, return_complex=True,
+        )[0]
+        mel = torch.log(_MEL_FB @ spec.abs().pow(2) + 1e-5)
+    mel = mel.numpy().astype(np.float32)
+    return mel, wav
 
 
 def group_quantize(times, phase, grid, min_run=3):
@@ -176,8 +156,7 @@ def output_tja_path(wave_name, course, directory):
     return os.path.join(directory, f"{stem}_{course}.tja")
 
 
-def write_tja(gen, bpm, title, course, level, wave, downbeats=None, grid_fit=None,
-              plan=None):
+def write_tja(gen, bpm, title, course, level, wave, downbeats=None, grid_fit=None):
     hits = sorted((h["t"], CHAR[h["type"]]) for h in gen["hits"])
     beat = 60.0 / bpm
     grid = beat / (SUB / 4)
@@ -229,17 +208,10 @@ def write_tja(gen, bpm, title, course, level, wave, downbeats=None, grid_fit=Non
             slots = {k - shift: v for k, v in slots.items()}
             phase += shift * grid
     n_meas = (max(slots) // SUB + 1) if slots else 1
-    measure_starts = phase + np.arange(n_meas + 1, dtype=float) * (4 * beat)
-    gogo_mask = gogo_measure_mask(plan, measure_starts, n_meas)
-    lines = []
-    in_gogo = False
-    for m in range(n_meas):
-        in_gogo = append_measure_with_gogo(
-            lines,
-            "".join(slots.get(m * SUB + k, "0") for k in range(SUB)) + ",",
-            m, gogo_mask, in_gogo)
-    if in_gogo:
-        lines.append("#GOGOEND")
+    lines = [
+        "".join(slots.get(m * SUB + k, "0") for k in range(SUB)) + ","
+        for m in range(n_meas)
+    ]
     balloons = [10] * sum(1 for s in gen["spans"] if s["type"] == "balloon")
     return "\n".join([
         f"TITLE:{title} (SoftChart)", f"BPM:{bpm:g}", f"WAVE:{wave}",
@@ -248,7 +220,7 @@ def write_tja(gen, bpm, title, course, level, wave, downbeats=None, grid_fit=Non
         "", "#START", *lines, "#END"]) + "\n"
 
 
-def render_audio_plan(mel, title, course, out_path, plan=None):
+def render_song_structure(mel, title, course, out_path):
     import matplotlib
     matplotlib.use("Agg")
     from matplotlib import font_manager
@@ -261,32 +233,14 @@ def render_audio_plan(mel, title, course, out_path, plan=None):
     matplotlib.rcParams["axes.unicode_minus"] = False
 
     dur = mel.shape[1] / FPS
-    fig = plt.figure(figsize=(13, 4.4 if plan else 3.2))
-    gs = fig.add_gridspec(2 if plan else 1, 1,
-                          height_ratios=[3.0, 1.0] if plan else [1],
-                          hspace=0.14 if plan else 0.0)
-    ax0 = fig.add_subplot(gs[0])
+    fig, ax0 = plt.subplots(figsize=(13, 3.2))
     ax0.imshow(mel, aspect="auto", origin="lower",
                cmap="magma", extent=[0, dur, 0, N_MELS])
     ax0.set_ylabel("mel")
     ax0.set_title(f"{title} — {course}  |  full-song mel spectrogram")
     ax0.set_xlim(0, dur)
     ax0.grid(axis="x", alpha=0.18)
-    if plan:
-        ax0.set_xticklabels([])
-        ax1 = fig.add_subplot(gs[1], sharex=ax0)
-        for a, b, d, f in plan:
-            c = "#d64545" if f == 2 else ("#4a90d9" if f == 1 else "#999999")
-            ax1.bar((a + b) / 2, max(d, 0.15), width=max((b - a) * 0.92, 0.01),
-                    color=c, alpha=0.85)
-        ax1.set_xlim(0, dur)
-        ax1.set_ylim(0, 8)
-        ax1.set_yticks([0, 4, 8])
-        ax1.set_ylabel("plan", fontsize=8)
-        ax1.set_xlabel("time (s)  —  plan: grey=density blue=gap red=climax")
-        ax1.grid(axis="x", alpha=0.18)
-    else:
-        ax0.set_xlabel("time (s)")
+    ax0.set_xlabel("time (s)")
     fig.savefig(out_path, dpi=130, bbox_inches="tight")
     plt.close(fig)
     return out_path
@@ -354,7 +308,13 @@ async def homepage():
 
 @app.get("/health", include_in_schema=False)
 async def health():
-    return {"status": "ok", "device": DEVICE, "models_loaded": "gen" in _MODELS}
+    return {
+        "status": "ok",
+        "version": "1.8",
+        "model": MODEL_REPO,
+        "device": DEVICE,
+        "model_loaded": "generator" in _MODEL,
+    }
 
 
 @app.api(
@@ -371,14 +331,11 @@ def generate_chart(
     course: str,
     level: int,
     bpm_override: float,
-    auto_plan_on: bool,
     use_beat: bool,
-    use_planner: bool,
     sampling: bool,
     temperature: float,
     top_p: float,
     drum_volume: float,
-    model_choice: str = DEFAULT_MODEL,
 ) -> dict[str, object]:
     """Stream the actual inference stages to the custom frontend."""
     workdir = tempfile.mkdtemp(prefix="softchart-request-")
@@ -391,15 +348,11 @@ def generate_chart(
         top_p = float(top_p)
         drum_volume = float(drum_volume)
 
-        if model_choice not in MODELS:
-            raise ValueError(
-                f"Unknown model {model_choice!r}. Choose one of: {', '.join(MODELS)}."
-            )
         yield _progress(
             "loading", 0.03, "Preparing",
-            f"Loading SoftChart {model_choice}.",
+            "Loading SoftChart V1.8 hierarchical.",
         )
-        models = get_models(model_choice, include_planner=use_planner)
+        model = get_model()
 
         yield _progress(
             "audio", 0.11, "Listening",
@@ -413,7 +366,7 @@ def generate_chart(
         )
         grid = dbs = None
         if use_beat:
-            grid = fit_grid_piecewise(models["beat"], mel, device=DEVICE)
+            grid = fit_grid_piecewise(model, mel, device=DEVICE)
             if grid is not None:
                 dbs = grid["downbeats"] if grid["ok"] else grid["db_peaks"]
                 if not grid["ok"]:
@@ -421,7 +374,7 @@ def generate_chart(
         if bpm_override > 0:
             bpm = bpm_override
             if use_beat and (grid is None or abs(grid["bpm"] - bpm) > 0.5):
-                fixed_grid = fit_grid_fixed_bpm(models["beat"], mel, bpm, device=DEVICE)
+                fixed_grid = fit_grid_fixed_bpm(model, mel, bpm, device=DEVICE)
                 grid = fixed_grid if fixed_grid is not None and fixed_grid["ok"] else None
                 if grid is not None:
                     dbs = grid["downbeats"]
@@ -445,43 +398,25 @@ def generate_chart(
             bpm = float(round(bpm))
 
         yield _progress(
-            "plan", 0.34, "Shaping the arc",
-            "Planning density and climaxes.",
+            "structure", 0.34, "Reading the structure",
+            "Building whole-song hierarchical context.",
         )
-        plan = None
-        if getattr(models["gen"], "_has_plan", False):
-            if use_planner:
-                plan = learned_plan(models["plan"], mel, course, bpm, dbs)
-            elif auto_plan_on:
-                plan = auto_plan(mel, bpm, dbs)
 
         yield _progress(
             "generate", 0.47, "Writing the chart",
             "Writing playable Taiko patterns.",
         )
         title = os.path.splitext(wave_name)[0]
-        slot_used = grid is not None
-        if slot_used:
-            generated = generate_song_slot(
-                models["slot"], mel, grid, course, level=level,
-                density_bucket=COURSE_DENS[course], greedy=not sampling, seed=0,
-                temperature=temperature, top_p=top_p, device=DEVICE, plan=plan,
-            )
-            tja = write_tja_slots(
-                generated, grid, title, course, level, wave_name, plan=plan,
-            )
-        else:
-            generated = generate_song(
-                models["gen"], mel, course, level=level,
-                density_bucket=COURSE_DENS[course], greedy=not sampling,
-                temperature=temperature, top_p=top_p, seed=0,
-                device=DEVICE, plan=plan,
-            )
-            generated = snap_chart(generated, bpm)
-            tja = write_tja(
-                generated, bpm, title, course, level, wave_name, dbs,
-                grid_fit=grid, plan=plan,
-            )
+        generated = generate_song(
+            model, mel, course, level=level,
+            density_bucket=COURSE_DENS[course], greedy=not sampling,
+            temperature=temperature, top_p=top_p, seed=0, device=DEVICE,
+        )
+        generated = snap_chart(generated, bpm)
+        tja = write_tja(
+            generated, bpm, title, course, level, wave_name, dbs,
+            grid_fit=grid,
+        )
 
         yield _progress(
             "export", 0.81, "Rendering",
@@ -491,9 +426,9 @@ def generate_chart(
         with open(tja_path, "w", encoding="utf-8") as output_file:
             output_file.write(tja)
         chart_image_path = os.path.join(workdir, "chart.png")
-        plan_image_path = os.path.join(workdir, "song-plan.png")
+        structure_image_path = os.path.join(workdir, "song-structure.png")
         render_tja_image(tja, out_path=chart_image_path)
-        render_audio_plan(mel, title, course, plan=plan, out_path=plan_image_path)
+        render_song_structure(mel, title, course, out_path=structure_image_path)
 
         yield _progress(
             "mix", 0.91, "Mixing",
@@ -514,7 +449,7 @@ def generate_chart(
             "bpm": round(float(bpm), 1),
             "notes": len(generated["hits"]),
             "spans": len(generated["spans"]),
-            "timing": "slot-exact" if slot_used else "time-quantized",
+            "timing": "hierarchical time + grid quantization",
             "grid_rms_ms": grid_rms,
             "preview_hits": int(mix_stats["rendered_hit_count"]),
         }
@@ -529,7 +464,9 @@ def generate_chart(
                 "tja": _file_data(tja_path, mime_type="text/plain"),
                 "audio": _file_data(preview_path, mime_type="audio/wav"),
                 "chart_image": _file_data(chart_image_path, mime_type="image/png"),
-                "plan_image": _file_data(plan_image_path, mime_type="image/png"),
+                "structure_image": _file_data(
+                    structure_image_path, mime_type="image/png"
+                ),
             },
         }
     except Exception as exc:

packages.txt

@@ -1 +1,2 @@
+ffmpeg
 fonts-noto-cjk

softchart/baseline.py

@@ -1,43 +0,0 @@
-"""Onset-detection baseline: spectral-flux peak picking on the cached log-mel
-+ band-energy don/ka heuristic.
-
-This is the "chart generation is just onset detection" strawman the model must beat.
-Operates on the same cached mel features as the model, so no raw audio is needed.
-"""
-
-import numpy as np
-from scipy.signal import find_peaks
-
-from .vocab import FPS
-
-
-def baseline_chart(mel, target_nps):
-    """mel: (n_mels, T) log-mel (natural log). Returns list of (time_sec, class)."""
-    m = mel.astype(np.float32)
-    # spectral flux onset envelope
-    flux = np.maximum(0.0, np.diff(m, axis=1)).sum(axis=0)
-    flux = np.concatenate([[0.0], flux])
-    # smooth lightly
-    k = np.hanning(5)
-    k /= k.sum()
-    env = np.convolve(flux, k, mode="same")
-
-    dur = m.shape[1] / FPS
-    want = max(1, int(round(target_nps * dur)))
-    peaks, props = find_peaks(env, distance=int(0.09 * FPS), height=0.0)
-    if len(peaks) == 0:
-        return []
-    if len(peaks) > want:
-        strongest = np.argsort(props["peak_heights"])[-want:]
-        peaks = np.sort(peaks[strongest])
-
-    # don/ka: low vs high mel-band energy at the onset (linear power domain)
-    p = np.exp(m)
-    low = p[:40].sum(axis=0)   # ~ <500 Hz
-    high = p[64:].sum(axis=0)  # upper bands
-    events = []
-    for fr in peaks:
-        fr2 = min(fr + 1, m.shape[1] - 1)
-        ratio = high[fr2] / (low[fr2] + 1e-9)
-        events.append((float(fr / FPS), "ka" if ratio > 0.35 else "don"))
-    return events

softchart/checkpoint.py

@@ -0,0 +1,238 @@
+"""Strict generator-checkpoint metadata and architecture reconstruction.
+
+V1.8 condition semantics cannot be recovered from tensor shapes.  A missing
+``dual`` or ``ctx`` flag changes the decoder prefix while all weights still
+load successfully, so release loading must reject incomplete or contradictory
+metadata instead of guessing from state-dict keys.
+"""
+
+from __future__ import annotations
+
+import math
+from collections.abc import Mapping
+from numbers import Integral, Real
+
+import torch
+
+from .vocab import N_MELS, VOCAB
+
+
+REQUIRED_GENERATOR_ARGS = frozenset({
+    "d_model", "nhead", "dropout", "enc_layers", "dec_layers", "ffn",
+    "emb_factor", "func_time", "aux", "global_ctx", "hierarchical_ctx",
+    "section_layers", "section_ffn", "clean_phase", "enc_share",
+    "dec_share", "adapter_rank", "unique_layernorm", "adapter_rank_ffn",
+    "depth_emb", "unshare_last_dec", "align", "beat_head", "beat_hires",
+    "ctx", "sibling", "style", "sync_token", "plan", "mask_infill",
+    "complexity", "slot", "dual",
+})
+
+_BOOL_ARGS = frozenset({
+    "func_time", "aux", "global_ctx", "hierarchical_ctx", "clean_phase",
+    "unique_layernorm", "depth_emb", "unshare_last_dec", "align",
+    "beat_head", "beat_hires", "ctx", "sibling", "style", "sync_token",
+    "plan", "complexity", "slot", "dual",
+})
+
+GENERATOR_CAPABILITY_KEYS = (
+    "ctx", "sibling", "style", "sync_token", "plan", "beat_head", "align",
+    "aux", "func_time", "mask_infill", "complexity", "slot", "dual",
+    "global_ctx", "hierarchical_ctx",
+)
+
+
+def _state_has(state: Mapping[str, torch.Tensor], prefix: str) -> bool:
+    return any(key.startswith(prefix) for key in state)
+
+
+def _require_presence(
+    state: Mapping[str, torch.Tensor], prefix: str, expected: bool, label: str,
+) -> None:
+    present = _state_has(state, prefix)
+    if present != expected:
+        raise ValueError(
+            f"checkpoint args/state disagree for {label}: "
+            f"metadata={expected}, state={present}"
+        )
+
+
+def _positive_int(args: Mapping[str, object], name: str) -> int:
+    value = args[name]
+    if not isinstance(value, Integral) or isinstance(value, bool) or int(value) <= 0:
+        raise ValueError(f"checkpoint arg {name} must be a positive integer")
+    return int(value)
+
+
+def _nonnegative_int(args: Mapping[str, object], name: str) -> int:
+    value = args[name]
+    if not isinstance(value, Integral) or isinstance(value, bool) or int(value) < 0:
+        raise ValueError(f"checkpoint arg {name} must be a non-negative integer")
+    return int(value)
+
+
+def parse_generator_checkpoint(checkpoint: Mapping[str, object]) -> tuple[
+    Mapping[str, torch.Tensor], dict, dict, dict
+]:
+    """Validate and return ``state, args, model_config, capabilities``.
+
+    The returned configuration is suitable for ``ChartModel``.  Tensor shapes
+    are still checked a second time by its strict ``load_state_dict`` call.
+    """
+    if not isinstance(checkpoint, Mapping):
+        raise ValueError("generator checkpoint must be a mapping")
+    state = checkpoint.get("model")
+    args = checkpoint.get("args")
+    if not isinstance(state, Mapping) or not state:
+        raise ValueError("generator checkpoint model must be a non-empty mapping")
+    if not isinstance(args, Mapping):
+        raise ValueError("generator checkpoint args must be a mapping")
+    if not all(isinstance(key, str) and isinstance(value, torch.Tensor)
+               for key, value in state.items()):
+        raise ValueError("generator state must map string keys to tensors")
+    args = dict(args)
+    missing = sorted(REQUIRED_GENERATOR_ARGS - set(args))
+    if missing:
+        raise ValueError(f"generator checkpoint lacks required args: {missing}")
+
+    for name in _BOOL_ARGS:
+        if not isinstance(args[name], bool):
+            raise ValueError(f"checkpoint arg {name} must be boolean")
+    for name in ("d_model", "nhead", "enc_layers", "dec_layers", "ffn",
+                 "section_layers"):
+        _positive_int(args, name)
+    for name in ("emb_factor", "section_ffn", "adapter_rank",
+                 "adapter_rank_ffn"):
+        _nonnegative_int(args, name)
+    dropout = args["dropout"]
+    if (not isinstance(dropout, Real) or isinstance(dropout, bool)
+            or not math.isfinite(float(dropout)) or not 0.0 <= float(dropout) < 1.0):
+        raise ValueError("checkpoint arg dropout must be finite in [0, 1)")
+    if args["enc_share"] is not None and not isinstance(args["enc_share"], (str, int)):
+        raise ValueError("checkpoint arg enc_share must be null, integer, or string")
+    if args["dec_share"] is not None and not isinstance(args["dec_share"], (str, int)):
+        raise ValueError("checkpoint arg dec_share must be null, integer, or string")
+    if args["global_ctx"] and args["hierarchical_ctx"]:
+        raise ValueError("global_ctx and hierarchical_ctx are mutually exclusive")
+    if args["hierarchical_ctx"] and not args["aux"]:
+        raise ValueError("hierarchical_ctx requires aux skeleton supervision")
+    if args["beat_hires"] and not args["beat_head"]:
+        raise ValueError("beat_hires requires beat_head")
+    if (args["slot"] or args["dual"]) and not args["clean_phase"]:
+        raise ValueError("V1.8 slot/dual checkpoints require clean_phase")
+    if args["complexity"]:
+        raise ValueError(
+            "complexity-conditioned checkpoints are not supported by the "
+            "release generation API"
+        )
+
+    required_tensors = ("tok_emb.weight", "frontend.0.weight")
+    absent = [name for name in required_tensors if name not in state]
+    if absent:
+        raise ValueError(f"generator state lacks required tensors: {absent}")
+    token_weight = state["tok_emb.weight"]
+    front_weight = state["frontend.0.weight"]
+    if token_weight.ndim != 2 or front_weight.ndim != 3:
+        raise ValueError("generator embedding/frontend tensors have invalid rank")
+
+    d_model = int(args["d_model"])
+    nhead = int(args["nhead"])
+    if d_model % nhead:
+        raise ValueError("checkpoint d_model must be divisible by nhead")
+    vocab_size, embedding_width = map(int, token_weight.shape)
+    if vocab_size != VOCAB.size:
+        raise ValueError(
+            f"checkpoint vocab has {vocab_size} rows; current exact vocabulary "
+            f"requires {VOCAB.size}"
+        )
+    emb_factor = int(args["emb_factor"])
+    expected_width = emb_factor or d_model
+    if embedding_width != expected_width:
+        raise ValueError(
+            "checkpoint args/state disagree for emb_factor: "
+            f"expected width {expected_width}, state has {embedding_width}"
+        )
+    if int(front_weight.shape[0]) != d_model:
+        raise ValueError("frontend output channels disagree with d_model")
+    expected_front_channels = (
+        N_MELS if args["clean_phase"]
+        else (N_MELS + 2 if (args["slot"] or args["dual"]) else N_MELS)
+    )
+    if int(front_weight.shape[1]) != expected_front_channels:
+        raise ValueError(
+            "frontend input channels disagree with clean slot/time metadata: "
+            f"expected {expected_front_channels}, state has {front_weight.shape[1]}"
+        )
+
+    _require_presence(state, "emb_proj.", emb_factor > 0, "emb_factor")
+    _require_presence(
+        state, "section_encoder.", args["hierarchical_ctx"], "hierarchical_ctx")
+    _require_presence(
+        state, "skeleton_proj.", args["hierarchical_ctx"], "hierarchical skeleton")
+    _require_presence(state, "gsum_proj.", args["global_ctx"], "global_ctx")
+    _require_presence(state, "aux.", args["aux"], "aux")
+    _require_presence(state, "phase_proj.", args["clean_phase"], "clean_phase")
+    _require_presence(state, "time_proj.", args["func_time"], "func_time")
+    _require_presence(state, "ptr.", args["align"], "align")
+    _require_presence(state, "beat.", args["beat_head"], "beat_head")
+    _require_presence(
+        state, "beat.up.", args["beat_head"] and args["beat_hires"], "beat_hires")
+
+    beat: bool | str = (
+        "hires" if args["beat_head"] and args["beat_hires"]
+        else bool(args["beat_head"])
+    )
+    model_config = {
+        "d_model": d_model,
+        "nhead": nhead,
+        "dropout": float(dropout),
+        "enc_layers": int(args["enc_layers"]),
+        "dec_layers": int(args["dec_layers"]),
+        "ffn": int(args["ffn"]),
+        "vocab_size": vocab_size,
+        "aux": bool(args["aux"]),
+        "global_ctx": bool(args["global_ctx"]),
+        "hierarchical_ctx": bool(args["hierarchical_ctx"]),
+        "section_layers": int(args["section_layers"]),
+        "section_ffn": int(args["section_ffn"]) or None,
+        "func_time": bool(args["func_time"]),
+        "ptr": bool(args["align"]),
+        "beat": beat,
+        "in_ch": N_MELS + 2 if (args["slot"] or args["dual"]) else N_MELS,
+        "emb_factor": emb_factor or None,
+        "clean_phase": bool(args["clean_phase"]),
+        "enc_share": args["enc_share"],
+        "dec_share": args["dec_share"],
+        "adapter_rank": int(args["adapter_rank"]),
+        "unique_layernorm": bool(args["unique_layernorm"]),
+        "adapter_rank_ffn": int(args["adapter_rank_ffn"]),
+        "depth_emb": bool(args["depth_emb"]),
+        "unshare_last_dec": bool(args["unshare_last_dec"]),
+    }
+    capabilities = {
+        name: (bool(args[name]) if name != "hierarchical_ctx"
+               else bool(args["hierarchical_ctx"]))
+        for name in GENERATOR_CAPABILITY_KEYS
+    }
+    return state, args, model_config, capabilities
+
+
+def attach_generator_capabilities(model, capabilities: Mapping[str, bool]) -> None:
+    """Attach the exact semantic flags consumed by generation."""
+    missing = sorted(set(GENERATOR_CAPABILITY_KEYS) - set(capabilities))
+    if missing:
+        raise ValueError(f"generator capabilities are incomplete: {missing}")
+    invalid = sorted(
+        name for name in GENERATOR_CAPABILITY_KEYS
+        if not isinstance(capabilities[name], bool)
+    )
+    if invalid:
+        raise ValueError(f"generator capabilities must be boolean: {invalid}")
+    model._has_ctx = bool(capabilities["ctx"])
+    model._has_sib = bool(capabilities["sibling"])
+    model._has_style = bool(capabilities["style"])
+    model._has_sync = bool(capabilities["sync_token"])
+    model._has_plan = bool(capabilities["plan"])
+    model._dual = bool(capabilities["dual"])
+    model._slot = bool(capabilities["slot"]) or model._dual
+    model._hierarchical_ctx = bool(capabilities["hierarchical_ctx"])
+    model._has_gctx = bool(capabilities["global_ctx"] or model._hierarchical_ctx)

softchart/data.py

@@ -1,590 +0,0 @@
-"""PyTorch dataset: (mel window, token sequence) pairs from the preprocessed cache."""
-
-import json
-import os
-import random
-
-import numpy as np
-import torch
-from torch.utils.data import Dataset
-
-from .vocab import (FPS, MAX_TGT, MEAS_MAX, NOTE_CLASSES, SLOTS, VOCAB, WINDOW,
-                    encode_window)
-
-G_CHUNK = 172  # ~2s per whole-song summary chunk
-G_LEN = 100  # fixed number of summary chunks (covers ~200s songs)
-
-
-def song_summary(mel):
-    """(n_mels, T) log-mel -> (G_LEN, n_mels) coarse whole-song summary."""
-    n_mels, T = mel.shape
-    n = min(G_LEN, max(1, T // G_CHUNK))
-    out = np.full((G_LEN, n_mels), np.log(1e-5), dtype=np.float32)
-    for k in range(n):
-        out[k] = mel[:, k * G_CHUNK : (k + 1) * G_CHUNK].astype(np.float32).mean(axis=1)
-    return out
-
-
-class ChartWindowDataset(Dataset):
-    """One item = one (song, course) chart; a random window is cropped per access."""
-
-    def __init__(
-        self,
-        cache_dir,
-        song_ids,
-        train=True,
-        cond_drop=0.15,
-        spec_augment=True,
-        windows_per_chart=8,
-        use_ctx=False,
-        aux=False,
-        global_ctx=False,
-        importance_sampling=False,
-        tempo_aug=False,
-        sibling=False,
-        style=False,
-        beat_head=False,
-        sync_token=False,
-        plan=False,
-        mask_infill=0.0,
-        complexity=False,
-        slot=False,
-        dual=False,
-        dual_slot_p=0.65,
-        beat_hires=False,
-    ):
-        self.cache = cache_dir
-        self.train = train
-        self.slot = slot
-        self.dual = dual  # mixed slot/time batches with an explicit mode token
-        self.dual_slot_p = dual_slot_p
-        self._meas_cache = {}
-        self.cond_drop = cond_drop if train else 0.0
-        self.spec_augment = spec_augment and train
-        self.use_ctx = use_ctx
-        self.aux = aux
-        self.global_ctx = global_ctx
-        self.importance = importance_sampling and train
-        self.tempo_aug = tempo_aug and train
-        self.sibling = sibling
-        self.beat_head = beat_head
-        self.beat_hires = beat_hires
-        self.sync_token = sync_token
-        self.mask_infill = mask_infill if train else 0.0
-        self.complexity = complexity
-        self.plans = None
-        if plan:
-            with open(os.path.join(cache_dir, "plans.json")) as f:
-                self.plans = json.load(f)
-        self.styles = None
-        if style:
-            with open(os.path.join(cache_dir, "styles.json")) as f:
-                self.styles = json.load(f)["styles"]
-        with open(os.path.join(cache_dir, "index.json")) as f:
-            index = {e["id"]: e for e in json.load(f)}
-        self.items = []  # (sid, course, level, n_frames)
-        for sid in song_ids:
-            e = index.get(sid)
-            if e is None:
-                continue
-            for c, info in e["courses"].items():
-                self.items.append((sid, c, info["level"], e["n_frames"]))
-        if self.slot and not self.dual:  # pure slot mode needs a grid per song
-            keep = []
-            for it in self.items:
-                src = it[0].split("x")[0] if "x" in it[0] else it[0]
-                if os.path.exists(os.path.join(cache_dir, f"{src}.beats.npz")):
-                    keep.append(it)
-            self.items = keep
-        # virtual epoch length: several windows per chart
-        self.mult = windows_per_chart if train else 1
-        self._rng = random.Random(int(os.environ.get('SC_DATA_SEED', 1234)))
-
-    def __len__(self):
-        return len(self.items) * self.mult
-
-    def _load(self, sid):
-        mel = np.load(os.path.join(self.cache, f"{sid}.mel.npy"), mmap_mode="r")
-        notes = np.load(os.path.join(self.cache, f"{sid}.notes.npz"))
-        return mel, notes
-
-    def _beats(self, sid):
-        try:
-            z = np.load(os.path.join(self.cache, f"{sid}.beats.npz"))
-            return z["beats"], z["downbeats"]
-        except Exception:
-            return None, None
-
-    def _measures(self, sid):
-        """Slot mode: (downbeat_times, beats_per_measure, ok_mask) — measure m
-        spans [db[m], db[m+1]). ok = sane meter (2..6 beats) and duration.
-        4/4-only models pass meter_filter=4 at window time."""
-        if sid in self._meas_cache:
-            return self._meas_cache[sid]
-        bts, dbs = self._beats(sid)
-        out = None
-        if dbs is not None and len(dbs) >= 3:
-            db = np.asarray(dbs, np.float64)
-            durs = np.diff(db)
-            if bts is not None and len(bts):
-                nb = np.searchsorted(bts, db[1:] - 1e-4) - np.searchsorted(bts, db[:-1] - 1e-4)
-            else:
-                nb = np.full(len(durs), 4)
-            ok = (nb >= 2) & (nb <= 6) & (durs > 0.5) & (durs < 8.0)
-            if ok.any():
-                out = (db, nb.astype(np.int64), ok)
-        self._meas_cache[sid] = out
-        return out
-
-    def _getitem_slot(self, i):
-        """Slot-token window: measure-aligned crop; note positions are exact
-        TJA lattice indices instead of audio frames. Lattice (v3) is
-        SLOTS_PER_BEAT per beat, so any meter is exact (4/4 -> 96/measure,
-        3/4 -> 72). Slot labels are tempo-INVARIANT, so tempo augmentation
-        only touches the mel. Two phase channels (measure/beat sawtooth) are
-        appended so the decoder reads the grid instead of inferring it."""
-        from .vocab import BEATS_MAX, SLOTS_PER_BEAT
-
-        sid, course, level, n_frames = self.items[i % len(self.items)]
-        src_sid = sid.split("x")[0] if "x" in sid else sid
-        tab = self._measures(src_sid)
-        if tab is None:  # unusable grid: deterministic redirect to another item
-            return self._getitem_slot((i + 9973) % len(self.items))
-        db, nbm, okm = tab
-        mel, notes = self._load(sid)
-        times = notes[f"{course}_t"]
-        classes = notes[f"{course}_c"]
-
-        rng0 = random.Random(i * 31337 + (self._rng.randint(0, 1 << 30) if self.train else 0))
-        rng = random.Random(i * 7919 + (self._rng.randint(0, 1 << 30) if self.train else 0))
-        r = 1.0
-        if self.tempo_aug and rng0.random() < 0.5:
-            r = rng0.uniform(0.9, 1.111)
-
-        # start measure: importance-weighted among valid measures
-        cand = np.nonzero(okm)[0]
-        cand = cand[db[cand] * FPS < max(n_frames - 1, 1)]
-        if len(cand) == 0:
-            return self._getitem_slot((i + 9973) % len(self.items))
-        if self.train:
-            if self.importance:
-                lo = np.searchsorted(times, db[cand])
-                hi = np.searchsorted(times, db[cand] + WINDOW / FPS * r)
-                w = (hi - lo).astype(np.float64) + 3.0
-                j = int(rng.choices(cand.tolist(), weights=w.tolist())[0])
-            else:
-                j = int(rng.choice(cand.tolist()))
-        else:
-            j = int(cand[(i * 2654435761) % len(cand)])
-
-        # extend K whole measures while valid, inside audio, and within both
-        # the frame window and the 72-beat token budget
-        K = 0
-        nbeats = 0
-        while (j + K < len(okm) and okm[j + K]
-               and nbeats + nbm[j + K] <= BEATS_MAX
-               and (db[j + K + 1] - db[j]) * FPS / r <= WINDOW
-               and db[j + K + 1] * FPS <= n_frames + FPS):
-            nbeats += nbm[j + K]
-            K += 1
-        if K == 0:
-            return self._getitem_slot((i + 9973) % len(self.items))
-        t0, t_end = db[j], db[j + K]
-        boff = np.concatenate([[0], np.cumsum(nbm[j : j + K])])  # beat offsets
-
-        def to_slot(t, m):
-            a, b = db[j + m], db[j + m + 1]
-            s = int(np.floor(SLOTS_PER_BEAT * nbm[j + m] * (t - a) / (b - a) + 0.5))
-            return min(boff[m] * SLOTS_PER_BEAT + s, WINDOW - 1)
-
-        # notes -> exact lattice indices; keep frame positions for the aux head
-        pairs, aux_frames = [], []
-        seen = set()
-        for m in range(K):
-            a, b = db[j + m], db[j + m + 1]
-            sel = (times >= a - 1e-4) & (times < b - 1e-4)
-            for t, c in zip(times[sel], classes[sel]):
-                g = to_slot(t, m)
-                if (g, int(c)) not in seen:
-                    seen.add((g, int(c)))
-                    pairs.append((g, int(c)))
-                    aux_frames.append((t - t0) / r * FPS)
-        pairs_sorted = sorted(range(len(pairs)), key=lambda k: pairs[k][0])
-        pairs = [pairs[k] for k in pairs_sorted]
-        aux_frames = [aux_frames[k] for k in pairs_sorted]
-
-        sib = None
-        if self.sibling:
-            order = ["easy", "normal", "hard", "oni", "ura"]
-            sib_pairs = []
-            if not (self.train and rng0.random() < 0.2):
-                for c2 in reversed(order[: order.index(course)]):
-                    if f"{c2}_t" in notes:
-                        t2, c2c = notes[f"{c2}_t"], notes[f"{c2}_c"]
-                        ev = []
-                        for m in range(K):
-                            a, b = db[j + m], db[j + m + 1]
-                            s2 = (t2 >= a - 1e-4) & (t2 < b - 1e-4) & (c2c <= 3)
-                            for t, c in zip(t2[s2], c2c[s2]):
-                                ev.append((to_slot(t, m), int(c)))
-                        if len(ev) > 12:
-                            idx2 = np.linspace(0, len(ev) - 1, 12).astype(int)
-                            ev = [ev[k] for k in idx2]
-                        sib_pairs = ev
-                        break
-            sib = sib_pairs
-
-        style = None
-        if self.styles is not None:
-            style = self.styles.get(f"{sid}:{course}", -1)
-
-        plan_slice = None
-        if self.plans is not None:
-            blocks = self.plans.get(f"{sid}:{course}") or []
-            plan_slice = [(b[2], b[3]) for b in blocks if b[1] > t0 and b[0] < t_end]
-
-        seq, prefix_len = encode_window(
-            VOCAB, course, level, pairs, cond_drop=self.cond_drop, rng=rng,
-            sib_pairs=sib, style=style, plan_slice=plan_slice,
-            mode=("slot" if self.dual else None),
-        )
-        if len(seq) > MAX_TGT:
-            seq = seq[: MAX_TGT - 1] + [VOCAB.eos]
-            if VOCAB.is_time(seq[-2]):
-                seq = seq[:-2] + [VOCAB.eos]
-
-        # mel crop [t0, t_end) -> stretch by r -> pad to WINDOW
-        f0 = int(round(t0 * FPS))
-        src_len = max(1, int(round((t_end - t0) * FPS)))
-        x = mel[:, f0 : f0 + src_len].astype(np.float32)
-        if x.shape[1] < src_len:
-            x = np.pad(x, ((0, 0), (0, src_len - x.shape[1])), constant_values=np.log(1e-5))
-        x = torch.from_numpy(x)
-        tgt_len = min(WINDOW, max(1, int(round(src_len / r))))
-        if tgt_len != src_len:
-            x = torch.nn.functional.interpolate(
-                x[None], size=tgt_len, mode="linear", align_corners=False)[0]
-        if x.shape[1] < WINDOW:
-            x = torch.nn.functional.pad(x, (0, WINDOW - x.shape[1]),
-                                        value=float(np.log(1e-5)))
-        if self.tempo_aug and rng0.random() < 0.3:
-            k = rng0.randint(-4, 4)
-            if k:
-                x = torch.roll(x, k, dims=0)
-                if k > 0:
-                    x[:k] = x.min()
-                else:
-                    x[k:] = x.min()
-        if self.spec_augment:
-            for _ in range(2):
-                w = rng.randint(0, 12)
-                fq = rng.randint(0, x.shape[0] - w) if w else 0
-                if w:
-                    x[fq : fq + w] = x.mean()
-            for _ in range(2):
-                w = rng.randint(0, 24)
-                fq = rng.randint(0, x.shape[1] - w) if w else 0
-                if w:
-                    x[:, fq : fq + w] = x.mean()
-            x = x + (rng.random() * 1.38 - 0.69)
-        # phase channels AFTER augmentation (the grid is clean metadata):
-        # ch0 = measure phase 0..1, ch1 = beat phase 0..1; -1 marks padding
-        ph = torch.full((2, WINDOW), -1.0)
-        edges = (db[j : j + K + 1] - t0) / r * FPS
-        for m in range(K):
-            a, b = edges[m], edges[m + 1]
-            i0, i1 = int(np.ceil(a - 1e-6)), min(int(np.ceil(b - 1e-6)), WINDOW)
-            if i1 <= i0:
-                continue
-            frac = (torch.arange(i0, i1, dtype=torch.float32) - a) / max(b - a, 1e-6)
-            ph[0, i0:i1] = frac
-            ph[1, i0:i1] = (frac * float(nbm[j + m])) % 1.0
-        x = torch.cat([x, ph], dim=0)
-
-        aux_t = torch.zeros(0)
-        if self.aux:
-            aux_t = torch.zeros(WINDOW // 4)
-            for f, (g, c) in zip(aux_frames, pairs):
-                if NOTE_CLASSES[c] != "end":
-                    aux_t[min(max(int(f), 0) // 4, WINDOW // 4 - 1)] = 1.0
-            aux_t = torch.max(aux_t, 0.5 * torch.roll(aux_t, 1))
-            aux_t = torch.max(aux_t, 0.5 * torch.roll(aux_t, -1))
-
-        beat_t = torch.zeros(0)
-        if self.beat_head:
-            # measured LEAK (v15a): slot windows carry grid-phase INPUT
-            # channels, so a beat head trained here just copies the channels
-            # and collapses at inference (p50 723 ms). Mask beat supervision
-            # (-1) on slot windows; the head learns from gridless time-mode
-            # windows in dual training instead.
-            res = WINDOW if self.beat_hires else WINDOW // 4
-            beat_t = torch.full((res, 2), -1.0)
-
-        in_t = torch.tensor(seq, dtype=torch.long)
-        return (x, torch.tensor(seq, dtype=torch.long), prefix_len, aux_t,
-                torch.zeros(0), 0, beat_t, in_t)
-
-    def __getitem__(self, i):
-        if self.dual:
-            sid = self.items[i % len(self.items)][0]
-            src = sid.split("x")[0] if "x" in sid else sid
-            rngm = random.Random(i * 104729 + (self._rng.randint(0, 1 << 30) if self.train else 0))
-            if self._measures(src) is not None and (not self.train or rngm.random() < self.dual_slot_p):
-                return self._getitem_slot(i)
-            # else: fall through to the time path (mode token + blank grid channels)
-        elif self.slot:
-            return self._getitem_slot(i)
-        sid, course, level, n_frames = self.items[i % len(self.items)]
-        mel, notes = self._load(sid)
-        times = notes[f"{course}_t"]
-        classes = notes[f"{course}_c"]
-
-        # rhythm-preserving tempo augmentation: crop a longer/shorter source
-        # window and resample it to WINDOW (mel-domain time stretch); note
-        # times scale by the same factor, so all rhythmic relations survive.
-        rng0 = random.Random(i * 31337 + (self._rng.randint(0, 1 << 30) if self.train else 0))
-        r = 1.0
-        if self.tempo_aug and rng0.random() < 0.5:
-            r = rng0.uniform(0.9, 1.111)
-        src_len = int(round(WINDOW * r))
-
-        max_start = max(0, n_frames - src_len)
-        rng = random.Random(i * 7919 + (self._rng.randint(0, 1 << 30) if self.train else 0))
-        if self.train and max_start > 0:
-            if self.importance:
-                # importance sampling: bias window starts toward note-dense /
-                # informative regions (uniform floor keeps sparse regions seen)
-                cands = np.arange(0, max_start + 1, int(FPS))  # 1s stride
-                lo = np.searchsorted(times, cands / FPS)
-                hi = np.searchsorted(times, cands / FPS + WINDOW / FPS)
-                w = (hi - lo).astype(np.float64) + 3.0
-                start = int(rng.choices(cands.tolist(), weights=w.tolist())[0])
-                start = min(max_start, start + rng.randint(0, int(FPS)))  # sub-second jitter
-            else:
-                start = rng.randint(0, max_start)
-        else:  # deterministic pseudo-random window per item (not always the intro)
-            start = (i * 2654435761) % (max_start + 1) if max_start > 0 else 0
-
-        t0 = start / FPS
-        t1 = (start + src_len) / FPS
-        sel = (times >= t0) & (times < t1)
-        frames = np.round((times[sel] - t0) * FPS / r).astype(np.int64)
-        frames = np.clip(frames, 0, WINDOW - 1)
-        pairs = list(zip(frames.tolist(), classes[sel].tolist()))
-
-        sib = None
-        if self.sibling:  # skeleton hint from the nearest easier course
-            order = ["easy", "normal", "hard", "oni", "ura"]
-            sib_pairs = []
-            if not (self.train and rng0.random() < 0.2):  # p=0.2: train without hint
-                for c2 in reversed(order[: order.index(course)]):
-                    if f"{c2}_t" in notes:
-                        t2, c2c = notes[f"{c2}_t"], notes[f"{c2}_c"]
-                        s2 = (t2 >= t0) & (t2 < t1) & (c2c <= 3)  # hits only
-                        f2 = np.clip(np.round((t2[s2] - t0) * FPS / r), 0, WINDOW - 1)
-                        ev = list(zip(f2.astype(np.int64).tolist(), c2c[s2].tolist()))
-                        if len(ev) > 12:  # even subsample to 12 slots
-                            idx2 = np.linspace(0, len(ev) - 1, 12).astype(int)
-                            ev = [ev[k] for k in idx2]
-                        sib_pairs = ev
-                        break
-            sib = sib_pairs
-
-        ctx = None
-        if self.use_ctx:
-            from .generate import CTX_LEN, HIT_CLASSES
-
-            hit_ids = [NOTE_CLASSES.index(c) for c in HIT_CLASSES]
-            prev = (times >= t0 - 3.0) & (times < t0)
-            tail = [int(c) for c in classes[prev] if int(c) in hit_ids][-CTX_LEN:]
-            if self.train and rng.random() < 0.1:
-                tail = []  # simulate missing context (first window)
-            elif self.train:
-                tail = [c if rng.random() > 0.1 else rng.choice(hit_ids) for c in tail]
-            ctx = [-1] * (CTX_LEN - len(tail)) + tail
-
-        style = None
-        if self.styles is not None:
-            style = self.styles.get(f"{sid}:{course}", -1)
-
-        sync_band = None
-        beats_w = db_w = None
-        if self.sync_token or self.beat_head:
-            src_sid = sid.split("x")[0] if "x" in sid else sid
-            bts, dbs = self._beats(src_sid)
-            if bts is not None:
-                # window-local, tempo-scaled beat grid
-                bsel = (bts >= t0) & (bts < t1)
-                beats_w = (bts[bsel] - t0) / r
-                dsel = (dbs >= t0) & (dbs < t1)
-                db_w = (dbs[dsel] - t0) / r
-        if self.sync_token:
-            sync_band = lhl_band(
-                np.array([f / FPS for f, c in pairs if NOTE_CLASSES[c] != "end"]),
-                beats_w, db_w)
-
-        plan_slice = None
-        if self.plans is not None:
-            blocks = self.plans.get(f"{sid}:{course}") or []
-            # blocks overlapping this window; times already in source timescale,
-            # window is [t0, t1) in the same scale (r rescales note frames only,
-            # block membership is decided in source time)
-            plan_slice = [(b[2], b[3]) for b in blocks if b[1] > t0 and b[0] < t1]
-
-        cplx = None
-        if self.complexity:
-            from .vocab import complexity_band
-            cplx = complexity_band(pairs)
-
-        seq, prefix_len = encode_window(
-            VOCAB, course, level, pairs, cond_drop=self.cond_drop, rng=rng,
-            ctx_types=ctx, sib_pairs=sib, style=style, sync_band=sync_band,
-            plan_slice=plan_slice, complexity=cplx,
-            mode=("time" if self.dual else None),
-        )
-        in_seq = None
-        if self.mask_infill > 0 and rng.random() < self.mask_infill:
-            # skeleton->color curriculum: decoder INPUT sees MASK where note
-            # types were; gold targets keep the true types
-            note_ids = set(VOCAB.note.values())
-            in_seq = [VOCAB.mask if (i >= prefix_len and tok in note_ids) else tok
-                      for i, tok in enumerate(seq)]
-        if len(seq) > MAX_TGT:  # truncate overly dense windows, keep EOS
-            seq = seq[: MAX_TGT - 1] + [VOCAB.eos]
-            if VOCAB.is_time(seq[-2]):  # do not end on a dangling TIME token
-                seq = seq[:-2] + [VOCAB.eos]
-
-        x = mel[:, start : start + src_len].astype(np.float32)
-        if x.shape[1] < src_len:
-            x = np.pad(x, ((0, 0), (0, src_len - x.shape[1])), constant_values=np.log(1e-5))
-        x = torch.from_numpy(x)
-        if src_len != WINDOW:  # mel-domain time stretch to the fixed window size
-            x = torch.nn.functional.interpolate(
-                x[None], size=WINDOW, mode="linear", align_corners=False
-            )[0]
-        if self.tempo_aug and rng0.random() < 0.3:  # mild pitch shift (mel-bin roll)
-            k = rng0.randint(-4, 4)
-            if k:
-                x = torch.roll(x, k, dims=0)
-                if k > 0:
-                    x[:k] = x.min()
-                else:
-                    x[k:] = x.min()
-        if self.spec_augment:
-            for _ in range(2):  # freq masks
-                w = rng.randint(0, 12)
-                f0 = rng.randint(0, x.shape[0] - w) if w else 0
-                if w:
-                    x[f0 : f0 + w] = x.mean()
-            for _ in range(2):  # time masks
-                w = rng.randint(0, 24)
-                f0 = rng.randint(0, x.shape[1] - w) if w else 0
-                if w:
-                    x[:, f0 : f0 + w] = x.mean()
-            x = x + (rng.random() * 1.38 - 0.69)  # gain +-6 dB in log space
-        if self.dual:  # time mode carries no grid: phase channels = -1
-            x = torch.cat([x, torch.full((2, WINDOW), -1.0)], dim=0)
-
-        aux_t = torch.zeros(0)
-        if self.aux:  # onset heatmap at encoder resolution (WINDOW // 4)
-            aux_t = torch.zeros(WINDOW // 4)
-            for f, c in pairs:
-                if NOTE_CLASSES[c] != "end":
-                    aux_t[min(f // 4, WINDOW // 4 - 1)] = 1.0
-            aux_t = torch.max(aux_t, 0.5 * torch.roll(aux_t, 1))
-            aux_t = torch.max(aux_t, 0.5 * torch.roll(aux_t, -1))
-
-        beat_t = torch.zeros(0)
-        if self.beat_head:
-            res = WINDOW if self.beat_hires else WINDOW // 4
-            div = 1 if self.beat_hires else 4
-            beat_t = torch.zeros(res, 2)
-            for arr, ch in ((beats_w, 0), (db_w, 1)):
-                if arr is None:
-                    continue
-                for bt in arr:
-                    k = int(bt * FPS) // div
-                    if 0 <= k < res:
-                        beat_t[k, ch] = 1.0
-            for ch in range(2):
-                col = beat_t[:, ch]
-                beat_t[:, ch] = torch.max(col, 0.5 * torch.roll(col, 1))
-                beat_t[:, ch] = torch.max(beat_t[:, ch], 0.5 * torch.roll(col, -1))
-
-        gsum = torch.zeros(0)
-        posb = 0
-        if self.global_ctx:
-            gsum = torch.from_numpy(song_summary(np.asarray(mel)))
-            posb = min(15, int(16 * start / max(n_frames, 1)))
-
-        in_t = torch.tensor(in_seq if in_seq is not None else seq, dtype=torch.long)
-        return (x, torch.tensor(seq, dtype=torch.long), prefix_len, aux_t, gsum, posb,
-                beat_t, in_t)
-
-
-def lhl_band(note_times, beats, downbeats, band_width=0.6):
-    """Longuet-Higgins & Lee (1984) style syncopation, simplified to the
-    eighth-note grid, normalized per bar, bucketed into 6 bands."""
-    if beats is None or len(beats) < 4 or len(note_times) < 2:
-        return -1
-    beat_len = float(np.median(np.diff(beats))) if len(beats) > 1 else 0.5
-    grid = []  # (time, metrical level)
-    dbset = set(np.round(downbeats, 3)) if downbeats is not None else set()
-    for b in beats:
-        lvl = 3 if round(float(b), 3) in dbset else 2
-        grid.append((float(b), lvl))
-        grid.append((float(b) + beat_len / 2, 1))  # eighth positions
-    grid.sort()
-    tol = 0.04
-    score = 0.0
-    for i, (gt_pos, lvl) in enumerate(grid[:-1]):
-        has_note = np.any(np.abs(note_times - gt_pos) < tol)
-        nxt_pos, nxt_lvl = grid[i + 1]
-        nxt_note = np.any(np.abs(note_times - nxt_pos) < tol)
-        if has_note and not nxt_note and nxt_lvl > lvl:
-            score += nxt_lvl - lvl
-    n_bars = max(1.0, len([g for g in grid if g[1] == 3]))
-    return min(5, int(score / n_bars / band_width))
-
-
-def collate(batch):
-    xs, seqs, plens, auxs, gsums, posbs, beats, inseqs = zip(*batch)
-    x = torch.stack(xs)
-    maxlen = max(len(s) for s in seqs)
-    tgt = torch.full((len(seqs), maxlen), VOCAB.pad, dtype=torch.long)
-    loss_mask = torch.zeros((len(seqs), maxlen), dtype=torch.bool)
-    for i, (s, pl) in enumerate(zip(seqs, plens)):
-        tgt[i, : len(s)] = s
-        loss_mask[i, pl : len(s)] = True  # loss on events + EOS only
-    aux = torch.stack(auxs) if auxs[0].numel() else None
-    gsum = torch.stack(gsums) if gsums[0].numel() else None
-    posb = torch.tensor(posbs, dtype=torch.long)
-    beat = torch.stack(beats) if beats[0].numel() else None
-    in_tgt = torch.full((len(inseqs), maxlen), VOCAB.pad, dtype=torch.long)
-    for i, sq in enumerate(inseqs):
-        in_tgt[i, : len(sq)] = sq
-    return x, tgt, loss_mask, aux, gsum, posb, beat, in_tgt
-
-
-def load_split_ids(cache_dir, val_songs=40, seed=42, use_aug=False):
-    """Song-level split: train / val (carved from train) / test (provided).
-
-    Augmented variants (speed-aug entries with "aug": true) are never used for
-    val, and variants whose source song is in val are excluded from train
-    (no leakage). The base split is stable regardless of augmentation.
-    """
-    with open(os.path.join(cache_dir, "index.json")) as f:
-        index = json.load(f)
-    base_train = sorted(e["id"] for e in index
-                        if e["split"] == "train" and not e.get("aug"))
-    test_ids = sorted(e["id"] for e in index if e["split"] == "test")
-    rng = random.Random(seed)
-    rng.shuffle(base_train)
-    val_ids = base_train[:val_songs]
-    train_ids = base_train[val_songs:]
-    if use_aug:
-        val_set = set(val_ids)
-        train_ids = train_ids + sorted(
-            e["id"] for e in index
-            if e.get("aug") and e.get("src") not in val_set)
-    return train_ids, val_ids, test_ids

softchart/evaluate.py

@@ -1,151 +0,0 @@
-"""Objective evaluation metrics for generated charts."""
-
-import numpy as np
-
-HIT_CLASSES = ("don", "ka", "don_big", "ka_big")
-
-
-def match_onsets(ref_times, est_times, tol=0.05):
-    """Greedy bipartite matching (mir_eval style). Returns list of (ri, ei)."""
-    matches = []
-    ri, ei = 0, 0
-    used_r = set()
-    # standard approach: for each est, match nearest unmatched ref within tol
-    ref = np.asarray(ref_times)
-    est = np.asarray(est_times)
-    order = np.argsort(est)
-    ref_order = np.argsort(ref)
-    ref_sorted = ref[ref_order]
-    taken = np.zeros(len(ref), dtype=bool)
-    for e_i in order:
-        t = est[e_i]
-        lo = np.searchsorted(ref_sorted, t - tol)
-        hi = np.searchsorted(ref_sorted, t + tol)
-        best, best_d = -1, tol + 1
-        for k in range(lo, hi):
-            rid = ref_order[k]
-            if taken[rid]:
-                continue
-            d = abs(ref_sorted[k] - t)
-            if d < best_d:
-                best, best_d = rid, d
-        if best >= 0:
-            taken[best] = True
-            matches.append((best, e_i))
-    return matches
-
-
-def onset_prf(ref_times, est_times, tol=0.05):
-    if len(ref_times) == 0 and len(est_times) == 0:
-        return 1.0, 1.0, 1.0
-    if len(ref_times) == 0 or len(est_times) == 0:
-        return 0.0, 0.0, 0.0
-    m = len(match_onsets(ref_times, est_times, tol))
-    p = m / len(est_times)
-    r = m / len(ref_times)
-    f = 2 * p * r / (p + r) if p + r > 0 else 0.0
-    return p, r, f
-
-
-def type_accuracy(ref, est, tol=0.05):
-    """ref/est: list of (time, class). Accuracy + per-class stats on matched onsets."""
-    if not ref or not est:
-        return None
-    rt = [t for t, _ in ref]
-    et = [t for t, _ in est]
-    matches = match_onsets(rt, et, tol)
-    if not matches:
-        return None
-    correct = sum(1 for ri, ei in matches if ref[ri][1] == est[ei][1])
-    per_class = {}
-    for ri, ei in matches:
-        c = ref[ri][1]
-        d = per_class.setdefault(c, [0, 0])
-        d[1] += 1
-        if est[ei][1] == c:
-            d[0] += 1
-    # binary don/ka accuracy (big variants folded in)
-    fold = lambda c: "don" if "don" in c else ("ka" if "ka" in c else c)
-    correct2 = sum(1 for ri, ei in matches if fold(ref[ri][1]) == fold(est[ei][1]))
-    return {
-        "acc": correct / len(matches),
-        "acc_donka": correct2 / len(matches),
-        "n_matched": len(matches),
-        "per_class": {c: v[0] / v[1] for c, v in per_class.items()},
-    }
-
-
-def density(events, span=None):
-    times = [t for t, c in events if c in HIT_CLASSES]
-    if len(times) < 2:
-        return 0.0
-    span = span or (max(times) - min(times))
-    return len(times) / max(span, 1e-6) if span > 5 else 0.0
-
-
-def ioi_hist(events, bpm, bins=None):
-    """Inter-onset intervals in beat units, snapped to musical fractions."""
-    if bins is None:
-        bins = [0.125, 1 / 6, 0.25, 1 / 3, 0.5, 2 / 3, 0.75, 1.0, 1.5, 2.0, 3.0, 4.0]
-    times = sorted(t for t, c in events if c in HIT_CLASSES)
-    if len(times) < 3 or not bpm or bpm <= 0:
-        return None
-    ioi = np.diff(times) * bpm / 60.0  # beats
-    hist = np.zeros(len(bins) + 1)
-    for x in ioi:
-        d = [abs(x - b) / b for b in bins]
-        j = int(np.argmin(d))
-        if d[j] < 0.2:
-            hist[j] += 1
-        else:
-            hist[-1] += 1  # off-grid
-    s = hist.sum()
-    return hist / s if s > 0 else None
-
-
-def js_divergence(p, q, eps=1e-9):
-    p = np.asarray(p) + eps
-    q = np.asarray(q) + eps
-    p, q = p / p.sum(), q / q.sum()
-    m = (p + q) / 2
-    kl = lambda a, b: float(np.sum(a * np.log(a / b)))
-    return 0.5 * kl(p, m) + 0.5 * kl(q, m)
-
-
-def ngram_dist(events, n=2):
-    """Distribution over note-class n-grams (hits only)."""
-    seq = [c for _, c in sorted(events) if c in HIT_CLASSES]
-    if len(seq) < n + 1:
-        return {}
-    counts = {}
-    for i in range(len(seq) - n + 1):
-        g = tuple(seq[i : i + n])
-        counts[g] = counts.get(g, 0) + 1
-    tot = sum(counts.values())
-    return {g: c / tot for g, c in counts.items()}
-
-
-def ngram_js(ref_events, est_events, n=2):
-    pr = ngram_dist(ref_events, n)
-    pe = ngram_dist(est_events, n)
-    if not pr or not pe:
-        return None
-    keys = sorted(set(pr) | set(pe))
-    return js_divergence([pr.get(k, 0) for k in keys], [pe.get(k, 0) for k in keys])
-
-
-def distinct_n(events, n=3):
-    seq = [c for _, c in sorted(events) if c in HIT_CLASSES]
-    if len(seq) < n:
-        return None
-    grams = [tuple(seq[i : i + n]) for i in range(len(seq) - n + 1)]
-    return len(set(grams)) / len(grams)
-
-
-def spearman(x, y):
-    from scipy.stats import spearmanr
-
-    if len(x) < 2:
-        return None
-    r = spearmanr(x, y).statistic
-    return None if np.isnan(r) else float(r)

softchart/generate.py

@@ -12,9 +12,26 @@ Features:
 import numpy as np
 import torch
 
+from .checkpoint import (attach_generator_capabilities,
+                         parse_generator_checkpoint)
+from .meter import resolve_meter_layout
 from .model import ChartModel
 from .vocab import FPS, MAX_TGT, N_LEVELS, NOTE_CLASSES, VOCAB, WINDOW
 
+G_LEN = 100
+
+
+def song_summary(mel):
+    """Summarize a complete song into fixed relative-time sections."""
+    n_mels, frames = mel.shape
+    if frames < G_LEN:
+        raise ValueError(f"song has only {frames} mel frames; need at least {G_LEN}")
+    edges = np.linspace(0, frames, G_LEN + 1, dtype=np.int64)
+    summary = np.empty((G_LEN, n_mels), dtype=np.float32)
+    for index, (start, end) in enumerate(zip(edges[:-1], edges[1:])):
+        summary[index] = mel[:, start:end].astype(np.float32).mean(axis=1)
+    return summary
+
 HIT_CLASSES = ("don", "ka", "don_big", "ka_big")
 SPAN_CLASSES = ("roll", "roll_big", "balloon")
 
@@ -33,73 +50,15 @@ def _autocast(device):
 CTX_LEN = 12  # fixed-length previous-window context (note-type tokens)
 MIN_GAP_FRAMES = 4  # ~46 ms minimum inter-note gap enforced during decoding
 LATTICE_RATIOS = (1 / 3, 0.5, 2 / 3, 1.0, 4 / 3, 1.5, 2.0, 3.0)  # allowed IOI ratios
-# GT span-length p99 per class (train stats); generated spans beyond this are
-# truncated — a visual audit found generated balloons up to ~10s vs GT p50 ~1s
-SPAN_MAX = {"roll": 3.5, "roll_big": 4.0, "balloon": 6.5}
 
 
 def load_model(ckpt_path, device="cuda"):
-    ck = torch.load(ckpt_path, map_location=device)
-    sd = ck["model"]
-    vocab_size = sd["tok_emb.weight"].shape[0]
-    aux = any(k.startswith("aux") for k in sd)
-    gctx = any(k.startswith("gsum_proj") for k in sd)
-    a = ck.get("args", {}) or {}
-    front_ch = sd["frontend.0.weight"].shape[1]  # 128 (clean) or 130 (legacy slot)
-    emb_w = sd["tok_emb.weight"].shape[1]
-    d_model = a.get("d_model", 512)
-    # clean_phase (v1.6): encoder frontend is 128ch and a phase_proj is present.
-    clean_phase = bool(a.get("clean_phase", False)) or ("phase_proj.0.weight" in sd)
-    # legacy slot models fed phase to the encoder (in_ch=130); clean models keep
-    # in_ch=130 at the data level but only 128 reach the frontend.
-    in_ch = 130 if (clean_phase or front_ch == 130) else front_ch
-    # v1.7 arch flags: prefer saved args; fall back to state-dict keys so
-    # args-less checkpoints of the new arch still load
-    adapter_rank_ffn = int(a.get("adapter_rank_ffn", 0) or 0)
-    if not adapter_rank_ffn and "decoder.ffn_adapters.0.down.weight" in sd:
-        adapter_rank_ffn = sd["decoder.ffn_adapters.0.down.weight"].shape[0]
-    depth_emb = bool(a.get("depth_emb", False)) or ("decoder.depth_emb" in sd)
-    model = ChartModel(
-        d_model=d_model, nhead=a.get("nhead", 8),
-        enc_layers=a.get("enc_layers", 6),
-        dec_layers=a.get("dec_layers", 6), ffn=a.get("ffn", 2048),
-        vocab_size=vocab_size, aux=aux, global_ctx=gctx,
-        func_time=a.get("func_time", False), in_ch=in_ch,
-        emb_factor=emb_w if emb_w != d_model else None,
-        clean_phase=clean_phase,
-        enc_share=a.get("enc_share") or None,
-        dec_share=a.get("dec_share") or None,
-        adapter_rank=int(a.get("adapter_rank", 0) or 0),
-        unique_layernorm=bool(a.get("unique_layernorm", False)),
-        adapter_rank_ffn=adapter_rank_ffn, depth_emb=depth_emb,
-        unshare_last_dec=bool(a.get("unshare_last_dec", False)),
-    ).to(device)
-    if any(k.startswith("ptr") for k in sd):
-        model.enable_ptr()
-    if any(k.startswith("beat") for k in sd):
-        model.enable_beat(hires=any(k.startswith("beat.up") for k in sd))
-    model.to(device)  # newly enabled heads default to CPU
-    model.load_state_dict(sd)
+    checkpoint = torch.load(ckpt_path, map_location=device, weights_only=False)
+    state, _args, config, capabilities = parse_generator_checkpoint(checkpoint)
+    model = ChartModel(**config).to(device)
+    model.load_state_dict(state, strict=True)
     model.eval()
-    # capability detection: prefer explicit training args (vocab-size inference
-    # wrongly added prefix slots to models that never trained with them)
-    if a:
-        model._has_ctx = bool(a.get("ctx", False))
-        model._has_sib = bool(a.get("sibling", False))
-        model._has_style = bool(a.get("style", False))
-        model._has_sync = bool(a.get("sync_token", False))
-        model._has_plan = bool(a.get("plan", False))
-        model._dual = bool(a.get("dual", False))
-        model._slot = bool(a.get("slot", False)) or model._dual
-    else:  # legacy fallback
-        model._has_ctx = vocab_size > VOCAB.sep
-        model._has_sib = vocab_size > VOCAB.sib
-        model._has_style = False
-        model._has_sync = False
-        model._has_plan = False
-        model._slot = False
-        model._dual = False
-    model._has_gctx = gctx
+    attach_generator_capabilities(model, capabilities)
     return model
 
 
@@ -110,20 +69,23 @@ def load_hf(repo_or_dir, device="cuda"):
 
     hf = SoftChartGenerator.from_pretrained(repo_or_dir).to(device).eval()
     m = hf.net
-    caps = hf.capabilities or {}
-    m._has_ctx = bool(caps.get("ctx", False))
-    m._has_sib = bool(caps.get("sibling", False))
-    m._has_style = bool(caps.get("style", False))
-    m._has_sync = bool(caps.get("sync_token", False))
-    m._has_plan = bool(caps.get("plan", False))
-    m._dual = bool(caps.get("dual", False))
-    m._slot = bool(caps.get("slot", False)) or m._dual
-    m._has_gctx = m.gsum_proj is not None
+    caps = hf.capabilities
+    if not isinstance(caps, dict):
+        raise ValueError("HF generator capabilities must be an explicit mapping")
+    attach_generator_capabilities(m, caps)
+    m._softchart_capabilities = dict(caps)
+    if (m.section_encoder is not None) != m._hierarchical_ctx:
+        raise ValueError("HF hierarchy capability disagrees with its architecture")
+    if bool(m.gsum_proj is not None) != bool(caps.get("global_ctx", False)):
+        # New V1.8 artifacts include global_ctx explicitly.  Requiring it here
+        # prevents a semantically different flat-global model from loading as
+        # a local-only generator.
+        raise ValueError("HF global-context capability disagrees with architecture")
     return m
 
 
 def build_prefix(course=None, level=None, density_bucket=None, ctx_types=None,
-                 uncond=False, sib_pairs=None, style=None, sync_band=None,
+                 sib_pairs=None, style=None, sync_band=None,
                  plan_slice=None, mode=None):
     v = VOCAB
     seq = [v.bos]
@@ -142,14 +104,11 @@ def build_prefix(course=None, level=None, density_bucket=None, ctx_types=None,
             cls = c if isinstance(c, int) else NOTE_CLASSES.index(c)
             seq += [v.time(int(f)), v.note[NOTE_CLASSES[cls]]]
         seq += [v.unk_cond] * (2 * (SIB_EVENTS - len(pairs)))
-    if uncond:
-        seq += [v.unk_cond] * 3
-    else:
-        seq += [
-            v.course[course] if course else v.unk_cond,
-            v.level[max(1, min(N_LEVELS, level))] if level else v.unk_cond,
-            v.dens[density_bucket] if density_bucket is not None else v.unk_cond,
-        ]
+    seq += [
+        v.course[course] if course else v.unk_cond,
+        v.level[max(1, min(N_LEVELS, level))] if level else v.unk_cond,
+        v.dens[density_bucket] if density_bucket is not None else v.unk_cond,
+    ]
     if style is not None:  # -1 = style-capable model, no specific style requested
         seq.append(v.style[style] if style >= 0 else v.unk_cond)
     if sync_band is not None:
@@ -165,10 +124,38 @@ def build_prefix(course=None, level=None, density_bucket=None, ctx_types=None,
     return seq
 
 
+def _conditioning_token_ids():
+    """Token ids whose *values* CFG is allowed to remove.
+
+    Structural markers (BOS/SEP/SIB/PLAN/mode), prior-window note context, and
+    sibling-chart events deliberately stay intact.  Positional masking such as
+    ``prefix[:, -n:]`` is wrong as soon as a variable-size plan or another
+    field follows the basic course/level/density tuple.
+    """
+    fields = (
+        VOCAB.course, VOCAB.level, VOCAB.dens, VOCAB.style, VOCAB.sync,
+        VOCAB.cplx, VOCAB.pdens, VOCAB.pflag,
+    )
+    return frozenset(tok for field in fields for tok in field.values())
+
+
+_CONDITIONING_IDS = _conditioning_token_ids()
+
+
+def _make_unconditional_prefixes(prefixes):
+    """Return CFG prefixes with named condition values replaced by UNK."""
+    unc = prefixes.clone()
+    is_condition = torch.zeros_like(unc, dtype=torch.bool)
+    for tok in _CONDITIONING_IDS:
+        is_condition |= unc == tok
+    return unc.masked_fill(is_condition, VOCAB.unk_cond)
+
+
 @torch.no_grad()
 def decode_windows(model, mels, prefixes, device="cuda", temperature=1.0, top_p=0.95,
                    greedy=False, seed=0, cfg_w=0.0, gsum=None, pos_buckets=None,
-                   lattice=False, n_cond=3, min_gap=MIN_GAP_FRAMES):
+                   lattice=False, min_gap=MIN_GAP_FRAMES,
+                   position_limits=None):
     """mels: (B, n_mels, WINDOW); prefixes: list of B equal-length token lists.
     Returns list of lists of (frame, note_class)."""
     v = VOCAB
@@ -193,11 +180,21 @@ def decode_windows(model, mels, prefixes, device="cuda", temperature=1.0, top_p=
 
     seqs = torch.tensor(prefixes, dtype=torch.long, device=device)
     if use_cfg:  # rows B..2B: same ctx, conditions replaced by UNK
-        unc = seqs.clone()
-        unc[:, -n_cond:] = v.unk_cond
+        unc = _make_unconditional_prefixes(seqs)
         seqs = torch.cat([seqs, unc], dim=0)
         memory = torch.cat([memory, memory], dim=0)
 
+    if position_limits is None:
+        position_limits = torch.full((B,), WINDOW, dtype=torch.long, device=device)
+    else:
+        position_limits = torch.as_tensor(position_limits, dtype=torch.long,
+                                          device=device)
+        if position_limits.shape != (B,):
+            raise ValueError(f"position_limits must have shape ({B},), got "
+                             f"{tuple(position_limits.shape)}")
+        if ((position_limits < 1) | (position_limits > WINDOW)).any():
+            raise ValueError(f"position_limits must be within [1, {WINDOW}]")
+
     done = torch.zeros(B, dtype=torch.bool, device=device)
     last_time = torch.full((B,), -min_gap, dtype=torch.long, device=device)
     expect_note = torch.zeros(B, dtype=torch.bool, device=device)
@@ -223,7 +220,8 @@ def decode_windows(model, mels, prefixes, device="cuda", temperature=1.0, top_p=
             time_pos = torch.arange(WINDOW, device=device).unsqueeze(0)
             sub = torch.full((len(idx), logits.shape[1]), float("-inf"), device=device)
             sub[:, v.eos] = 0.0
-            ok = time_pos >= (last_time[idx] + min_gap).unsqueeze(1)
+            ok = ((time_pos >= (last_time[idx] + min_gap).unsqueeze(1))
+                  & (time_pos < position_limits[idx].unsqueeze(1)))
             tmask = torch.where(
                 ok, torch.zeros_like(sub[:, :WINDOW]),
                 torch.full_like(sub[:, :WINDOW], float("-inf")),
@@ -317,13 +315,19 @@ def _decode_pass(model, wins, starts, course, level, density_bucket, ctx_lists,
         g = pb = None
         if gsum is not None:
             g = gsum.unsqueeze(0).expand(chunk.shape[0], -1, -1)
+            n_pos = (100 if getattr(model, "_hierarchical_ctx", False)
+                     else getattr(getattr(model, "pos_emb", None),
+                                  "num_embeddings", 16))
             pb = torch.tensor(
-                [min(15, int(16 * starts[i + j] / max(T, 1))) for j in range(chunk.shape[0])],
+                [min(n_pos - 1, int(n_pos * starts[i + j] / max(T, 1)))
+                 for j in range(chunk.shape[0])],
                 dtype=torch.long)
+        limits = [min(WINDOW, max(1, int(T) - starts[i + j]))
+                  for j in range(chunk.shape[0])]
         evs = decode_windows(model, chunk, prefixes, device=device, greedy=greedy,
                              temperature=temperature, top_p=top_p, seed=seed + i,
                              cfg_w=cfg_w, gsum=g, pos_buckets=pb, lattice=lattice,
-                             n_cond=4 if style is not None else 3)
+                             min_gap=MIN_GAP_FRAMES, position_limits=limits)
         for j, events in enumerate(evs):
             per_window.append(events)
             t_off = starts[i + j] / FPS
@@ -343,7 +347,7 @@ def generate_song(model, mel, course, level=None, density_bucket=None, device="c
     if isinstance(mel, np.ndarray):
         mel = torch.from_numpy(mel.astype(np.float32))
     T = mel.shape[1]
-    starts = list(range(0, max(T - 1, 1), WINDOW))
+    starts = list(range(0, max(T, 1), WINDOW))
     dual = getattr(model, "_dual", False)
     wins = []
     for s in starts:
@@ -367,9 +371,7 @@ def generate_song(model, mel, course, level=None, density_bucket=None, device="c
     plan_default = getattr(model, "_has_plan", False) and plan is None
     gsum = None
     if getattr(model, "_has_gctx", False):
-        from .data import song_summary
-
-        gsum = torch.from_numpy(song_summary(mel.numpy()))
+        gsum = torch.from_numpy(song_summary(mel.detach().cpu().numpy()))
 
     all_events, per_window = _decode_pass(
         model, wins, starts, course, level, density_bucket,
@@ -392,29 +394,6 @@ def generate_song(model, mel, course, level=None, density_bucket=None, device="c
             sync_band=sync_val, plan_blocks=plan_blocks, plan_default=plan_default,
             mode=("time" if dual else None))
 
-    if True:
-        # rescue decoding: decoding occasionally EOSes a whole window early; if a
-        # window is near-empty while its audio is musically active, redo it with
-        # the opposite mode (sampling was empty -> greedy; greedy was empty ->
-        # sampled retry, since a greedy redo would reproduce the same output)
-        flux = [float(np.maximum(0, np.diff(w.numpy(), axis=1)).sum()) for w in wins]
-        med = float(np.median(flux)) if flux else 0.0
-        med_hits = float(np.median([len(ev) for ev in per_window])) if per_window else 0.0
-        retry = [i for i, ev in enumerate(per_window)
-                 if (len(ev) < 4 and flux[i] > 0.3 * med)
-                 or (len(ev) < 0.4 * med_hits and flux[i] > 0.7 * med)]
-        if retry:
-            r_evs, _ = _decode_pass(
-                model, [wins[i] for i in retry], [starts[i] for i in retry],
-                course, level, density_bucket, None, device, not greedy, 0.9,
-                top_p, seed + 7, 0.0, batch_windows, gsum=gsum, T=T,
-                lattice=lattice, sib_default=sib_default, style=style_val,
-                sync_band=sync_val, plan_blocks=plan_blocks, plan_default=plan_default,
-                mode=("time" if dual else None))
-            all_events = [e for i, w_ev in enumerate(per_window) if i not in retry
-                          for e in [(starts[i] / FPS + f / FPS, c) for f, c in w_ev]]
-            all_events += r_evs
-
     all_events.sort(key=lambda e: e[0])
     hits, spans = [], []
     open_span = None
@@ -422,15 +401,10 @@ def generate_song(model, mel, course, level=None, density_bucket=None, device="c
     for t, cls in all_events:
         if cls in HIT_CLASSES:
             if open_span is not None:
-                # a span whose end never arrived would swallow every following
-                # hit (visual audit: 20 s of silence) — force-close at SPAN_MAX
-                if t - open_span[0] > SPAN_MAX.get(open_span[1], 6.5):
-                    spans.append({"t0": round(open_span[0], 4),
-                                  "t1": round(open_span[0] + SPAN_MAX.get(open_span[1], 6.5), 4),
-                                  "type": open_span[1]})
-                    open_span = None
-                else:
-                    continue  # no hits inside an open span
+                # Missing END is malformed decoder output.  Drop the orphaned
+                # span and preserve the authored hit; inventing an endpoint or
+                # swallowing hits would silently rewrite the sequence.
+                open_span = None
             if t - last_hit_t < 0.025:
                 continue
             hits.append({"t": round(t, 4), "type": cls})
@@ -439,52 +413,102 @@ def generate_song(model, mel, course, level=None, density_bucket=None, device="c
             if open_span is None:
                 open_span = (t, cls)
         elif cls == "end":
-            if open_span is not None and t - open_span[0] > 0.05:
-                t1_span = min(t, open_span[0] + SPAN_MAX.get(open_span[1], 6.5))
-                spans.append({"t0": round(open_span[0], 4), "t1": round(t1_span, 4),
-                              "type": open_span[1]})
+            if open_span is not None:
+                if t - open_span[0] > 0.05:
+                    spans.append({"t0": round(open_span[0], 4), "t1": round(t, 4),
+                                  "type": open_span[1]})
+                # END always consumes the open span.  A too-short span is
+                # malformed and dropped; keeping it open would let a later END
+                # close the wrong start and swallow subsequent span starts.
                 open_span = None
     return {"hits": hits, "spans": spans, "course": course, "level": level,
             "density_bucket": density_bucket}
 
 
+def _grid_edges(grid, downbeats):
+    """Return explicitly supplied measure edges without inventing a tail."""
+    if len(downbeats) == 0:
+        raise ValueError("slot generation requires at least one downbeat")
+    if not np.all(np.isfinite(downbeats)) or (np.diff(downbeats) <= 0).any():
+        raise ValueError("grid downbeats must be finite and strictly increasing")
+    if "measure_edges" not in grid:
+        raise ValueError(
+            "slot generation requires explicit measure_edges (starts plus a "
+            "trusted terminal edge); terminal duration is never extrapolated"
+        )
+    edges = np.asarray(grid["measure_edges"], dtype=np.float64)
+    if (len(edges) != len(downbeats) + 1
+            or not np.allclose(edges[:-1], downbeats, rtol=0, atol=1e-6)
+            or not np.all(np.isfinite(edges))
+            or (np.diff(edges) <= 0).any()):
+        raise ValueError("grid measure_edges must be valid starts plus one end")
+    return edges
+
+
 @torch.no_grad()
 def generate_song_slot(model, mel, grid, course, level=None, density_bucket=None,
                        device="cuda", greedy=False, temperature=1.0, top_p=0.95,
                        seed=0, batch_windows=8, plan=None, on_progress=None):
-    """Slot-mode generation: windows are anchored at the fitted barlines and the
-    decoder emits exact TJA lattice indices (measure*96 + slot). No
-    quantization step exists — 'hits_slots' ARE the chart.
+    """Meter-aware generation on the exact 96-slots-per-4/4 TJA lattice.
 
-    mel: (n_mels, T); grid: fit_grid() result (must be trustworthy).
+    Decoder positions are slot-linear across complete measures. Rational
+    meters such as 1/16, 5/8, 7/8, and 8/4 are represented by
+    ``96 * numerator / denominator`` slots. No quantization step exists:
+    ``hits_slots`` are exact per-measure lattice positions.
+
+    mel: (n_mels, T); grid: trusted rational layout with explicit
+    ``measure_edges`` including its terminal edge.
     Returns {hits, spans (seconds, for rendering/metrics),
              hits_slots [(measure, slot, cls)], spans_slots, n_measures}.
     """
-    from .vocab import MEAS_MAX, SLOTS
+    from .vocab import SLOTS
 
     if isinstance(mel, np.ndarray):
         mel = torch.from_numpy(mel.astype(np.float32))
     T = mel.shape[1]
-    dur = T / FPS
     db = np.asarray(grid["downbeats"], np.float64)
-    bar = float(grid["bar"])
-    edges = np.append(db, db[-1] + bar)  # measure m spans [edges[m], edges[m+1])
-
-    wins, metas = [], []  # meta = (first_measure_idx, K)
+    measure_slots, measure_num, measure_den, quarter_beats = \
+        resolve_meter_layout(grid, len(db))
+    if "measure_slot_safe" in grid:
+        slot_safe = np.asarray(grid["measure_slot_safe"])
+        if (slot_safe.dtype != np.bool_ or slot_safe.ndim != 1
+                or len(slot_safe) != len(db)):
+            raise ValueError(
+                "measure_slot_safe must be one boolean per measure")
+        unsafe = np.flatnonzero(~slot_safe)
+        if len(unsafe):
+            raise ValueError(
+                "slot generation refuses unsafe BPM/DELAY/off-lattice "
+                f"measures: {unsafe.astype(int).tolist()}")
+    edges = _grid_edges(grid, db)
+
+    # meta = (first_measure_idx, measure_count, cumulative local slot offsets)
+    wins, metas = [], []
     j = 0
     while j < len(db):
         K = 0
-        while (j + K < len(db) and K < MEAS_MAX
+        nslots = 0
+        while (j + K < len(db)
+               and nslots + int(measure_slots[j + K]) <= WINDOW
                and (edges[j + K + 1] - edges[j]) * FPS <= WINDOW):
+            nslots += int(measure_slots[j + K])
             K += 1
         if K == 0:
-            break
+            raise ValueError(
+                f"measure {j} does not fit the {WINDOW}-frame/{WINDOW}-slot "
+                "slot window"
+            )
         t0 = edges[j]
         f0 = int(round(t0 * FPS))
         src = max(1, int(round((edges[j + K] - t0) * FPS)))
-        x = mel[:, f0 : f0 + src]
-        if x.shape[1] < src:
-            x = torch.nn.functional.pad(x, (0, src - x.shape[1]), value=float(np.log(1e-5)))
+        # Negative chart offsets are real; pad the pre-audio region instead of
+        # letting Python's negative slice wrap around to the song tail.
+        x = torch.full((mel.shape[0], src), float(np.log(1e-5)),
+                       dtype=mel.dtype, device=mel.device)
+        src0, src1 = max(0, f0), min(T, f0 + src)
+        if src1 > src0:
+            dst0 = src0 - f0
+            x[:, dst0:dst0 + (src1 - src0)] = mel[:, src0:src1]
         if x.shape[1] < WINDOW:
             x = torch.nn.functional.pad(x, (0, WINDOW - x.shape[1]), value=float(np.log(1e-5)))
         ph = torch.full((2, WINDOW), -1.0)
@@ -496,22 +520,29 @@ def generate_song_slot(model, mel, grid, course, level=None, density_bucket=None
                 continue
             frac = (torch.arange(i0, i1, dtype=torch.float32) - a) / max(b - a, 1e-6)
             ph[0, i0:i1] = frac
-            ph[1, i0:i1] = (frac * 4) % 1.0
+            ph[1, i0:i1] = (frac * float(quarter_beats[j + m])) % 1.0
         wins.append(torch.cat([x, ph], dim=0))
-        metas.append((j, K))
+        local_offsets = np.concatenate([
+            np.array([0], dtype=np.int64),
+            np.cumsum(measure_slots[j:j + K], dtype=np.int64),
+        ])
+        metas.append((j, K, local_offsets))
         j += K
 
     sib_default = getattr(model, "_has_sib", False)
     style_val = -1 if getattr(model, "_has_style", False) else None
     plan_blocks = plan if getattr(model, "_has_plan", False) else None
     plan_default = getattr(model, "_has_plan", False) and plan is None
+    gsum = None
+    if getattr(model, "_has_gctx", False):
+        gsum = torch.from_numpy(song_summary(mel.detach().cpu().numpy()))
 
     events = []  # (measure_global, slot, cls) in decode order
     for i0 in range(0, len(wins), batch_windows):
         chunk = torch.stack(wins[i0 : i0 + batch_windows])
         prefixes = []
         for b_ in range(chunk.shape[0]):
-            jj, KK = metas[i0 + b_]
+            jj, KK, _ = metas[i0 + b_]
             t0, t1 = edges[jj], edges[jj + KK]
             prefixes.append(build_prefix(
                 course, level, density_bucket,
@@ -520,20 +551,36 @@ def generate_song_slot(model, mel, grid, course, level=None, density_bucket=None
                 plan_slice=([(b[2], b[3]) for b in plan_blocks
                              if b[1] > t0 and b[0] < t1] if plan_blocks is not None
                             else ([] if plan_default else None))))
+        limits = [int(metas[i0 + b_][2][-1])
+                  for b_ in range(chunk.shape[0])]
+        g = pb = None
+        if gsum is not None:
+            g = gsum.unsqueeze(0).expand(chunk.shape[0], -1, -1)
+            n_pos = (100 if getattr(model, "_hierarchical_ctx", False)
+                     else getattr(getattr(model, "pos_emb", None),
+                                  "num_embeddings", 16))
+            pb = torch.tensor([
+                min(n_pos - 1, max(0, int(
+                    n_pos * edges[metas[i0 + b_][0]] * FPS / max(T, 1))))
+                for b_ in range(chunk.shape[0])
+            ], dtype=torch.long)
         evs = decode_windows(model, chunk, prefixes, device=device, greedy=greedy,
                              temperature=temperature, top_p=top_p, seed=seed + i0,
-                             n_cond=4 if style_val is not None else 3, min_gap=2)
+                             min_gap=2, position_limits=limits,
+                             gsum=g, pos_buckets=pb)
         for b_, w_ev in enumerate(evs):
-            jj, KK = metas[i0 + b_]
+            jj, KK, offsets = metas[i0 + b_]
             for g, cls in w_ev:
-                if g < KK * SLOTS:
-                    events.append((jj + g // SLOTS, g % SLOTS, cls))
+                if g < offsets[-1]:
+                    local_m = int(np.searchsorted(offsets[1:], g, side="right"))
+                    events.append((jj + local_m, int(g - offsets[local_m]), cls))
         if on_progress:
             on_progress(min(i0 + batch_windows, len(wins)), len(wins))
 
     events.sort(key=lambda e: (e[0], e[1]))
     def _t(me, sl):
-        return float(edges[me] + sl / SLOTS * (edges[me + 1] - edges[me])) \
+        slots_in_measure = int(measure_slots[me])
+        return float(edges[me] + sl / slots_in_measure * (edges[me + 1] - edges[me])) \
             if me < len(edges) - 1 else float(edges[-1])
 
     hits, spans, hits_slots, spans_slots = [], [], [], []
@@ -542,15 +589,11 @@ def generate_song_slot(model, mel, grid, course, level=None, density_bucket=None
     for me, sl, cls in events:
         t = _t(me, sl)
         if cls in HIT_CLASSES:
+            # A hit is never a valid implicit span terminator.  Keeping the
+            # authored hit and dropping the malformed, unclosed span avoids
+            # inventing an end position that the decoder did not emit.
             if open_span is not None:
-                if t - open_span[2] > SPAN_MAX.get(open_span[3], 6.5):
-                    t1s = open_span[2] + SPAN_MAX.get(open_span[3], 6.5)
-                    spans.append({"t0": round(open_span[2], 4), "t1": round(t1s, 4),
-                                  "type": open_span[3]})
-                    spans_slots.append((open_span[0], open_span[1], me, sl, open_span[3]))
-                    open_span = None
-                else:
-                    continue
+                open_span = None
             if (me, sl) == last_key:
                 continue
             hits.append({"t": round(t, 4), "type": cls})
@@ -560,13 +603,21 @@ def generate_song_slot(model, mel, grid, course, level=None, density_bucket=None
             if open_span is None:
                 open_span = (me, sl, t, cls)
         elif cls == "end":
-            if open_span is not None and t - open_span[2] > 0.05:
-                t1s = min(t, open_span[2] + SPAN_MAX.get(open_span[3], 6.5))
-                spans.append({"t0": round(open_span[2], 4), "t1": round(t1s, 4),
-                              "type": open_span[3]})
-                spans_slots.append((open_span[0], open_span[1], me, sl, open_span[3]))
+            if open_span is not None:
+                if t - open_span[2] > 0.05:
+                    # The explicit END slot is authoritative.  Derive both
+                    # preview seconds and TJA coordinates from that same slot;
+                    # duration caps must not move either representation.
+                    spans.append({"t0": round(open_span[2], 4), "t1": round(t, 4),
+                                  "type": open_span[3]})
+                    spans_slots.append(
+                        (open_span[0], open_span[1], me, sl, open_span[3]))
                 open_span = None
     n_meas = (metas[-1][0] + metas[-1][1]) if metas else 0
     return {"hits": hits, "spans": spans, "hits_slots": hits_slots,
             "spans_slots": spans_slots, "n_measures": n_meas,
+            "measure_slots": measure_slots[:n_meas].tolist(),
+            "measure_num": measure_num[:n_meas].tolist(),
+            "measure_den": measure_den[:n_meas].tolist(),
+            "slots_per_4_4": SLOTS,
             "course": course, "level": level, "density_bucket": density_bucket}

softchart/grid.py

@@ -59,7 +59,9 @@ def beat_activations(model, mel, device="cuda", hop=WINDOW // 2):
         if w.shape[0] < model.in_ch:  # dual/slot models: blank grid channels
             w = torch.cat([w, torch.full((model.in_ch - w.shape[0], WINDOW), -1.0)])
         with _autocast(device):
-            mem = model.encode(w[None].to(device))
+            # Beat/downbeat estimation must remain local and phase-free even
+            # when the chart decoder also owns a whole-song section encoder.
+            mem = model.encode_local(w[None].to(device))
             pr = torch.sigmoid(model.beat(mem[:, -L:]).float())[0].cpu().numpy().T
         if acc is None:
             factor = WINDOW // pr.shape[1]

softchart/hf.py

@@ -1,22 +1,17 @@
 """Hugging Face Hub-compatible wrappers (safetensors + from_pretrained/push_to_hub).
 
-Three model types, all <=16M params, MIT-licensed:
-  SoftChartGenerator  audio log-mel  -> chart event tokens (the main model; pr12)
-  SoftChartBeat       audio log-mel  -> beat/downbeat activations (barline anchor)
-  SoftChartPlanner    block features -> per-block plan tokens (auto-planning)
+Space runtime wrapper for the unified V1.8 generator.
 
 Usage:
     from softchart.hf import SoftChartGenerator
     gen = SoftChartGenerator.from_pretrained("JacobLinCool/softchart-generator")
 """
 
-import torch
 import torch.nn as nn
 from huggingface_hub import PyTorchModelHubMixin
 
-from .model import ChartModel, sinusoidal
-
-_CARD = "See https://github.com/JacobLinCool/SoftChart — MIT licensed."
+from .model import ChartModel
+from .vocab import VOCAB
 
 
 class SoftChartGenerator(
@@ -29,27 +24,21 @@ class SoftChartGenerator(
     """Encoder-decoder chart generator. Config is stored as config.json and the
     full architecture (condition tokens, heads) is reconstructed on load."""
 
-    def __init__(self, d_model=256, enc_layers=4, dec_layers=4, ffn=1024,
-                 vocab_size=1802, aux=True, global_ctx=False, func_time=False,
+    def __init__(self, d_model=256, nhead=8, enc_layers=4, dec_layers=4, ffn=1024,
+                 dropout=0.1,
+                 vocab_size=VOCAB.size, aux=True, global_ctx=False, func_time=False,
                  ptr=False, beat=False, in_ch=None, emb_factor=None,
-                 capabilities=None, nhead=8, clean_phase=False,
-                 enc_share=None, dec_share=None, adapter_rank=0,
-                 unique_layernorm=False, adapter_rank_ffn=0, depth_emb=False,
-                 unshare_last_dec=False):
+                 hierarchical_ctx=False, section_layers=2, section_ffn=None,
+                 clean_phase=False,
+                 capabilities=None):
         super().__init__()
-        # nhead and the v1.6/v1.7 sharing flags (enc_share/dec_share/adapter_rank/
-        # unique_layernorm/adapter_rank_ffn/depth_emb/unshare_last_dec) and
-        # clean_phase must be persisted in config.json: from_pretrained rebuilds
-        # the architecture from these kwargs alone, before any weights are loaded.
-        # Defaults reproduce the pre-v1.6 unshared architecture for older repos.
         self.net = ChartModel(
-            d_model=d_model, nhead=nhead, enc_layers=enc_layers, dec_layers=dec_layers,
-            ffn=ffn, vocab_size=vocab_size, aux=aux, global_ctx=global_ctx,
-            func_time=func_time, ptr=ptr, beat=beat, in_ch=in_ch, emb_factor=emb_factor,
-            clean_phase=clean_phase, enc_share=enc_share, dec_share=dec_share,
-            adapter_rank=adapter_rank, unique_layernorm=unique_layernorm,
-            adapter_rank_ffn=adapter_rank_ffn, depth_emb=depth_emb,
-            unshare_last_dec=unshare_last_dec,
+            d_model=d_model, nhead=nhead, enc_layers=enc_layers,
+            dec_layers=dec_layers, ffn=ffn, dropout=dropout,
+            vocab_size=vocab_size, aux=aux, global_ctx=global_ctx, func_time=func_time,
+            ptr=ptr, beat=beat, in_ch=in_ch, emb_factor=emb_factor,
+            hierarchical_ctx=hierarchical_ctx, section_layers=section_layers,
+            section_ffn=section_ffn, clean_phase=clean_phase,
         )
         # capabilities: which condition axes this checkpoint was trained with,
         # so inference knows which prefix tokens to emit
@@ -76,29 +65,3 @@ class SoftChartGenerator(
     @property
     def tok_emb(self):
         return self.net.tok_emb
-
-
-class SoftChartPlanner(
-    nn.Module,
-    PyTorchModelHubMixin,
-    repo_url="https://github.com/JacobLinCool/SoftChart",
-    license="mit",
-    tags=["taiko", "rhythm-game", "planning", "music"],
-):
-    """Song-level plan generator: block audio summaries -> (density, flag) tokens."""
-
-    def __init__(self, feat_dim=261, d=192, layers=4, n_dens=8, n_flag=3):
-        super().__init__()
-        self.inp = nn.Linear(feat_dim, d)
-        self.course = nn.Embedding(5, d)
-        self.register_buffer("pos", sinusoidal(128, d), persistent=False)
-        enc = nn.TransformerEncoderLayer(d, 6, d * 4, 0.1, activation="gelu",
-                                         batch_first=True, norm_first=True)
-        self.enc = nn.TransformerEncoder(enc, layers, nn.LayerNorm(d))
-        self.h_dens = nn.Linear(d, n_dens)
-        self.h_flag = nn.Linear(d, n_flag)
-
-    def forward(self, x, cid):
-        h = self.inp(x) + self.course(cid)[:, None] + self.pos[: x.shape[1]]
-        h = self.enc(h)
-        return self.h_dens(h), self.h_flag(h)

softchart/meter.py

@@ -0,0 +1,125 @@
+"""Exact rational-meter helpers for the 96-slot-per-4/4 TJA lattice."""
+
+from fractions import Fraction
+from math import gcd
+from numbers import Real
+
+import numpy as np
+
+from .vocab import SLOTS, WINDOW
+
+
+def _expand(value, n_measures, name, dtype):
+    array = np.asarray(value)
+    if array.ndim == 0:
+        array = np.full(n_measures, array.item())
+    else:
+        array = array.reshape(-1)
+        if len(array) != n_measures:
+            raise ValueError(
+                f"{name} must contain one value per measure "
+                f"({n_measures}), got {len(array)}"
+            )
+    if np.issubdtype(np.dtype(dtype), np.integer):
+        info = np.iinfo(dtype)
+        values = []
+        for raw in array.flat:
+            if isinstance(raw, (bool, np.bool_)):
+                raise ValueError(f"{name} must contain finite integers")
+            if isinstance(raw, (int, np.integer)):
+                integer = int(raw)
+            elif isinstance(raw, (float, np.floating)):
+                number = float(raw)
+                if not np.isfinite(number) or not number.is_integer():
+                    raise ValueError(f"{name} must contain finite integers")
+                integer = int(number)
+            else:
+                raise ValueError(f"{name} must contain finite integers")
+            if not info.min <= integer <= info.max:
+                raise ValueError(f"{name} is outside the {np.dtype(dtype)} range")
+            values.append(integer)
+        return np.asarray(values, dtype=dtype).reshape(array.shape)
+    values = []
+    for raw in array.flat:
+        if isinstance(raw, (bool, np.bool_)) or not isinstance(raw, Real):
+            raise ValueError(f"{name} must contain numeric values")
+        values.append(float(raw))
+    return np.asarray(values, dtype=dtype).reshape(array.shape)
+
+
+def rational_meter_arrays(measure_num, measure_den, n_measures=None):
+    """Validate TJA ``#MEASURE`` fractions and return lattice properties.
+
+    Returns ``(slots, numerator, denominator, quarter_beats)``. A meter is
+    representable exactly iff ``96 * numerator / denominator`` is an integer;
+    no rounding or coercion is permitted.
+    """
+    num = np.asarray(measure_num)
+    den = np.asarray(measure_den)
+    if n_measures is None:
+        if num.ndim == 0 and den.ndim == 0:
+            n_measures = 1
+        else:
+            n_measures = max(num.size, den.size)
+    num = _expand(num, n_measures, "measure_num", np.int64)
+    den = _expand(den, n_measures, "measure_den", np.int64)
+    if (num <= 0).any() or (den <= 0).any():
+        raise ValueError("measure numerator and denominator must be positive")
+
+    slot_values = []
+    for numerator, denominator in zip(num.tolist(), den.tolist()):
+        common = gcd(int(numerator), int(denominator))
+        reduced_num = int(numerator) // common
+        reduced_den = int(denominator) // common
+        if SLOTS % reduced_den:
+            raise ValueError(
+                "meter is not exactly representable on the 96-slot lattice")
+        slot_count = (SLOTS // reduced_den) * reduced_num
+        if not 1 <= slot_count <= WINDOW:
+            raise ValueError(
+                f"measure lattice length must be within 1..{WINDOW} slots")
+        slot_values.append(slot_count)
+    slots = np.asarray(slot_values, dtype=np.int64)
+    quarter_beats = 4.0 * num / den
+    return slots, num, den, quarter_beats.astype(np.float64)
+
+
+def resolve_meter_layout(grid, n_measures):
+    """Resolve exact per-measure layout from a generation grid.
+
+    Preferred input is ``measure_num``/``measure_den`` with optional
+    ``measure_slots`` for an integrity check. A uniform ``meter`` or a
+    ``beats_per_measure`` array remains a supported grid API: each value is a
+    quarter-note count and is converted to an exact reduced TJA fraction.
+    """
+    has_num = "measure_num" in grid
+    has_den = "measure_den" in grid
+    if has_num != has_den:
+        raise ValueError("measure_num and measure_den must be provided together")
+    if has_num:
+        num = _expand(grid["measure_num"], n_measures, "measure_num", np.int64)
+        den = _expand(grid["measure_den"], n_measures, "measure_den", np.int64)
+    else:
+        raw = grid.get("beats_per_measure", grid.get("meter"))
+        if raw is None:
+            raise ValueError(
+                "slot generation requires measure_num/measure_den or an "
+                "explicit uniform meter/beats_per_measure grid"
+            )
+        quarter = _expand(raw, n_measures, "beats_per_measure", np.float64)
+        if not np.isfinite(quarter).all() or (quarter <= 0).any():
+            raise ValueError("quarter beats per measure must be positive and finite")
+        fractions = [Fraction(float(q)).limit_denominator(768) for q in quarter]
+        if any(abs(float(f) - q) > 1e-9
+               for f, q in zip(fractions, quarter)):
+            raise ValueError("quarter-beat meter values must be exact rationals")
+        num = np.asarray([f.numerator for f in fractions], dtype=np.int64)
+        den = np.asarray([4 * f.denominator for f in fractions], dtype=np.int64)
+
+    slots, num, den, quarter = rational_meter_arrays(num, den, n_measures)
+    if "measure_slots" in grid:
+        supplied = _expand(
+            grid["measure_slots"], n_measures, "measure_slots", np.int64)
+        if not np.array_equal(supplied, slots):
+            raise ValueError("measure_slots disagrees with measure_num/measure_den")
+    return slots, num, den, quarter

softchart/model.py

@@ -33,6 +33,42 @@ class BeatHiRes(nn.Module):
         return self.out(h).transpose(1, 2)
 
 
+class HierarchicalSectionEncoder(nn.Module):
+    """Encode a fixed relative-time summary of the complete song.
+
+    The local encoder remains dedicated to the current 20-second window. This
+    separate branch models section-to-section structure, marks the section that
+    contains the current window, and exposes its states directly to decoder
+    cross-attention. It is trained from scratch with the chart objective; no
+    external audio representation is used.
+    """
+
+    def __init__(self, d, nhead, ffn, dropout, layers=2, sections=100):
+        super().__init__()
+        self.sections = sections
+        self.inp = nn.Linear(N_MELS, d)
+        self.pos = nn.Parameter(torch.randn(sections, d) * 0.02)
+        self.current = nn.Parameter(torch.randn(d) * 0.02)
+        layer = nn.TransformerEncoderLayer(
+            d, nhead, ffn, dropout, activation="gelu",
+            batch_first=True, norm_first=True,
+        )
+        self.encoder = nn.TransformerEncoder(
+            layer, layers, nn.LayerNorm(d), enable_nested_tensor=False)
+
+    def forward(self, summary, position):
+        if summary.ndim != 3 or summary.shape[1:] != (self.sections, N_MELS):
+            raise ValueError(
+                f"expected song summary (B,{self.sections},{N_MELS}), "
+                f"got {tuple(summary.shape)}"
+            )
+        position = position.to(summary.device).long().clamp(0, self.sections - 1)
+        h = self.inp(summary) + self.pos
+        marker = torch.zeros_like(h)
+        marker[torch.arange(h.shape[0], device=h.device), position] = self.current
+        return self.encoder(h + marker)
+
+
 def _parse_pattern(spec, n):
     """Layer-sharing schedule. spec is either an int (number of UNIQUE
     physical layers, cycled to length n) or a comma string like "E0,E1,E0,E1"
@@ -55,17 +91,18 @@ def _parse_pattern(spec, n):
 
 class Adapter(nn.Module):
     """Rank-r residual adapter applied after a (possibly reused) transformer
-    block: x + gate * up(gelu(down(x))). gate init 0 so the whole stack starts
-    identical to hard weight sharing; the adapter only differentiates a reused
-    physical layer as training moves the gate off zero."""
+    block: x + gate * up(gelu(down(x))). Both projections and the gate start on
+    an active gradient path. The up projection is initialized very small so the
+    initial adapter is a near-identity perturbation without the dead-gradient
+    state caused by zero-initializing both ``up`` and ``gate``."""
 
     def __init__(self, d, rank):
         super().__init__()
         self.down = nn.Linear(d, rank, bias=False)
         self.up = nn.Linear(rank, d, bias=False)
         nn.init.normal_(self.down.weight, std=0.02)
-        nn.init.zeros_(self.up.weight)
-        self.gate = nn.Parameter(torch.zeros(1))
+        nn.init.normal_(self.up.weight, std=1e-3)
+        self.gate = nn.Parameter(torch.ones(1))
 
     def forward(self, x):
         return x + self.gate * self.up(nn.functional.gelu(self.down(x)))
@@ -74,17 +111,17 @@ class Adapter(nn.Module):
 class SharedEncoder(nn.Module):
     """Encoder whose physical layers are reused according to a sharing pattern.
     Each POSITION in the depth keeps its own (a) rank-r adapter and (b) final
-    LayerNorm — so ΔW starts at 0 (hard sharing) but every position can drift.
+    LayerNorm, so every logical depth can specialize while reusing the block.
     With pattern=None this is exactly n unique layers + a norm, matching the
     stock nn.TransformerEncoder param count (adapters/pos-norms are opt-in).
 
-    v1.7 depth-diversity options (all default-off, init == v1.6 behaviour):
+    v1.7 depth-diversity options (all default-off):
       depth_emb: per-depth learned d-vector ADDED to the block input at each
         reuse (params = depth x d, zero-init so init == hard sharing).
       adapter_rank_ffn > 0: SPLIT adapters — instead of one post-block adapter,
         each depth owns a rank-`adapter_rank` adapter on the attention sublayer
         output and a rank-`adapter_rank_ffn` adapter on the FFN sublayer output
-        (patterning lives in the FFN). Gate init 0 keeps init == hard sharing.
+        (patterning lives in the FFN).
     """
 
     def __init__(self, d, nhead, ffn, dropout, pattern, adapter_rank=0,
@@ -123,7 +160,7 @@ class SharedEncoder(nn.Module):
             if self.split:
                 # norm_first decomposition of nn.TransformerEncoderLayer with
                 # per-depth adapters applied to each sublayer OUTPUT (Houlsby
-                # placement); at gate=0 this is bit-equal to the stock layer
+                # placement)
                 x = x + self.attn_adapters[k](
                     layer._sa_block(layer.norm1(x), None, src_key_padding_mask))
                 x = x + self.ffn_adapters[k](layer._ff_block(layer.norm2(x)))
@@ -204,6 +241,9 @@ class ChartModel(nn.Module):
         max_tgt=None,
         aux=False,
         global_ctx=False,
+        hierarchical_ctx=False,
+        section_layers=2,
+        section_ffn=None,
         func_time=False,
         ptr=False,
         beat=False,
@@ -224,6 +264,10 @@ class ChartModel(nn.Module):
         vocab_size = vocab_size or VOCAB.size
         max_tgt = max_tgt or MAX_TGT
         self.d_model = d_model
+        if global_ctx and hierarchical_ctx:
+            raise ValueError("global_ctx and hierarchical_ctx are mutually exclusive")
+        self.hierarchical_ctx = bool(hierarchical_ctx)
+        self._hierarchical_ctx = self.hierarchical_ctx
         # clean_phase (v1.6): the encoder never sees the 2 grid-phase channels
         # (in_ch stays N_MELS), and phase is injected into the DECODER memory
         # only, via a small projection. This removes the condition leak that
@@ -267,8 +311,9 @@ class ChartModel(nn.Module):
             enc_layer = nn.TransformerEncoderLayer(
                 d_model, nhead, ffn, dropout, activation="gelu",
                 batch_first=True, norm_first=True)
-            self.encoder = nn.TransformerEncoder(enc_layer, enc_layers,
-                                                 nn.LayerNorm(d_model))
+            self.encoder = nn.TransformerEncoder(
+                enc_layer, enc_layers, nn.LayerNorm(d_model),
+                enable_nested_tensor=False)
             dec_layer = nn.TransformerDecoderLayer(
                 d_model, nhead, ffn, dropout, activation="gelu",
                 batch_first=True, norm_first=True)
@@ -306,8 +351,17 @@ class ChartModel(nn.Module):
             self.out = nn.Linear(d_model, vocab_size, bias=False)
             self.out.weight = self.tok_emb.weight  # weight tying
         self.dropout = nn.Dropout(dropout)
-        # auxiliary per-position onset-heatmap head on the encoder (v2)
+        # Auxiliary local rhythmic skeleton. In the hierarchical model its
+        # predicted occupancy is also projected back into decoder memory, so
+        # the event realizer is explicitly conditioned on a coarse audio-led
+        # skeleton rather than learning placement and pattern in one flat pass.
+        if hierarchical_ctx and not aux:
+            raise ValueError("hierarchical_ctx requires the auxiliary skeleton head")
         self.aux = nn.Linear(d_model, 1) if aux else None
+        self.skeleton_proj = nn.Linear(1, d_model) if hierarchical_ctx else None
+        if self.skeleton_proj is not None:
+            nn.init.zeros_(self.skeleton_proj.weight)
+            nn.init.zeros_(self.skeleton_proj.bias)
         # pointer alignment head: each generated note must point to its audio
         # frame (differentiable provenance; explainability that trains alignment)
         self.ptr = nn.Linear(d_model, d_model) if ptr else None
@@ -323,10 +377,18 @@ class ChartModel(nn.Module):
         if global_ctx:
             self.gsum_proj = nn.Linear(N_MELS, d_model)
             self.seg_emb = nn.Parameter(torch.zeros(2, d_model))
-            self.pos_emb = nn.Embedding(16, d_model)  # window position bucket
+            self.pos_emb = nn.Embedding(100, d_model)  # relative song position
             self.gpos = nn.Parameter(torch.randn(128, d_model) * 0.02)
         else:
             self.gsum_proj = None
+        if hierarchical_ctx:
+            self.section_encoder = HierarchicalSectionEncoder(
+                d_model, nhead, section_ffn or (2 * d_model), dropout,
+                layers=section_layers,
+            )
+            self.hier_seg_emb = nn.Parameter(torch.zeros(2, d_model))
+        else:
+            self.section_encoder = None
         # functional time embeddings: the 1728 TIME tokens share a sinusoidal
         # basis + small projection instead of free embeddings (fewer params,
         # neighbouring times get similar representations)
@@ -339,15 +401,31 @@ class ChartModel(nn.Module):
         else:
             self.time_proj = None
 
+    def _local_input(self, mel):
+        audio = mel[:, :N_MELS] if self.clean_phase else mel
+        h = self.frontend(audio).transpose(1, 2)  # (B, T/4, d)
+        return h + self.enc_pos[: h.shape[1]]
+
+    def encode_local(self, mel):
+        """Encode only the current window for beat/downbeat estimation."""
+        h = self._local_input(mel)
+        if self.section_encoder is not None:
+            h = h + self.hier_seg_emb[0]
+        return self.encoder(self.dropout(h))
+
     def encode(self, mel, gsum=None, pos_bucket=None):
         # mel: (B, C, T). clean_phase: C may be N_MELS (audio only) or N_MELS+2
         # (audio + 2 grid-phase channels); only the audio channels reach the
         # encoder, so the returned memory is CLEAN (phase-free) and safe for the
-        # beat head. gsum: (B, G, n_mels) whole-song summary chunks;
-        # pos_bucket: (B,) window-position bucket in [0, 16).
-        audio = mel[:, :N_MELS] if self.clean_phase else mel
-        h = self.frontend(audio).transpose(1, 2)  # (B, T/4, d)
-        h = h + self.enc_pos[: h.shape[1]]
+        # beat head. gsum: (B, 100, n_mels) complete-song section summaries;
+        # pos_bucket: (B,) current relative song section in [0, 100).
+        h = self._local_input(mel)
+        if self.section_encoder is not None:
+            if gsum is None or pos_bucket is None:
+                raise ValueError("hierarchical_ctx requires whole-song summary and position")
+            local = self.encoder(self.dropout(h + self.hier_seg_emb[0]))
+            sections = self.section_encoder(gsum, pos_bucket) + self.hier_seg_emb[1]
+            return torch.cat([sections, local], dim=1)
         if self.gsum_proj is not None and gsum is not None:
             g = self.gsum_proj(gsum) + self.seg_emb[1] + self.gpos[: gsum.shape[1]]
             h = h + self.seg_emb[0]
@@ -360,17 +438,25 @@ class ChartModel(nn.Module):
         Adds a projection of the grid-phase channels to the audio-frame slice
         of memory; the beat head keeps reading the clean `memory`. No-op unless
         clean_phase is on and phase channels are present. Time-mode windows
-        (raw phase = -1) get valid=0 and zeroed phase -> ~zero conditioning."""
-        nch = N_MELS
-        if not self.clean_phase or mel.shape[1] < nch + 2:
-            return memory
-        ph = mel[:, nch:nch + 2]  # (B, 2, T) in [0,1); -1 marks no-grid frames
-        valid = (ph[:, :1] >= 0).float()  # 1 where a grid exists, else 0
-        ph_in = torch.cat([ph.clamp(min=0.0), valid], dim=1)  # (B, 3, T)
-        p = self.phase_proj(ph_in).transpose(1, 2)  # (B, T/4, d)
-        L = p.shape[1]
+        (raw phase = -1) get valid=0 and exactly zero phase conditioning."""
         out = memory.clone()
-        out[:, -L:] = out[:, -L:] + p  # audio frames are the last L memory slots
+        L = WINDOW // 4
+        nch = N_MELS
+        if self.clean_phase and mel.shape[1] >= nch + 2:
+            ph = mel[:, nch:nch + 2]  # [0,1); -1 marks no-grid frames
+            valid = (ph[:, :1] >= 0).float()
+            ph_in = torch.cat([ph.clamp(min=0.0), valid], dim=1)
+            p = self.phase_proj(ph_in)
+            # Mask *after* both biased convolutions. Masking only their input
+            # leaves a learned bias residual in gridless time-mode windows.
+            valid_out = nn.functional.adaptive_max_pool1d(valid, p.shape[-1])
+            p = (p * valid_out).transpose(1, 2)
+            L = p.shape[1]
+            out[:, -L:] = out[:, -L:] + p
+        if self.skeleton_proj is not None:
+            clean_local = memory[:, -L:]
+            skeleton = torch.sigmoid(self.aux(clean_local))
+            out[:, -L:] = out[:, -L:] + self.skeleton_proj(skeleton)
         return out
 
     def emb_matrix(self):
@@ -388,7 +474,7 @@ class ChartModel(nn.Module):
         L = tgt_in.shape[1]
         E = self.emb_matrix()
         h = nn.functional.embedding(tgt_in, E) * math.sqrt(self.d_model) + self.dec_pos[:L]
-        mask = nn.Transformer.generate_square_subsequent_mask(L, device=tgt_in.device)
+        mask = torch.ones((L, L), dtype=torch.bool, device=tgt_in.device).triu(1)
         pad_mask = tgt_in == VOCAB.pad
         h = self.decoder(
             self.dropout(h),
@@ -416,7 +502,7 @@ class ChartModel(nn.Module):
         E = self.emb_matrix()
         import math as _m
         h = nn.functional.embedding(dec_in, E) * _m.sqrt(self.d_model) + self.dec_pos[:L]
-        dm = nn.Transformer.generate_square_subsequent_mask(L, device=tgt.device)
+        dm = torch.ones((L, L), dtype=torch.bool, device=tgt.device).triu(1)
         h = self.decoder(self.dropout(h), dec_mem, tgt_mask=dm,
                          tgt_key_padding_mask=dec_in == VOCAB.pad, tgt_is_causal=True)
         logits = h @ E.T

softchart/preprocess.py

@@ -1,156 +0,0 @@
-"""Convert the HF dataset into per-song cache files: log-mel (.npy) + notes (.npz).
-
-Usage (on the training machine):
-    python -m softchart.preprocess --out /workspace/softchart/cache [--split train]
-"""
-
-import argparse
-import json
-import os
-from collections import Counter
-
-import numpy as np
-
-from .vocab import HOP, N_FFT, N_MELS, NOTE_CLASSES, NOTE_TYPE_MAP, SR, COURSES
-
-
-def to_mono_22k(audio):
-    """Handle both datasets<=3 dict audio and datasets>=4 torchcodec AudioDecoder."""
-    import torch
-
-    if isinstance(audio, dict):
-        arr = np.asarray(audio["array"], dtype=np.float32)
-        if arr.ndim == 2:
-            arr = arr.mean(axis=0)
-        sr = audio["sampling_rate"]
-    else:  # torchcodec AudioDecoder
-        samples = audio.get_all_samples()
-        arr = samples.data.to(torch.float32).mean(dim=0).numpy()
-        sr = samples.sample_rate
-    if sr != SR:
-        import librosa
-
-        arr = librosa.resample(arr, orig_sr=sr, target_sr=SR, res_type="soxr_hq")
-    return torch.from_numpy(np.ascontiguousarray(arr))[None]  # (1, T)
-
-
-_mel_fb = None
-_window = None
-
-
-def logmel(wav):
-    """(1, T) float32 -> (n_mels, frames) float16 log-mel (torch.stft + librosa fb)."""
-    global _mel_fb, _window
-    import torch
-
-    if _mel_fb is None:
-        import librosa
-
-        fb = librosa.filters.mel(sr=SR, n_fft=N_FFT, n_mels=N_MELS, fmin=20.0, fmax=SR / 2)
-        _mel_fb = torch.from_numpy(fb)  # (n_mels, n_fft//2+1)
-        _window = torch.hann_window(N_FFT)
-    with torch.no_grad():
-        spec = torch.stft(
-            wav, N_FFT, hop_length=HOP, window=_window, center=True, return_complex=True
-        )[0]
-        power = spec.abs().pow(2)  # (freq, frames)
-        m = torch.log(_mel_fb @ power + 1e-5)
-    return m.numpy().astype(np.float16)
-
-
-def extract_notes(course_struct):
-    """course struct -> (times, classes, level) or None."""
-    if course_struct is None or not course_struct.get("segments"):
-        return None
-    times, classes, bpms = [], [], []
-    unknown = Counter()
-    for seg in course_struct["segments"]:
-        for n in seg.get("notes") or []:
-            nt = n["note_type"]
-            canon = NOTE_TYPE_MAP.get(nt) or NOTE_TYPE_MAP.get(nt.lower())
-            if canon is None:
-                unknown[nt] += 1
-                continue
-            times.append(n["timestamp"])
-            classes.append(NOTE_CLASSES.index(canon))
-            if n.get("bpm"):
-                bpms.append(n["bpm"])
-    if not times:
-        return None
-    order = np.argsort(times, kind="stable")
-    return (
-        np.asarray(times, np.float64)[order],
-        np.asarray(classes, np.int8)[order],
-        int(course_struct.get("level") or 0),
-        float(np.median(bpms)) if bpms else 0.0,
-        unknown,
-    )
-
-
-def main():
-    ap = argparse.ArgumentParser()
-    ap.add_argument("--out", default="cache")
-    ap.add_argument("--splits", nargs="+", default=["train", "test"])
-    args = ap.parse_args()
-
-    from datasets import Audio, load_dataset
-
-    ds = load_dataset("JacobLinCool/taiko-1000-parsed")
-    ds = ds.cast_column("audio", Audio(sampling_rate=SR))
-    os.makedirs(args.out, exist_ok=True)
-    index = []
-    unknown_total = Counter()
-
-    for split in args.splits:
-        d = ds[split]
-        for i in range(len(d)):
-            row = d[i]
-            sid = f"{split}_{i:05d}"
-            mel_path = os.path.join(args.out, f"{sid}.mel.npy")
-            notes_path = os.path.join(args.out, f"{sid}.notes.npz")
-            entry = {
-                "id": sid,
-                "split": split,
-                "title": (row.get("metadata") or {}).get("TITLE", ""),
-                "genre": (row.get("metadata") or {}).get("GENRE", ""),
-                "courses": {},
-            }
-            arrays = {}
-            for c in COURSES:
-                ex = extract_notes(row.get(c))
-                if ex is None:
-                    continue
-                times, classes, level, bpm, unknown = ex
-                unknown_total.update(unknown)
-                arrays[f"{c}_t"] = times
-                arrays[f"{c}_c"] = classes
-                entry["courses"][c] = {
-                    "level": level,
-                    "n_notes": int(len(times)),
-                    "bpm": bpm,
-                }
-            if not arrays:
-                print(f"skip {sid}: no charts")
-                continue
-            if not (os.path.exists(mel_path) and os.path.exists(notes_path)):
-                try:
-                    mel = logmel(to_mono_22k(row["audio"]))
-                except Exception as e:
-                    raise RuntimeError(f"{sid}: target-rate audio decode failed") from e
-                np.save(mel_path, mel)
-                np.savez(notes_path, **arrays)
-                entry["n_frames"] = int(mel.shape[1])
-            else:
-                entry["n_frames"] = int(np.load(mel_path, mmap_mode="r").shape[1])
-            index.append(entry)
-            if i % 50 == 0:
-                print(f"{split} {i}/{len(d)}", flush=True)
-
-    with open(os.path.join(args.out, "index.json"), "w") as f:
-        json.dump(index, f)
-    print("unknown note types:", dict(unknown_total))
-    print(f"done: {len(index)} songs")
-
-
-if __name__ == "__main__":
-    main()

softchart/proxies.py

@@ -1,238 +0,0 @@
-"""Automated proxies for playability, groove, and pattern structure.
-
-Every proxy is computable without human input, from (events, bpm[, mel]).
-Together with rhythm.py metrics they form the scorecard that closes the
-evaluate -> diagnose -> improve loop.
-
-Proxy design notes:
-- playability = physical feasibility on the drum (min gaps, stream length,
-  big-note crowding) + learnable structure (motif reuse, compressibility)
-- groove = alignment with the metrical grid (strong-beat rate, accent placement)
-  + audio grounding (onset-envelope energy at note times)
-Reference values come from ground-truth charts of the same course (see scorecard).
-"""
-
-import zlib
-
-import numpy as np
-
-HIT_CLASSES = ("don", "ka", "don_big", "ka_big")
-BIG_CLASSES = ("don_big", "ka_big")
-
-
-def _hits(events):
-    return sorted((t, c) for t, c in events if c in HIT_CLASSES)
-
-
-def min_gap_violation_rate(events, min_gap=0.045):
-    """Fraction of consecutive hits closer than a physically playable gap."""
-    h = _hits(events)
-    if len(h) < 2:
-        return 0.0
-    ioi = np.diff([t for t, _ in h])
-    return float(np.mean(ioi < min_gap))
-
-
-def stream_p99_nps(events, win=2.0):
-    """99th percentile of local density (notes/sec in rolling windows) — stamina proxy."""
-    t = np.array([x for x, _ in _hits(events)])
-    if len(t) < 4:
-        return 0.0
-    counts = [np.sum((t >= x) & (t < x + win)) for x in np.arange(t[0], t[-1], 0.5)]
-    return float(np.percentile(counts, 99) / win) if counts else 0.0
-
-
-def big_crowding_rate(events, bpm):
-    """Fraction of big notes followed by another hit within 0.45 beat.
-
-    Big notes are played with both hands; crowding them against the next note
-    is physically awkward and rarely done in official charts.
-    """
-    h = _hits(events)
-    bigs = [(i, t) for i, (t, c) in enumerate(h) if c in BIG_CLASSES]
-    if not bigs or not bpm or bpm <= 0:
-        return None
-    thresh = 0.45 * 60.0 / bpm
-    crowded = sum(1 for i, t in bigs if i + 1 < len(h) and h[i + 1][0] - t < thresh)
-    return crowded / len(bigs)
-
-
-def big_note_rate(events):
-    h = _hits(events)
-    if not h:
-        return 0.0
-    return sum(1 for _, c in h if c in BIG_CLASSES) / len(h)
-
-
-def _symbol_stream(events, bpm):
-    """Quantized (ioi-class, note-type) symbol stream for structure metrics."""
-    h = _hits(events)
-    if len(h) < 3 or not bpm or bpm <= 0:
-        return []
-    beat = 60.0 / bpm
-    syms = []
-    for i in range(1, len(h)):
-        ioi = h[i][0] - h[i - 1][0]
-        frac = ioi / beat
-        # classify IOI into musical classes
-        classes = [0.25, 1 / 3, 0.5, 2 / 3, 0.75, 1.0, 1.5, 2.0]
-        j = int(np.argmin([abs(frac - c) / c for c in classes]))
-        ioi_cls = j if abs(frac - classes[j]) / classes[j] < 0.25 else len(classes)
-        syms.append((ioi_cls, h[i][1]))
-    return syms
-
-
-def motif_reuse(events, bpm, n=4):
-    """Fraction of n-grams (rhythm+type) that occur at least twice in the chart.
-
-    Human charts repeat and transform motifs; random note salads do not.
-    """
-    syms = _symbol_stream(events, bpm)
-    if len(syms) < n + 2:
-        return None
-    grams = [tuple(syms[i : i + n]) for i in range(len(syms) - n + 1)]
-    from collections import Counter
-
-    counts = Counter(grams)
-    return float(sum(1 for g in grams if counts[g] >= 2) / len(grams))
-
-
-def compression_ratio(events, bpm):
-    """zlib-compressed size / raw size of the symbol stream. Lower = more structured."""
-    syms = _symbol_stream(events, bpm)
-    if len(syms) < 16:
-        return None
-    raw = bytes(b for s in syms for b in (s[0], HIT_CLASSES.index(s[1])))
-    return float(len(zlib.compress(raw, 9)) / len(raw))
-
-
-def strong_beat_rate(events, bpm, phase=None, tol=0.03):
-    """Fraction of hits on integer beats (downbeat proxy needs measures; beat is enough)."""
-    from .rhythm import estimate_phase
-
-    t = np.array([x for x, _ in _hits(events)])
-    if len(t) < 5 or not bpm or bpm <= 0:
-        return None
-    beat = 60.0 / bpm
-    if phase is None:
-        phase = estimate_phase(t, beat)
-    rel = (t - phase) / beat
-    return float(np.mean(np.abs(rel - np.round(rel)) * beat < tol))
-
-
-def accent_on_strong(events, bpm, phase=None, tol=0.03):
-    """Fraction of BIG notes that fall on integer beats — accent placement proxy."""
-    from .rhythm import estimate_phase
-
-    h = _hits(events)
-    tb = np.array([t for t, c in h if c in BIG_CLASSES])
-    if len(tb) < 3 or not bpm or bpm <= 0:
-        return None
-    beat = 60.0 / bpm
-    if phase is None:
-        phase = estimate_phase(np.array([t for t, _ in h]), beat)
-    rel = (tb - phase) / beat
-    return float(np.mean(np.abs(rel - np.round(rel)) * beat < tol))
-
-
-def audio_energy_alignment(events, mel, fps):
-    """Mean z-scored spectral-flux strength at hit times (GT-free audio grounding)."""
-    t = np.array([x for x, _ in _hits(events)])
-    if len(t) < 5:
-        return None
-    flux = np.maximum(0.0, np.diff(mel.astype(np.float32), axis=1)).sum(axis=0)
-    flux = (flux - flux.mean()) / (flux.std() + 1e-9)
-    fr = np.clip(np.round(t * fps).astype(int), 1, len(flux) - 1)
-    return float(np.mean(np.maximum.reduce([flux[fr - 1], flux[fr], flux[np.minimum(fr + 1, len(flux) - 1)]])))
-
-
-def _pair_spans(events):
-    """Pair span-start/end events into (t0, t1) tuples (for GT event streams)."""
-    out = []
-    open_t = None
-    for t, c in sorted(events):
-        if c in ("roll", "roll_big", "balloon") and open_t is None:
-            open_t = t
-        elif c == "end" and open_t is not None:
-            out.append((open_t, t))
-            open_t = None
-    return out
-
-
-def span_stats(events, spans=None):
-    """Span-length distribution + rate. Added after a visual audit found
-    generated balloons ~10s vs GT p50 ~1s (a metric blind spot)."""
-    if spans is None:
-        pairs = _pair_spans(events)
-    else:
-        pairs = [(s["t0"], s["t1"]) if isinstance(s, dict) else tuple(s) for s in spans]
-    h = _hits(events)
-    dur_min = max((h[-1][0] - h[0][0]) / 60.0, 1e-6) if len(h) > 1 else None
-    if not pairs or dur_min is None:
-        return {"span_p90_len": None, "span_per_min": 0.0 if dur_min else None}
-    lens = np.array([b - a for a, b in pairs])
-    return {"span_p90_len": float(np.percentile(lens, 90)),
-            "span_per_min": float(len(pairs) / dur_min)}
-
-
-def compute_proxies(events, bpm, mel=None, fps=None, spans=None):
-    """All proxies for one chart. Returns dict (None where not computable)."""
-    from .rhythm import grid_consistency, tuplet_evenness
-
-    t = [x for x, _ in _hits(events)]
-    out = {
-        "nps": len(t) / max(t[-1] - t[0], 1e-6) if len(t) > 1 else 0.0,
-        "min_gap_violation": min_gap_violation_rate(events),
-        "stream_p99_nps": stream_p99_nps(events),
-        "big_rate": big_note_rate(events),
-        "big_crowding": big_crowding_rate(events, bpm),
-        "motif_reuse": motif_reuse(events, bpm),
-        "compression": compression_ratio(events, bpm),
-        "strong_beat": strong_beat_rate(events, bpm),
-        "accent_on_strong": accent_on_strong(events, bpm),
-    }
-    g = grid_consistency(t, bpm)
-    if g:
-        out["offgrid_rate"] = 1.0 - g["on_grid_frac"]
-        out["grid_dev_ms"] = g["mean_dev_ms"]
-    tp = tuplet_evenness(t, bpm)
-    if tp:
-        out["tuplet_cv"] = tp["cv_mean"]
-    if mel is not None and fps:
-        out["energy_align"] = audio_energy_alignment(events, mel, fps)
-    out.update(span_stats(events, spans))
-    return out
-
-
-# direction: +1 higher is better, -1 lower is better, 0 match GT reference
-PROXY_DIRECTION = {
-    "nps": 0, "min_gap_violation": -1, "stream_p99_nps": 0, "big_rate": 0,
-    "big_crowding": -1, "motif_reuse": 0, "compression": 0, "strong_beat": 0,
-    "accent_on_strong": 0, "offgrid_rate": -1, "grid_dev_ms": -1,
-    "tuplet_cv": -1, "energy_align": 1, "span_p90_len": 0, "span_per_min": 0,
-}
-
-
-def score_against_reference(proxy_row, ref_stats):
-    """Normalized deviation of each proxy vs the GT reference distribution.
-
-    ref_stats: {proxy: {"p25": .., "p50": .., "p75": ..}} per course.
-    Returns {proxy: score} where score = robust z of |dev| (0 = at GT median;
-    for directional proxies, only penalize the bad direction).
-    """
-    scores = {}
-    for k, v in proxy_row.items():
-        if v is None or k not in ref_stats:
-            continue
-        r = ref_stats[k]
-        iqr = max(r["p75"] - r["p25"], 1e-6)
-        d = PROXY_DIRECTION.get(k, 0)
-        dev = (v - r["p50"]) / iqr
-        if d == -1:
-            dev = max(0.0, dev)  # only worse-than-GT (higher) penalized
-        elif d == 1:
-            dev = max(0.0, -dev)
-        else:
-            dev = abs(dev)
-        scores[k] = float(dev)
-    return scores

softchart/tja.py

@@ -1,114 +1,143 @@
-"""TJA emission for slot-mode charts: slot indices ARE the chart.
+"""Meter-aware TJA emission for slot-mode charts.
 
 No quantization happens here — generate_song_slot() already produced exact
-lattice positions (measure, slot in 0..95), so this writer just prints them.
-Preview rendering and the .tja are identical by construction. Piecewise
-grids additionally emit #BPMCHANGE lines between measures.
+lattice positions (96 slots per 4/4 measure), so this writer just prints them.
+Preview rendering and the .tja are identical by construction. Authored measure
+edges determine exact per-measure tempo; meter changes emit #MEASURE directives.
 """
 
 import numpy as np
 
+from .meter import resolve_meter_layout
 from .vocab import SLOTS
 
 CHAR = {"don": "1", "ka": "2", "don_big": "3", "ka_big": "4",
         "roll": "5", "roll_big": "6", "balloon": "7"}
 
 
-def grid_measure_starts(grid, n_measures):
-    """Return measure boundary times for a slot grid, extending if needed."""
-    db = np.asarray(grid.get("downbeats", []), dtype=float)
-    if len(db) >= n_measures + 1:
-        return db[: n_measures + 1]
-    if len(db) >= 2:
-        step = float(np.median(np.diff(db)))
-        start = float(db[0])
-    else:
-        step = 240.0 / float(grid["bpm"])
-        start = float(db[0]) if len(db) else 0.0
-    return start + np.arange(n_measures + 1, dtype=float) * step
+def _format_number(value):
+    """Emit enough digits for a float to round-trip through the TJA text."""
+    value = float(value)
+    if not np.isfinite(value):
+        raise ValueError("TJA numeric fields must be finite")
+    if value == 0.0:
+        value = 0.0  # avoid a cosmetic negative zero in OFFSET
+    return format(value, ".17g")
 
 
-def gogo_measure_mask(plan, measure_starts, n_measures):
-    """Map plan climax blocks (flag == 2) to TJA measures."""
-    mask = [False] * n_measures
-    if not plan:
-        return mask
-    starts = np.asarray(measure_starts, dtype=float)
-    if len(starts) < n_measures + 1:
-        return mask
-    for block in plan:
-        if len(block) < 4 or int(block[3]) != 2:
-            continue
-        a, b = float(block[0]), float(block[1])
-        if b <= a:
-            continue
-        for m in range(n_measures):
-            if starts[m] < b and starts[m + 1] > a:
-                mask[m] = True
-    return mask
-
+def write_tja_slots(gen, grid, title, course, level, wave, out_path=None,
+                    balloon_count=10):
+    """Serialize a slot generation on an explicit trusted meter timeline.
 
-def append_measure_with_gogo(lines, measure_line, measure_idx, gogo_mask, in_gogo):
-    """Append a measure line, opening/closing #GOGO commands at boundaries."""
-    want_gogo = bool(gogo_mask[measure_idx]) if measure_idx < len(gogo_mask) else False
-    if in_gogo and not want_gogo:
-        lines.append("#GOGOEND")
-        in_gogo = False
-    if want_gogo and not in_gogo:
-        lines.append("#GOGOSTART")
-        in_gogo = True
-    lines.append(measure_line)
-    return in_gogo
+    ``grid`` must contain exact rational meter metadata and ``measure_edges``
+    with one trusted terminal edge. Returns the TJA text and optionally writes
+    it to ``out_path``.
+    """
+    if int(gen.get("slots_per_4_4", SLOTS)) != SLOTS:
+        raise ValueError(f"expected {SLOTS} slots per 4/4 measure")
+    n_meas = max(int(gen.get("n_measures", 0)),
+                 (max((m for m, _, _ in gen.get("hits_slots", [])), default=-1) + 1),
+                 (max((m1 for _, _, m1, _, _ in gen.get("spans_slots", [])),
+                      default=-1) + 1), 1)
+    fields = ("measure_slots", "measure_num", "measure_den")
+    gen_has_layout = any(name in gen for name in fields)
+    grid_has_layout = any(name in grid for name in fields) \
+        or "meter" in grid or "beats_per_measure" in grid
+    gen_layout = resolve_meter_layout(gen, n_meas) if gen_has_layout else None
+    grid_layout = resolve_meter_layout(grid, n_meas) if grid_has_layout else None
+    if gen_layout is not None and grid_layout is not None:
+        gen_slots, gen_num, gen_den, _ = gen_layout
+        grid_slots, grid_num, grid_den, _ = grid_layout
+        same_fraction = all(
+            int(gn) * int(gd) == int(gdn) * int(gnd)
+            for gn, gnd, gdn, gd in zip(
+                gen_num, gen_den, grid_num, grid_den)
+        )
+        if not np.array_equal(gen_slots, grid_slots) or not same_fraction:
+            raise ValueError("generated meter layout disagrees with the timing grid")
+    layout = grid_layout if grid_layout is not None else gen_layout
+    if layout is None:
+        raise ValueError(
+            "TJA slot export requires exact meter metadata; refusing to assume 4/4")
+    slots_per_measure, measure_num, measure_den, quarter_beats = layout
+    offsets = np.concatenate([[0], np.cumsum(slots_per_measure)])
 
+    def flat(measure, slot):
+        measure, slot = int(measure), int(slot)
+        if not 0 <= measure < n_meas:
+            raise ValueError(f"measure index {measure} is outside 0..{n_meas - 1}")
+        if not 0 <= slot < slots_per_measure[measure]:
+            raise ValueError(
+                f"slot {slot} is outside measure {measure}'s "
+                f"0..{slots_per_measure[measure] - 1} range"
+            )
+        return int(offsets[measure] + slot)
 
-def write_tja_slots(gen, grid, title, course, level, wave, out_path=None,
-                    balloon_count=10, plan=None):
-    """gen: generate_song_slot() result; grid: fit_grid() result (or a dict
-    with downbeats+bpm). Returns the TJA text (and writes it if out_path)."""
     slots = {}
     for me, sl, cls in gen["hits_slots"]:
-        slots.setdefault((me, sl), CHAR[cls])
+        position = flat(me, sl)
+        if position in slots:
+            raise ValueError(
+                f"slot collision at measure {int(me)}, slot {int(sl)}")
+        slots[position] = CHAR[cls]
     for m0, s0, m1, s1, typ in gen.get("spans_slots", []):
-        a = (m0, s0)
-        while a in slots:  # span start yields to hits: next free slot
-            a = (a[0] + (a[1] + 1) // SLOTS, (a[1] + 1) % SLOTS)
-        b = (m1, s1)
-        while b in slots or b <= a:
-            b = (b[0] + (b[1] + 1) // SLOTS, (b[1] + 1) % SLOTS)
+        a = flat(m0, s0)
+        b = flat(m1, s1)
+        if b <= a:
+            raise ValueError("span end must be strictly after its start")
+        if a in slots:
+            raise ValueError(
+                f"slot collision at span start measure {int(m0)}, slot {int(s0)}")
+        if b in slots:
+            raise ValueError(
+                f"slot collision at span end measure {int(m1)}, slot {int(s1)}")
         slots[a] = CHAR[typ]
         slots[b] = "8"
-    n_meas = max(gen.get("n_measures", 0),
-                 (max(m for m, _ in slots) + 1) if slots else 1)
-    # piecewise-tempo grids: emit #BPMCHANGE whenever the per-measure BPM
-    # (from consecutive fitted barlines) moves; TJA measure lines are unchanged
-    db = np.asarray(grid.get("downbeats", []), float) if grid.get("piecewise") else None
-    measure_starts = grid_measure_starts(grid, n_meas)
-    gogo_mask = gogo_measure_mask(plan, measure_starts, n_meas)
+
+    # Slot output is timing-exact only when every start and the terminal edge
+    # are explicit.  Never fall back to BPM/downbeat extrapolation here: that
+    # would make the serialized TJA disagree with the lattice used to decode.
+    if "measure_edges" not in grid:
+        raise ValueError(
+            "slot TJA export requires explicit measure_edges including the "
+            "trusted terminal edge"
+        )
+    tempo_edges = np.asarray(grid["measure_edges"], dtype=float)
+    if (tempo_edges.ndim != 1 or len(tempo_edges) != n_meas + 1
+            or not np.all(np.isfinite(tempo_edges))
+            or np.any(np.diff(tempo_edges) <= 0)):
+        raise ValueError(
+            "measure_edges must contain one finite, increasing edge per "
+            "measure plus the final edge")
     lines = []
-    cur_bpm = float(grid["bpm"])
-    in_gogo = False
+    base_bpm = float(grid["bpm"])
+    if not np.isfinite(base_bpm) or base_bpm <= 0:
+        raise ValueError("grid bpm must be positive and finite")
+    cur_bpm = base_bpm
+    cur_meter = (4, 4)
     for m in range(n_meas):
-        if in_gogo and not gogo_mask[m]:
-            lines.append("#GOGOEND")
-            in_gogo = False
-        if db is not None and m + 1 < len(db):
-            bpm_m = 240.0 / (db[m + 1] - db[m])
-            if abs(bpm_m - round(bpm_m)) < 0.05:
-                bpm_m = float(round(bpm_m))
-            if abs(bpm_m - cur_bpm) > 0.05:
-                lines.append(f"#BPMCHANGE {bpm_m:g}")
-                cur_bpm = bpm_m
-        in_gogo = append_measure_with_gogo(
-            lines, "".join(slots.get((m, k), "0") for k in range(SLOTS)) + ",",
-            m, gogo_mask, in_gogo)
-    if in_gogo:
-        lines.append("#GOGOEND")
+        duration = float(tempo_edges[m + 1] - tempo_edges[m])
+        if not np.isfinite(duration) or duration <= 0:
+            raise ValueError(f"measure {m} duration must be positive and finite")
+        bpm_m = 60.0 * float(quarter_beats[m]) / duration
+        # No integer snap and no perceptual threshold: even a small tempo
+        # delta accumulates into a real chart/audio timing error.
+        if bpm_m != cur_bpm:
+            lines.append(f"#BPMCHANGE {_format_number(bpm_m)}")
+            cur_bpm = bpm_m
+        meter = (int(measure_num[m]), int(measure_den[m]))
+        if meter != cur_meter:
+            lines.append(f"#MEASURE {meter[0]}/{meter[1]}")
+            cur_meter = meter
+        start = int(offsets[m])
+        lines.append("".join(slots.get(start + k, "0")
+                             for k in range(int(slots_per_measure[m]))) + ",")
     balloons = [balloon_count] * sum(1 for s in gen.get("spans_slots", [])
                                      if s[4] == "balloon")
-    offset = float(grid["downbeats"][0]) if len(grid.get("downbeats", [])) else 0.0
+    offset = float(tempo_edges[0])
     tja = "\n".join([
-        f"TITLE:{title} (SoftChart)", f"BPM:{grid['bpm']:g}", f"WAVE:{wave}",
-        f"OFFSET:{-offset:.3f}",
+        f"TITLE:{title} (SoftChart)", f"BPM:{_format_number(base_bpm)}",
+        f"WAVE:{wave}", f"OFFSET:{_format_number(-offset)}",
         f"COURSE:{'Oni' if course == 'oni' else str(course).capitalize()}",
         f"LEVEL:{level}",
         f"BALLOON:{','.join(map(str, balloons))}" if balloons else "BALLOON:",

softchart/train.py

@@ -1,433 +0,0 @@
-"""Training loop.
-
-    python -m softchart.train --cache /workspace/softchart/cache --out runs/base
-"""
-
-import argparse
-import json
-import math
-import os
-import time
-
-import torch
-import torch.nn.functional as F
-from torch.utils.data import DataLoader
-
-from .data import ChartWindowDataset, collate, load_split_ids
-from .model import ChartModel, count_params
-from .vocab import VOCAB
-
-
-# class-balanced weights for rare note types (~1/sqrt(train frequency), capped)
-TYPE_WEIGHTS = {"don": 1.0, "ka": 1.23, "don_big": 3.19, "ka_big": 5.29,
-                "roll": 7.18, "roll_big": 8.0, "balloon": 8.0, "end": 5.0}
-
-
-def build_class_weights(device, cap=8.0):
-    w = torch.ones(VOCAB.size, device=device)
-    for name, weight in TYPE_WEIGHTS.items():
-        w[VOCAB.note[name]] = min(weight, cap)
-    return w
-
-
-def loss_fn(logits, tgt, loss_mask, label_smoothing=0.1, class_weight=None):
-    # logits: (B, L-1, V) predicting tgt[:, 1:]
-    gold = tgt[:, 1:]
-    mask = loss_mask[:, 1:]
-    ls = F.cross_entropy(
-        logits.reshape(-1, logits.shape[-1]),
-        gold.reshape(-1),
-        reduction="none",
-        label_smoothing=label_smoothing,
-        weight=class_weight,
-    ).reshape(gold.shape)
-    return (ls * mask).sum() / mask.sum().clamp(min=1)
-
-
-def corrupt_events(tgt, loss_mask, rng_gen, hard=False):
-    """Synthetic negative for DPO. hard=True keeps corruptions NEAR the snap
-    tolerance (small jitter, few swaps) so negatives resemble the policy's own
-    mistakes rather than being trivially separable (fixes dpo_loss->0 collapse)."""
-    from .vocab import WINDOW
-
-    rej = tgt.clone()
-    ev = loss_mask.clone()
-    t0 = VOCAB.time0
-    is_time = (rej >= t0) & (rej < t0 + WINDOW) & ev
-    if hard:  # +-1..3 frames (~12-35ms, straddles the snap grid) on 60% of notes
-        jit = torch.randint(-3, 4, rej.shape, device=rej.device, generator=rng_gen)
-        jit = torch.where(jit == 0, torch.ones_like(jit), jit)
-        do_t = torch.rand(rej.shape, device=rej.device, generator=rng_gen) < 0.6
-        jit = torch.where(do_t, jit, torch.zeros_like(jit))
-        swap_p = 0.12
-    else:
-        jit = torch.randint(-5, 6, rej.shape, device=rej.device, generator=rng_gen)
-        jit = torch.where(jit.abs() < 2, jit.sign() * 2, jit)
-        swap_p = 0.25
-    rej = torch.where(is_time, (rej + jit).clamp(t0, t0 + WINDOW - 1), rej)
-    hit0 = VOCAB.note["don"]
-    is_hit = (rej >= hit0) & (rej < hit0 + 4) & ev
-    swap = torch.randint(0, 4, rej.shape, device=rej.device, generator=rng_gen) + hit0
-    do = torch.rand(rej.shape, device=rej.device, generator=rng_gen) < swap_p
-    rej = torch.where(is_hit & do, swap, rej)
-    return rej
-
-
-def seq_logprob(model, mel, tgt, mask):
-    import torch.nn.functional as F
-
-    logits = model(mel, tgt)
-    lp = F.log_softmax(logits.float(), dim=-1)
-    gold = tgt[:, 1:]
-    g = lp.gather(-1, gold.unsqueeze(-1)).squeeze(-1)
-    return (g * mask[:, 1:]).sum(1)
-
-
-@torch.no_grad()
-def validate(model, loader, device):
-    model.eval()
-    tot, n = 0.0, 0
-    for mel, tgt, mask, _aux, gsum, posb, _beat, _in in loader:
-        mel, tgt, mask = mel.to(device), tgt.to(device), mask.to(device)
-        gsum = gsum.to(device) if gsum is not None else None
-        with torch.autocast("cuda", dtype=torch.bfloat16):
-            logits = model(mel, tgt, gsum=gsum, pos_bucket=posb.to(device))
-            l = loss_fn(logits, tgt, mask, label_smoothing=0.0)
-        tot += l.item() * mel.shape[0]
-        n += mel.shape[0]
-    model.train()
-    return tot / max(n, 1)
-
-
-def main():
-    ap = argparse.ArgumentParser()
-    ap.add_argument("--cache", required=True)
-    ap.add_argument("--out", required=True)
-    ap.add_argument("--steps", type=int, default=60000)
-    ap.add_argument("--batch", type=int, default=32)
-    ap.add_argument("--lr", type=float, default=3e-4)
-    ap.add_argument("--warmup", type=int, default=1000)
-    ap.add_argument("--cond-drop", type=float, default=0.15)
-    ap.add_argument("--no-augment", action="store_true")
-    ap.add_argument("--balanced", action="store_true", help="class-balanced type loss")
-    ap.add_argument("--balance-cap", type=float, default=8.0,
-                    help="cap for class weights (v3: 2.5 after v2 over-corrected)")
-    ap.add_argument("--ctx", action="store_true", help="prev-window context tokens")
-    ap.add_argument("--aux", action="store_true", help="onset-heatmap auxiliary head")
-    ap.add_argument("--global-ctx", action="store_true",
-                    help="whole-song summary + window-position context (v4)")
-    ap.add_argument("--importance-sampling", action="store_true",
-                    help="bias window sampling toward informative regions (v4)")
-    ap.add_argument("--d-model", type=int, default=512)
-    ap.add_argument("--enc-layers", type=int, default=6)
-    ap.add_argument("--dec-layers", type=int, default=6)
-    ap.add_argument("--ffn", type=int, default=2048)
-    ap.add_argument("--func-time", action="store_true",
-                    help="functional (sinusoidal-basis) TIME token embeddings")
-    ap.add_argument("--emb-factor", type=int, default=0,
-                    help="ALBERT-style factorized embeddings (e.g. 64); 0 = off")
-    ap.add_argument("--distill", default=None,
-                    help="teacher checkpoint for knowledge distillation")
-    ap.add_argument("--use-aug", action="store_true",
-                    help="include waveform speed-augmented variants in training")
-    ap.add_argument("--init-from", default=None,
-                    help="initialize matching weights (decoder/tok_emb) from a checkpoint")
-    ap.add_argument("--kd-alpha", type=float, default=0.5, help="CE weight (rest = KL)")
-    ap.add_argument("--kd-tau", type=float, default=2.0, help="distillation temperature")
-    ap.add_argument("--tempo-aug", action="store_true",
-                    help="rhythm-preserving tempo/pitch augmentation in mel domain")
-    ap.add_argument("--sibling", action="store_true",
-                    help="easier-course skeleton hint in the prefix (easy subset-of hard)")
-    ap.add_argument("--style", action="store_true",
-                    help="charting-intent style token (cache/styles.json)")
-    ap.add_argument("--align", action="store_true",
-                    help="pointer alignment loss: notes must attend to their onset frame")
-    ap.add_argument("--beat-head", action="store_true",
-                    help="beat/downbeat auxiliary head (labels from TJA measures)")
-    ap.add_argument("--beat-weight", type=float, default=0.1,
-                    help="beat-head loss weight (raise for a dedicated beat model)")
-    ap.add_argument("--beat-hires", action="store_true",
-                    help="frame-resolution beat head (11.6 ms bins instead of 46 ms)")
-    ap.add_argument("--sync-token", action="store_true",
-                    help="LHL syncopation-band condition token")
-    ap.add_argument("--plan", action="store_true",
-                    help="song-level plan-block conditioning (cache/plans.json)")
-    ap.add_argument("--mask-infill", type=float, default=0.0,
-                    help="probability of type-mask (skeleton->color) curriculum per window")
-    ap.add_argument("--complexity", action="store_true",
-                    help="rhythmic-complexity band token (density-independent difficulty)")
-    ap.add_argument("--slot", action="store_true",
-                    help="slot-token mode: notes as exact TJA lattice indices "
-                         "(measure*96+slot), measure-aligned windows, grid-phase "
-                         "input channels — zero quantization error by construction")
-    ap.add_argument("--dual-slot-p", type=float, default=0.65,
-                    help="dual mode: probability of a slot window (rest = time)")
-    ap.add_argument("--dual", action="store_true",
-                    help="dual-mode: mixed slot/time windows with an explicit "
-                         "MODE token; one model serves both the grid-exact path "
-                         "and the gridless time path")
-    ap.add_argument("--dpo", default=None,
-                    help="reference checkpoint: switch to DPO finetuning with synthetic negatives")
-    ap.add_argument("--dpo-beta", type=float, default=0.1)
-    ap.add_argument("--dpo-hard", action="store_true",
-                    help="near-tolerance negatives (resemble policy errors)")
-    ap.add_argument("--val-every", type=int, default=2000)
-    ap.add_argument("--workers", type=int, default=12)
-    ap.add_argument("--resume", default=None)
-    ap.add_argument("--seed", type=int, default=1234)
-    ap.add_argument("--data-frac", type=float, default=1.0,
-                    help="fraction of train songs to use (data-scaling ablation); "
-                         "val/test splits are unaffected")
-    args = ap.parse_args()
-    torch.manual_seed(args.seed)
-
-    os.makedirs(args.out, exist_ok=True)
-    device = "cuda"
-    torch.backends.cuda.matmul.allow_tf32 = True
-    torch.backends.cudnn.allow_tf32 = True
-
-    train_ids, val_ids, _ = load_split_ids(args.cache, use_aug=args.use_aug,
-                                           frac=args.data_frac)
-    train_ds = ChartWindowDataset(
-        args.cache, train_ids, train=True, cond_drop=args.cond_drop,
-        spec_augment=not args.no_augment, use_ctx=args.ctx, aux=args.aux,
-        global_ctx=args.global_ctx, importance_sampling=args.importance_sampling,
-        tempo_aug=args.tempo_aug, sibling=args.sibling, style=args.style,
-        beat_head=args.beat_head, sync_token=args.sync_token,
-        plan=args.plan, mask_infill=args.mask_infill, complexity=args.complexity,
-        slot=args.slot, dual=args.dual, dual_slot_p=args.dual_slot_p,
-        beat_hires=args.beat_hires,
-    )
-    val_ds = ChartWindowDataset(args.cache, val_ids, train=False, windows_per_chart=1,
-                                use_ctx=args.ctx, aux=args.aux, global_ctx=args.global_ctx,
-                                sibling=args.sibling, style=args.style,
-                                beat_head=args.beat_head, sync_token=args.sync_token,
-                                plan=args.plan, complexity=args.complexity,
-                                slot=args.slot, dual=args.dual,
-                                beat_hires=args.beat_hires)
-    print(f"train charts {len(train_ds.items)}, val charts {len(val_ds.items)}")
-
-    train_loader = DataLoader(
-        train_ds, batch_size=args.batch, shuffle=True, collate_fn=collate,
-        num_workers=args.workers, pin_memory=True, drop_last=True, persistent_workers=True,
-    )
-    val_loader = DataLoader(
-        val_ds, batch_size=args.batch, shuffle=False, collate_fn=collate, num_workers=4,
-    )
-
-    from .vocab import N_MELS as _NM
-    model = ChartModel(
-        d_model=args.d_model, enc_layers=args.enc_layers, dec_layers=args.dec_layers,
-        ffn=args.ffn, aux=args.aux, global_ctx=args.global_ctx,
-        func_time=args.func_time, in_ch=(_NM + 2) if (args.slot or args.dual) else None,
-        emb_factor=args.emb_factor or None,
-    ).to(device)
-    if args.align:
-        model.enable_ptr()
-    if args.beat_head:
-        model.enable_beat(hires=args.beat_hires)
-    model.to(device)
-    print(f"params: {count_params(model)/1e6:.1f}M, vocab {VOCAB.size}")
-    if args.init_from and os.path.exists(args.init_from):
-        src = torch.load(args.init_from, map_location=device)["model"]
-        own = model.state_dict()
-        hit = {k: v for k, v in src.items()
-               if k in own and own[k].shape == v.shape
-               and k.startswith(("decoder", "tok_emb", "out", "dec_pos"))}
-        own.update(hit)
-        model.load_state_dict(own)
-        print(f"initialized {len(hit)} tensors from {args.init_from}")
-    class_weight = build_class_weights(device, args.balance_cap) if args.balanced else None
-
-    teacher = None
-    if args.distill:
-        from .generate import load_model
-
-        teacher = load_model(args.distill, device=device)
-        for p in teacher.parameters():
-            p.requires_grad_(False)
-        t_vocab = teacher.tok_emb.weight.shape[0]
-        print(f"distilling from {args.distill} (teacher vocab {t_vocab})")
-    dpo_ref = None
-    if args.dpo:
-        from .generate import load_model as _lm
-
-        dpo_ref = _lm(args.dpo, device=device)
-        for p_ in dpo_ref.parameters():
-            p_.requires_grad_(False)
-        # initialize the policy from the reference (vocab may have grown since:
-        # copy overlapping embedding rows, everything else matches the 12M arch)
-        ref_sd = dpo_ref.state_dict()
-        own = model.state_dict()
-        for k, v in ref_sd.items():
-            if k in own:
-                if own[k].shape == v.shape:
-                    own[k] = v
-                elif k == "tok_emb.weight":
-                    own[k][: v.shape[0]] = v
-        model.load_state_dict(own)
-        # disable dropout for DPO: logprob-margin noise from dropout on long
-        # sequences swamps the beta-scaled objective (verified: init loss 3.3
-        # instead of log 2)
-        import torch.nn as _nn
-
-        for m_ in model.modules():
-            if isinstance(m_, _nn.Dropout):
-                m_.p = 0.0
-        print(f"DPO: policy initialized from {args.dpo}, beta={args.dpo_beta}, dropout off")
-
-    opt = torch.optim.AdamW(model.parameters(), lr=args.lr, weight_decay=0.01, betas=(0.9, 0.98))
-
-    def lr_at(step):
-        if step < args.warmup:
-            return args.lr * step / args.warmup
-        p = (step - args.warmup) / max(1, args.steps - args.warmup)
-        return args.lr * (0.1 + 0.9 * 0.5 * (1 + math.cos(math.pi * p)))
-
-    step = 0
-    best_val = float("inf")
-    if args.resume and os.path.exists(args.resume):
-        ck = torch.load(args.resume, map_location=device)
-        model.load_state_dict(ck["model"])
-        opt.load_state_dict(ck["opt"])
-        step = ck["step"]
-        best_val = ck.get("best_val", best_val)
-        print(f"resumed from {args.resume} at step {step}")
-
-    log_path = os.path.join(args.out, "train_log.jsonl")
-    model.train()
-    t0 = time.time()
-    run_loss, run_n = 0.0, 0
-    done = False
-    while not done:
-        for mel, tgt, mask, aux_t, gsum, posb, beat_t, in_tgt in train_loader:
-            step += 1
-            if step > args.steps:
-                done = True
-                break
-            for g in opt.param_groups:
-                g["lr"] = lr_at(step)
-            mel, tgt, mask = mel.to(device, non_blocking=True), tgt.to(device), mask.to(device)
-            gsum = gsum.to(device) if gsum is not None else None
-            posb = posb.to(device)
-            in_tgt = in_tgt.to(device)
-            if dpo_ref is not None:
-                gen_rng = torch.Generator(device=device)
-                gen_rng.manual_seed(args.seed * 100003 + step)
-                rej = corrupt_events(tgt, mask, gen_rng, hard=args.dpo_hard)
-                with torch.autocast("cuda", dtype=torch.bfloat16):
-                    pc = seq_logprob(model, mel, tgt, mask)
-                    pr = seq_logprob(model, mel, rej, mask)
-                    with torch.no_grad():
-                        rc = seq_logprob(dpo_ref, mel, tgt, mask)
-                        rr = seq_logprob(dpo_ref, mel, rej, mask)
-                    loss = -F.logsigmoid(
-                        args.dpo_beta * ((pc - pr) - (rc - rr))).mean()
-                opt.zero_grad(set_to_none=True)
-                loss.backward()
-                torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
-                opt.step()
-                run_loss += loss.item()
-                run_n += 1
-                if step % 100 == 0:
-                    msg = {"step": step, "dpo_loss": round(run_loss / run_n, 4)}
-                    print(json.dumps(msg), flush=True)
-                    with open(log_path, "a") as f:
-                        f.write(json.dumps(msg) + "\n")
-                    run_loss, run_n = 0.0, 0
-                if step % args.val_every == 0 or step == args.steps:
-                    ck = {"model": model.state_dict(), "opt": opt.state_dict(),
-                          "step": step, "best_val": 0.0, "args": vars(args)}
-                    torch.save(ck, os.path.join(args.out, "last.pt"))
-                    torch.save(ck, os.path.join(args.out, "best.pt"))
-                continue
-            with torch.autocast("cuda", dtype=torch.bfloat16):
-                if args.align or args.beat_head:
-                    logits, aux_logits, ptr_logits, beat_logits = model(
-                        mel, tgt, return_aux=True, return_extras=True,
-                        gsum=gsum, pos_bucket=posb, in_tgt=in_tgt)
-                elif args.aux and aux_t is not None:
-                    logits, aux_logits = model(mel, tgt, return_aux=True,
-                                               gsum=gsum, pos_bucket=posb, in_tgt=in_tgt)
-                    ptr_logits = beat_logits = None
-                else:
-                    logits = model(mel, tgt, gsum=gsum, pos_bucket=posb, in_tgt=in_tgt)
-                    aux_logits = ptr_logits = beat_logits = None
-                aux_loss = 0.0
-                if args.aux and aux_t is not None and aux_logits is not None:
-                    aux_loss = F.binary_cross_entropy_with_logits(
-                        aux_logits.float(), aux_t.to(device))
-                extra = 0.0
-                if ptr_logits is not None:
-                    # pointer alignment: positions whose GOLD token is TIME must
-                    # point at that frame's encoder position
-                    gold = tgt[:, 1:]
-                    is_t = (gold >= VOCAB.time0) & (gold < VOCAB.time0 + 1728) & mask[:, 1:]
-                    if is_t.any():
-                        target_pos = ((gold - VOCAB.time0) // 4).clamp(0, ptr_logits.shape[-1] - 1)
-                        pl = F.cross_entropy(
-                            ptr_logits.reshape(-1, ptr_logits.shape[-1]).float(),
-                            target_pos.reshape(-1), reduction="none",
-                        ).reshape(gold.shape)
-                        extra = extra + 0.2 * (pl * is_t).sum() / is_t.sum()
-                if beat_logits is not None and beat_t is not None:
-                    bt = beat_t.to(device)
-                    bm = bt >= 0  # -1 marks masked (grid-visible) windows
-                    if bm.any():
-                        extra = extra + args.beat_weight * F.binary_cross_entropy_with_logits(
-                            beat_logits.float()[bm], bt[bm])
-                loss = loss_fn(logits, tgt, mask, class_weight=class_weight) + 0.1 * aux_loss + extra
-                if teacher is not None:
-                    with torch.no_grad():
-                        # teacher may have a smaller vocab (no SIB/STYLE tokens):
-                        # clamp unseen ids to UNK and distill over the shared slice
-                        t_tgt = torch.where(tgt >= t_vocab,
-                                            torch.full_like(tgt, VOCAB.unk_cond), tgt)
-                        t_logits = teacher(mel, t_tgt)
-                    V = min(t_logits.shape[-1], logits.shape[-1])
-                    tau = args.kd_tau
-                    kl = F.kl_div(
-                        F.log_softmax(logits[..., :V].float() / tau, dim=-1),
-                        F.softmax(t_logits[..., :V].float() / tau, dim=-1),
-                        reduction="none",
-                    ).sum(-1)
-                    kl = (kl * mask[:, 1:]).sum() / mask[:, 1:].sum().clamp(min=1)
-                    loss = args.kd_alpha * loss + (1 - args.kd_alpha) * (tau * tau) * kl
-            opt.zero_grad(set_to_none=True)
-            loss.backward()
-            torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
-            opt.step()
-            run_loss += loss.item()
-            run_n += 1
-
-            if step % 100 == 0:
-                msg = {
-                    "step": step,
-                    "loss": round(run_loss / run_n, 4),
-                    "lr": round(lr_at(step), 6),
-                    "sec": round(time.time() - t0, 1),
-                }
-                print(json.dumps(msg), flush=True)
-                with open(log_path, "a") as f:
-                    f.write(json.dumps(msg) + "\n")
-                run_loss, run_n = 0.0, 0
-
-            if step % args.val_every == 0 or step == args.steps:
-                vl = validate(model, val_loader, device)
-                msg = {"step": step, "val_loss": round(vl, 4)}
-                print(json.dumps(msg), flush=True)
-                with open(log_path, "a") as f:
-                    f.write(json.dumps(msg) + "\n")
-                ck = {"model": model.state_dict(), "opt": opt.state_dict(),
-                      "step": step, "best_val": best_val, "args": vars(args)}
-                torch.save(ck, os.path.join(args.out, "last.pt"))
-                if vl < best_val:
-                    best_val = vl
-                    ck["best_val"] = best_val
-                    torch.save(ck, os.path.join(args.out, "best.pt"))
-
-    print(f"done. best_val={best_val:.4f}")
-
-
-if __name__ == "__main__":
-    main()

softchart/vocab.py

@@ -5,7 +5,9 @@ Sequence layout (decoder):
 Conditions may be UNK (condition dropout / unspecified at inference).
 """
 
+import math
 from dataclasses import dataclass
+from numbers import Real
 
 # --- audio / windowing constants (single source of truth) ---
 SR = 22050
@@ -14,7 +16,7 @@ HOP = 256
 N_MELS = 128
 FPS = SR / HOP  # 86.1328125 frames/sec
 WINDOW = 1728  # frames per training window (~20.06 s)
-MAX_TGT = 1536  # max decoder length (prefix + events + eos)
+MAX_TGT = 768  # max decoder length (prefix + events + eos)
 
 SLOTS = 96  # TJA lattice: slots per 4/4 measure (LCM of 16ths/triplets/32nds)
 MEAS_MAX = WINDOW // SLOTS  # max whole measures per window in slot mode (18)
@@ -64,7 +66,7 @@ class Vocab:
         base += len(NOTE_CLASSES)
         self.time0 = base
         base += WINDOW
-        # Extended condition tokens append after the original token ids.
+        # v2+ tokens appended at the end so older checkpoints stay loadable
         self.sep = base  # separates prev-window context from conditions
         self.sib = base + 1  # marks the sibling-chart (easier course) segment
         self.style = {s: base + 2 + s for s in range(8)}  # charting-intent codes
@@ -125,7 +127,7 @@ def complexity_band(frames):
 
 def encode_window(vocab, course, level, notes, cond_drop=0.0, rng=None, ctx_types=None,
                   sib_pairs=None, style=None, sync_band=None, plan_slice=None,
-                  complexity=None, mode=None):
+                  complexity=None, mode=None, duration_seconds=None):
     """Build a token sequence for one window.
 
     notes: list of (frame_idx, note_class_id) sorted by frame, frame in [0, WINDOW).
@@ -136,8 +138,15 @@ def encode_window(vocab, course, level, notes, cond_drop=0.0, rng=None, ctx_type
         missing slots filled with UNK. Encoded after [SIB].
     Returns (tokens, prefix_len) where loss should be applied after the prefix.
     """
+    if duration_seconds is None:
+        duration_seconds = WINDOW / FPS
+    if (not isinstance(duration_seconds, Real)
+            or isinstance(duration_seconds, bool)
+            or not math.isfinite(duration_seconds)
+            or duration_seconds <= 0):
+        raise ValueError("duration_seconds must be positive and finite")
     n_hits = sum(1 for _, c in notes if NOTE_CLASSES[c] not in ("end",))
-    nps = n_hits / (WINDOW / FPS)
+    nps = n_hits / float(duration_seconds)
     d = vocab.dens_bucket(nps)
 
     def maybe(tok):

static/app.css

@@ -594,29 +594,6 @@ audio {
   font-weight: 750;
 }
 
-.model-select {
-  width: 100%;
-  margin-top: 8px;
-  padding: 10px 12px;
-  color: var(--ink);
-  background: var(--surface-solid);
-  border: 1px solid var(--line);
-  border-radius: 10px;
-  font-size: 13px;
-  font-weight: 600;
-  cursor: pointer;
-  transition: border-color .18s, box-shadow .18s;
-}
-
-.model-select:hover {
-  border-color: var(--line-strong);
-}
-
-.model-select:focus-visible {
-  outline: 3px solid rgba(39, 111, 145, .35);
-  outline-offset: 1px;
-}
-
 .range {
   --range-progress: 88.89%;
   width: 100%;
@@ -1081,7 +1058,7 @@ audio {
 .loading-core,
 .metronome,
 .beat-pulses circle,
-.plan-cursor,
+.structure-cursor,
 .note,
 .export-sheet,
 .mix-wave,
@@ -1119,19 +1096,19 @@ audio {
 .is-active .beat-pulses circle:nth-child(5) { animation-delay: -1.6s; }
 
 /* Plan */
-.plan-bars rect { fill: currentColor; opacity: .13; }
-.plan-bars .plan-climax { fill: var(--red); opacity: .42; }
-.is-active .plan-bars rect { animation: plan-rise 1.9s var(--ease) infinite alternate both; transform-box: fill-box; transform-origin: center bottom; }
-.is-active .plan-bars rect:nth-child(2) { animation-delay: .12s; }
-.is-active .plan-bars rect:nth-child(3) { animation-delay: .24s; }
-.is-active .plan-bars rect:nth-child(4) { animation-delay: .36s; }
-.is-active .plan-bars rect:nth-child(5) { animation-delay: .48s; }
-.is-active .plan-bars rect:nth-child(6) { animation-delay: .6s; }
-.is-active .plan-bars rect:nth-child(7) { animation-delay: .72s; }
-.is-active .plan-bars rect:nth-child(8) { animation-delay: .84s; }
-.plan-path { stroke-dasharray: 520; stroke-dashoffset: 520; }
-.is-active .plan-path { animation: plan-draw 2.2s var(--ease) infinite; }
-.is-active .plan-cursor { animation: plan-cursor 2.2s var(--ease) infinite; }
+.structure-bars rect { fill: currentColor; opacity: .13; }
+.structure-bars .structure-peak { fill: var(--red); opacity: .42; }
+.is-active .structure-bars rect { animation: structure-rise 1.9s var(--ease) infinite alternate both; transform-box: fill-box; transform-origin: center bottom; }
+.is-active .structure-bars rect:nth-child(2) { animation-delay: .12s; }
+.is-active .structure-bars rect:nth-child(3) { animation-delay: .24s; }
+.is-active .structure-bars rect:nth-child(4) { animation-delay: .36s; }
+.is-active .structure-bars rect:nth-child(5) { animation-delay: .48s; }
+.is-active .structure-bars rect:nth-child(6) { animation-delay: .6s; }
+.is-active .structure-bars rect:nth-child(7) { animation-delay: .72s; }
+.is-active .structure-bars rect:nth-child(8) { animation-delay: .84s; }
+.structure-path { stroke-dasharray: 520; stroke-dashoffset: 520; }
+.is-active .structure-path { animation: structure-draw 2.2s var(--ease) infinite; }
+.is-active .structure-cursor { animation: structure-cursor 2.2s var(--ease) infinite; }
 
 /* Generate */
 .note { fill: var(--red); transform: translate(560px, 140px); }
@@ -1483,17 +1460,17 @@ footer a:hover {
   48%, 100% { opacity: .16; transform: scale(.75); }
 }
 
-@keyframes plan-rise {
+@keyframes structure-rise {
   0% { opacity: .12; transform: scaleY(.45); }
   100% { opacity: .35; transform: scaleY(1); }
 }
 
-@keyframes plan-draw {
+@keyframes structure-draw {
   0% { stroke-dashoffset: 520; }
   75%, 100% { stroke-dashoffset: 0; }
 }
 
-@keyframes plan-cursor {
+@keyframes structure-cursor {
   0% { transform: translate(0, 0); opacity: 0; }
   8% { opacity: 1; }
   12% { transform: translate(0, 0); }

static/app.js

@@ -15,13 +15,10 @@ const ui = {
   fileMeta: $("#file-meta"),
   removeFile: $("#remove-file"),
   fileError: $("#file-error"),
-  model: $("#model"),
   level: $("#level"),
   levelOutput: $("#level-output"),
   bpm: $("#bpm"),
-  autoPlan: $("#auto-plan"),
   useBeat: $("#use-beat"),
-  usePlanner: $("#use-planner"),
   sampling: $("#sampling"),
   samplingParameters: $("#sampling-parameters"),
   temperature: $("#temperature"),
@@ -53,13 +50,13 @@ const ui = {
   metricsGrid: $("#metrics-grid"),
   previewBlock: $(".preview-block"),
   chartTab: $("#chart-tab"),
-  planTab: $("#plan-tab"),
+  structureTab: $("#structure-tab"),
   chartPreview: $("#chart-preview"),
-  planPreview: $("#plan-preview"),
+  structurePreview: $("#structure-preview"),
   chartImage: $("#chart-image"),
-  planImage: $("#plan-image"),
+  structureImage: $("#structure-image"),
   chartEmpty: $("#chart-empty"),
-  planEmpty: $("#plan-empty"),
+  structureEmpty: $("#structure-empty"),
   downloadTja: $("#download-tja"),
   downloadAudio: $("#download-audio"),
   newChartButton: $("#new-chart-button"),
@@ -82,10 +79,10 @@ const stageDefaults = {
     title: "Finding the beat",
     detail: "Finding beats and tempo."
   },
-  plan: {
-    eyebrow: "PLANNING",
-    title: "Shaping the arc",
-    detail: "Planning density and climaxes."
+  structure: {
+    eyebrow: "STRUCTURE",
+    title: "Reading the structure",
+    detail: "Building whole-song hierarchical context."
   },
   generate: {
     eyebrow: "COMPOSING",
@@ -410,10 +407,6 @@ function setDownload(anchor, fileValue, defaultName) {
 
 function metricValue(key, value) {
   if (value === null || value === undefined || value === "") return "—";
-  if (key === "timing") {
-    if (value === "slot-exact") return "Grid-exact";
-    if (value === "time-quantized") return "Time-quantized";
-  }
   if (key === "grid_rms_ms" && Number.isFinite(Number(value))) return `${Number(value).toFixed(1)} ms`;
   if (key === "bpm" && Number.isFinite(Number(value))) return Number(value).toFixed(1);
   if (typeof value === "boolean") return value ? "Yes" : "No";
@@ -451,14 +444,14 @@ function renderMetrics(metrics) {
 function activatePreview(name, focus = false) {
   const chartActive = name === "chart";
   ui.chartTab.classList.toggle("is-active", chartActive);
-  ui.planTab.classList.toggle("is-active", !chartActive);
+  ui.structureTab.classList.toggle("is-active", !chartActive);
   ui.chartTab.setAttribute("aria-selected", String(chartActive));
-  ui.planTab.setAttribute("aria-selected", String(!chartActive));
+  ui.structureTab.setAttribute("aria-selected", String(!chartActive));
   ui.chartTab.tabIndex = chartActive ? 0 : -1;
-  ui.planTab.tabIndex = chartActive ? -1 : 0;
+  ui.structureTab.tabIndex = chartActive ? -1 : 0;
   ui.chartPreview.hidden = !chartActive;
-  ui.planPreview.hidden = chartActive;
-  if (focus) (chartActive ? ui.chartTab : ui.planTab).focus();
+  ui.structurePreview.hidden = chartActive;
+  if (focus) (chartActive ? ui.chartTab : ui.structureTab).focus();
 }
 
 function renderResult(payload) {
@@ -475,11 +468,15 @@ function renderResult(payload) {
 
   renderMetrics(payload.metrics);
   const hasChart = setImage(ui.chartImage, ui.chartEmpty, files.chart_image);
-  const hasPlan = setImage(ui.planImage, ui.planEmpty, files.plan_image);
-  ui.previewBlock.hidden = !hasChart && !hasPlan;
+  const hasStructure = setImage(
+    ui.structureImage,
+    ui.structureEmpty,
+    files.structure_image
+  );
+  ui.previewBlock.hidden = !hasChart && !hasStructure;
   ui.chartTab.hidden = !hasChart;
-  ui.planTab.hidden = !hasPlan;
-  activatePreview(hasChart ? "chart" : "plan");
+  ui.structureTab.hidden = !hasStructure;
+  activatePreview(hasChart ? "chart" : "structure");
 
   setDownload(ui.downloadTja, files.tja, "softchart.tja");
   setDownload(ui.downloadAudio, files.audio, "softchart-taiko-preview.wav");
@@ -524,14 +521,11 @@ function requestValues() {
     selectedCourse ? selectedCourse.value : "oni",
     Number.parseInt(ui.level.value, 10),
     Number.isFinite(bpm) ? bpm : 0,
-    ui.autoPlan.checked,
     ui.useBeat.checked,
-    ui.usePlanner.checked,
     ui.sampling.checked,
     Number(ui.temperature.value),
     Number(ui.topP.value),
-    Number(ui.drumVolume.value),
-    ui.model ? ui.model.value : "v1.7"
+    Number(ui.drumVolume.value)
   ];
 }
 
@@ -657,13 +651,13 @@ function bindEvents() {
   });
 
   ui.chartTab.addEventListener("click", () => activatePreview("chart"));
-  ui.planTab.addEventListener("click", () => activatePreview("plan"));
+  ui.structureTab.addEventListener("click", () => activatePreview("structure"));
   $$(".preview-tab").forEach((tab) => {
     tab.addEventListener("keydown", (event) => {
       if (!["ArrowLeft", "ArrowRight"].includes(event.key)) return;
       event.preventDefault();
-      const next = tab === ui.chartTab ? "plan" : "chart";
-      const target = next === "chart" ? ui.chartTab : ui.planTab;
+      const next = tab === ui.chartTab ? "structure" : "chart";
+      const target = next === "chart" ? ui.chartTab : ui.structureTab;
       if (!target.hidden) activatePreview(next, true);
     });
   });

static/index.html

@@ -3,7 +3,7 @@
 <head>
   <meta charset="utf-8">
   <meta name="viewport" content="width=device-width, initial-scale=1">
-  <meta name="description" content="SoftChart turns music into playable, musical Taiko charts with an instant drum-mix audition.">
+  <meta name="description" content="SoftChart V1.8 turns music into playable Taiko charts with a scratch-trained hierarchical model.">
   <meta name="theme-color" content="#f6f3ec">
   <title>SoftChart — AI Taiko Chart Studio</title>
   <link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 64 64'%3E%3Ccircle cx='32' cy='32' r='25' fill='%23e14b3f'/%3E%3Ccircle cx='32' cy='32' r='18' fill='%23fff8ed'/%3E%3Ccircle cx='32' cy='32' r='9' fill='%23e14b3f'/%3E%3C/svg%3E">
@@ -34,11 +34,11 @@
     <section class="hero" aria-labelledby="hero-title">
       <div class="hero-copy">
         <h1 id="hero-title">Music in.<br><em>Taiko out.</em></h1>
-        <p class="hero-lede">Upload a track. Get a playable TJA. Hear it instantly.</p>
+        <p class="hero-lede">Hierarchical V1.8 listens across the whole song, writes a playable TJA, and lets you hear it instantly.</p>
       </div>
       <ul class="hero-features" aria-label="Key features">
-        <li><strong>Beat-locked</strong></li>
-        <li><strong>Structure-aware</strong></li>
+        <li><strong>V1.8 scratch model</strong></li>
+        <li><strong>Whole-song hierarchy</strong></li>
         <li><strong>Instant audition</strong></li>
       </ul>
     </section>
@@ -83,23 +83,9 @@
         <div class="form-section split-rule">
           <div class="field-group">
             <div class="label-row">
-              <label for="model">Model</label>
+              <span>Model</span>
+              <strong class="field-hint">V1.8 · Hierarchical · 9.0M</strong>
             </div>
-            <select id="model" class="model-select">
-              <optgroup label="v1.7 — richest patterns (recommended)">
-                <option value="v1.7" selected>v1.7 · 7.9M — best quality</option>
-                <option value="v1.7-small">v1.7-small · 3.6M</option>
-                <option value="v1.7-tiny">v1.7-tiny · 1.4M — fastest</option>
-              </optgroup>
-              <optgroup label="v1.6 — scaling ladder">
-                <option value="v1.6">v1.6 · 7.9M</option>
-                <option value="v1.6-small">v1.6-small · 3.6M</option>
-                <option value="v1.6-tiny">v1.6-tiny · 1.4M</option>
-              </optgroup>
-              <optgroup label="v1.5 — single model">
-                <option value="v1.5">v1.5 · 7.9M</option>
-              </optgroup>
-            </select>
           </div>
 
           <fieldset class="field-group">
@@ -138,21 +124,11 @@
           </summary>
           <div class="advanced-body">
             <div class="toggle-list">
-              <label class="toggle-card compact">
-                <span><strong>Structure</strong></span>
-                <input id="auto-plan" type="checkbox" checked>
-                <span class="switch" aria-hidden="true"></span>
-              </label>
               <label class="toggle-card compact">
                 <span><strong>Beat grid</strong></span>
                 <input id="use-beat" type="checkbox" checked>
                 <span class="switch" aria-hidden="true"></span>
               </label>
-              <label class="toggle-card compact">
-                <span><strong>AI planner</strong></span>
-                <input id="use-planner" type="checkbox" checked>
-                <span class="switch" aria-hidden="true"></span>
-              </label>
             </div>
             <label class="toggle-card compact">
               <span><strong>Creative sampling</strong></span>
@@ -237,20 +213,20 @@
               </svg>
             </div>
 
-            <div class="stage-visual" data-stage="plan" aria-hidden="true">
+            <div class="stage-visual" data-stage="structure" aria-hidden="true">
               <svg viewBox="0 0 520 280">
-                <g class="plan-bars">
+                <g class="structure-bars">
                   <rect x="42" y="166" width="45" height="60" rx="5"/>
                   <rect x="96" y="143" width="45" height="83" rx="5"/>
                   <rect x="150" y="153" width="45" height="73" rx="5"/>
                   <rect x="204" y="105" width="45" height="121" rx="5"/>
                   <rect x="258" y="127" width="45" height="99" rx="5"/>
                   <rect x="312" y="83" width="45" height="143" rx="5"/>
-                  <rect x="366" y="53" width="45" height="173" rx="5" class="plan-climax"/>
+                  <rect x="366" y="53" width="45" height="173" rx="5" class="structure-peak"/>
                   <rect x="420" y="118" width="45" height="108" rx="5"/>
                 </g>
-                <path class="plan-path" d="M64 158 118 135l54 10 54-49 54 22 54-43 54-31 54 65" fill="none" stroke="currentColor" stroke-width="3" stroke-linecap="round" stroke-linejoin="round"/>
-                <circle class="plan-cursor" cx="64" cy="158" r="7" fill="currentColor"/>
+                <path class="structure-path" d="M64 158 118 135l54 10 54-49 54 22 54-43 54-31 54 65" fill="none" stroke="currentColor" stroke-width="3" stroke-linecap="round" stroke-linejoin="round"/>
+                <circle class="structure-cursor" cx="64" cy="158" r="7" fill="currentColor"/>
               </svg>
             </div>
 
@@ -351,16 +327,16 @@
           <div class="preview-block">
             <div class="preview-tabs" role="tablist" aria-label="Preview type">
               <button id="chart-tab" class="preview-tab is-active" type="button" role="tab" aria-selected="true" aria-controls="chart-preview">Chart preview</button>
-              <button id="plan-tab" class="preview-tab" type="button" role="tab" aria-selected="false" aria-controls="plan-preview" tabindex="-1">Song structure</button>
+              <button id="structure-tab" class="preview-tab" type="button" role="tab" aria-selected="false" aria-controls="structure-preview" tabindex="-1">Song structure</button>
             </div>
             <div class="preview-frame">
               <div id="chart-preview" role="tabpanel" aria-labelledby="chart-tab">
                 <img id="chart-image" alt="Preview of the generated Taiko chart">
                 <p class="preview-empty" id="chart-empty" hidden>No chart preview is available.</p>
               </div>
-              <div id="plan-preview" role="tabpanel" aria-labelledby="plan-tab" hidden>
-                <img id="plan-image" alt="Song energy and chart section plan">
-                <p class="preview-empty" id="plan-empty" hidden>No song structure preview is available.</p>
+              <div id="structure-preview" role="tabpanel" aria-labelledby="structure-tab" hidden>
+                <img id="structure-image" alt="Full-song mel spectrogram used for hierarchical context">
+                <p class="preview-empty" id="structure-empty" hidden>No song structure preview is available.</p>
               </div>
             </div>
           </div>