File size: 8,692 Bytes
f3a668e
 
548d5d3
 
 
f3a668e
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32a505c
f3a668e
 
 
 
32a505c
f3a668e
 
 
 
 
 
 
32a505c
 
 
f3a668e
 
 
 
 
 
32a505c
f3a668e
 
 
 
 
 
 
 
 
 
 
 
 
 
32a505c
 
f3a668e
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32a505c
f3a668e
 
 
 
 
 
 
 
 
 
32a505c
f3a668e
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32a505c
f3a668e
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32a505c
f3a668e
 
 
 
 
 
 
 
 
 
32a505c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
# Compute and providers

> **Status:** historical planning document. For what actually exists today, see
> [STATUS.md](STATUS.md).

Three credits, three different things. Only one of them runs a GPU you control.

| credit | what it actually is | what it buys | what it cannot do |
|---|---|---|---|
| **Baseten** $100 | inference on one hosted model | tokens on `meta-models/muse-spark-open-1026`, up to 2M TPM | training, weights, activations, Glimmer, or any other model |
| **Meta Model API** $150 | tokens | Muse Spark 1.1–1.3, Muse Wonder, Muse Image, SAM 3.1 | weights |
| **Modal** $30 | GPU seconds | ~12 A100-80GB hours, or ~7 H100 hours | a comfortable 30B training run |

The Baseten key is scoped to one model name. It is a **black-box inference grant**,
not a compute grant: no `from_pretrained`, no hooks, no residuals. That is fine for
the black-box channel and useless for the white-box one.

---

## 1. Answering "is it enough"

**Enough for:** the black-box channel on Spark Open, the judge, and a *small-model*
white-box channel. Approximately $65 of the $280 in credits, no out-of-pocket spend.

**Not enough for:** the white-box channel on Glimmer 30B. That needs a 30B model on
a GPU you control with hooks in the same process, about $35 of Vast, which none of
these credits cover.

So there are two honest paths.

### Path A, fully funded (recommended)

Run the white-box channel on a small Meta checkpoint (Llama-3.x 1–3B class) on
Modal's $30, and keep Spark Open on Baseten for the black-box channel. The method
is identical; the model is smaller. Cost fits inside the credits with margin.

| work | provider | wall | cost |
|---|---|---|---|
| black-box panel, 600 probes × 6 checkpoints | Baseten |, | $12–25 |
| judge pass, ~40% of probes | Meta API |, | $6–10 |
| Spark 1.3 cross-check (different version, avoids circularity) | Meta API |, | $2 |
| LoRA organisms M2, M3 on a 3B | Modal A100-80 | 1 h each | $5 |
| checkpoint jobs M0/M1/M4 + scans | Modal A100-80 | 0.5 h × 5 | $6.25 |
| attribution, 2 positives | Modal A100-80 | 2 h | $5 |
| demo endpoint | Modal A100-80 | 2 h | $5 |
| **total** | | | **~$49** |

### Path B, Glimmer 30B (stretch)

Same design with the 30B as the white-box subject. Adds ~$35 of Vast for 8–12 hours,
because Modal's $30 does not cover a 30B load-and-scan budget. Worth it only if the
extra scale is the story you want to tell.

A useful compromise: build the adapter against Glimmer, run the funded experiments on
the 3B, and upgrade if time and money allow. The adapter is architecture-generic; the
science does not depend on the parameter count.

## 2. The constraint that shapes execution

**Dedicated inference returns tokens, not residuals.** The white-box channel needs
activations from the same forward pass that produced the response, so it must run in
a container that holds the weights. Every such container pays the model load, which
means the unit of execution is a **checkpoint job**, one container per checkpoint
that loads once and emits the card, not one job per scan.

## 3. Baseten: black-box on Spark Open

```
base_url  https://inference.baseten.co/v1
model     meta-models/muse-spark-open-1026
```

OpenAI-compatible, so `keel/api.py` works unchanged with a base-URL and model
override. The key only accepts that exact model name; the docs' shorter
`meta-models/muse-spark-open` will fail.

Measured prices on this key: **$1.25/M prompt, $4.25/M completion, $0.15/M cached
prompt**. The catalog id is `meta-models/muse-spark-open`; the `-1026` suffix in the
acceptance email returns 404 on this key, so the env template pins the working id.

### Reasoning effort: a real lever, a noisy one

Spark Open always reasons. Baseten's table says the model has no reasoning control,
but `reasoning_effort` demonstrably changes output length. Three runs per setting,
same prompt:

| effort | completion tokens (3 runs) | mean |
|---|---|---|
| default | 311, 112, 404 | 276 |
| minimal | 202, 231, 38 | 157 |
| low | 77, 68, 97 | **81** |

`low` is reliably cheapest; `minimal` overlaps the default range and is not worth
using. The variance is large, treat any single measurement as noise.

### Four constraints that shape the black-box channel

1. **`logprobs` and `top_logprobs` are accepted but ignored**, so there is no
   token-distribution drift to measure. The black-box channel scores behaviour only.
2. **`n > 1` returns HTTP 400.** One response per request; sample by issuing separate
   requests.
3. **Neither `seed` nor `temperature: 0` guarantees reproducibility.** Sampling noise
   is a real variance component, which is why the panel needs enough probes rather
   than enough repetitions of a few.
4. **No structured outputs and no JSON mode**, Spark Open is the only model on the
   platform without them. The judge scores through tool calling (all models support
   it) or a strict parser, never a schema it cannot enforce.

### Cost, and the two-tier panel

The measured figure is about **$11.50 for a 600-probe sweep**, not the $2.40 extrapolated
from a one-line answer. Five full sweeps would consume most of the $100, so the panel
runs in two tiers.

| pass | panel | checkpoints | cost |
|---|---|---|---|
| baseline | 200 probes | M0 through M4 | 5 x ~$4 = ~$20 |
| confirmatory | 600 probes | M0, M1, M2, M3 | 4 x ~$11.50 = ~$46 |
| re-runs and failures | | | ~$15 |
| **Baseten total** | | | **~$81 of $100** |

The 200-probe pass finds out whether anything moved. The 600-probe pass is for the
checkpoints whose numbers get published, and for tightening an interval that came back
ambiguous. Shrinking the baseline pass is the lever that preserves the credit.

**White-box attribution does not come out of the Baseten credit.** Baseten is inference
on Spark Open only. Attribution and white-box scans run on Modal's $30.

| work | hours | cost |
|---|---|---|
| white-box scans, M0, M1, M2 | 1.5 h A100 | ~$3.75 |
| attribution, M1 and M2 | 3 h A100 | ~$7.50 |
| demo endpoint | 2 h A100 | ~$5 |
| **Modal total** | | **~$16 of $30** |

That leaves about $14 of Modal headroom for a failed job or a longer attribution search,
and keeps each credit doing the job it was granted for.

Training is $0 on Kaggle, so the credit pressure is entirely on the scanning side.

## 4. Meta Model API: the judge

| item | tokens | cost |
|---|---|---|
| judge calls on the subjective axes (~40% of 3,600 responses) | ~1.4M | ~$6 |
| Spark 1.3 as the cross-model judge (different version from the target) | ~0.3M | ~$1.50 |
| buffer for re-runs |, | ~$20 |
| **committed** | | **~$28** |

Judging with the same model and version you are testing is circular. Judging Spark
Open 1026 with Spark 1.3 is defensible: different checkpoint, same family. Say so in
the card.

## 5. Modal: the GPU

Verified 26 Oct 2026: the token launches a `Tesla T4, 15360 MiB` in workspace
`iamtrillion11`, and the billing summary reported `$0.00` metered before that run.
The CLI does not expose a credit balance, so entitlement is proven by launching,
not by reading a number.

$30 is ~12 A100-80GB hours at $2.50, ~50 T4-hours at $0.59, or ~37 L4-hours at $0.80.
That is enough for a 3B white-box run and a demo, not for a 30B training run.
A T4's 15 GB fits a 3B in BF16 or a 3B in 4-bit for LoRA; it is slow, so use it for
scans and dev loops rather than training.

Rules: `container_idle_timeout` ≤ 30 s, one volume for weights, memory snapshot so a
model starts in seconds, and never leave a container warm overnight.

## 6. What a scan costs

| scenario | GPU | tokens | total |
|---|---|---|---|
| black-box, 600 probes, Spark Open | $0 | ~$2.30 | **~$2.30** |
| white-box 3B, cold job | $1.25 | ~$2.30 | **~$3.55** |
| white-box 30B, cold job | $2.00 | ~$2.30 | **~$4.30** |

The honest product claim is **under $5 per checkpoint**. The card prints GPU and
token cost separately so the number stays auditable.

## 7. Day-0 checks

1. Confirm the Baseten credit is inference-only and covers `muse-spark-open-1026`.
2. Confirm the key works from your region.
3. ~~Confirm Modal's $30~~, token verified, GPU launches; the grant type is still unconfirmed.
4. Export a spend cap on all three accounts before the first job.
5. Run `python scripts/muse_smoke.py --meta` and `--baseten` before any task.

## 8. Fallbacks

| failure | response |
|---|---|
| Baseten quota exhausted | shift the black-box channel to Meta API Spark 1.3; the panel is portable |
| Modal grant is monthly | run the 3B white-box on a single Vast A100 for ~$3 |
| 3B too weak to show drift | go to Path B and spend ~$35 on Vast for Glimmer |
| Everything slower than planned | cut harden, then M5, then composition, never M3/M4 |