Spaces:
Running on Zero
Running on Zero
Commit History
-
Quote the budget for the settings that are actually set Running
Alexander +1: Claude Opus 5 (1M context)The "this clip costs N GPU-s" note was computed at eight steps and one forward pass no matter where the sliders stood, and left out the fixed cost of entering the worker. At Audio CFG 2 it under-quoted a 30 s clip by a third, which is the direction that hurts: the reservation is what gets billed, and the visitor sees the shortfall only as a call that dies. It also hid a real trap. `generate` passes the live pass count to `_longest_clip`, so guidance does not merely cost time -- it lowers the ceiling on clip length. Steps 16 with Audio CFG above 1.0 halves what a free visitor can score, 30 s down to 15, and nothing said so; the clip was simply cut. The note now names the sliders responsible. Audio CFG moves in 0.1 rather than 0.5, since the negative prompt weighs CFG - 1 and the useful low end lives between 1.0 and 1.5. Its info line says what the slider cannot: the extra forward pass is a step in the price, not a slope, so 1.1 costs what 2.0 costs for a tenth of the effect. The exam…f8d8dc2
-
Draw fresh noise for every window
Alexander +1: Claude Opus 5 (1M context)The latent shape is identical across windows, so a shared seed made the starting noise bit-identical -- and at 8 distilled steps the noise largely decides the character of the sound. Over similar-looking video the model restarted from the same point every window and produced the same gesture, heard as the opening seconds looping through the whole clip. Lengthening the window to 15 s cut the number of restarts but not the cause. The shared seed was hiding seams, since an independent draw gives each window its own timbre and level. Both of those are now handled directly: levels are fitted globally over the clip, and each window is conditioned on the previous one's tail. Window i gets seed + i. With a previous segment supplied the chain starts at an offset derived from that segment's audio rather than at zero, so parts of a long video do not all open on the same noise as part one -- which would put back at every part boundary the repetition this removes inside a part. FOLEY_SEED_PER_WI…0c4c394 -
Give the model 15 seconds to work with instead of four
Alexander +1: Claude Opus 5 (1M context)Windows were 121 frames, and each one started from a bit-identical noise tensor -- same seed, same latent shape -- over similar-looking video. Every four seconds the model restarted from the same point and produced the same gesture, which is heard as the opening seconds looping across the whole clip. The seam reference then pushed each window to continue the one before it, compounding it. Short windows lose twice: they multiply the hops in the reference chain, and they leave 4 s of new material per window for the model to develop anything in. 361 frames takes a 60 s clip from 15 windows to 5. Cost is close to a wash. The transformer sees tokens, the VAE takes 8 pixel frames to a latent frame, and the twice-generated overlap falls from 20% of a window to 7%. Only attention is quadratic in that; ATTENTION_SHARE prices it, deliberately high, because 97 and 121 frames are too close together to measure it and their ratio came out slightly sublinear. A 60 s clip reserves 127 s. The last w…425d490
-
Fit window levels globally, and stop one spike setting the clip's loudness
Alexander +1: Claude Opus 5 (1M context)Two fixes for one root cause, rather than another turn around the loop. Matching each window to its left neighbour is an exactly determined chain, and its solution is the cumulative product -- which drifts, because nothing holds it near unity. Capping each step only relocated that: the last run hit the cap at five seams out of eight, so each kept a ~40% step the crossfade could merely smear, and the transients that produced drove the peak normaliser into pushing the whole clip down. Confining the correction to the overlap, which is what stopped the drift earlier, is what left the steps. Both goals now go into one least-squares problem -- match at the seams, and stay near unity -- solved over the clip at once, with lambda chosen against three runs of ratios measured on the Space rather than picked. On the cleanest it leaves 3.0% mean residual mismatch for a 1.30x total excursion, where the unregularised chain drifted 1.56x; on the worst, 13%. None of the three reaches the guard, so MA…f32e42b -
Accept the previous segment as a reference, for clips longer than one run
Alexander +1: Claude Opus 5 (1M context)The chain that joins windows inside a clip now reaches across runs: hand back the result of the previous part and its closing audio conditions the first window, the same mechanism seeded from a file instead of from the window before. The one piece that was missing is the audio encoder, so AudioConditioner joins the resident set under `audio_vae`. It has to be built at module scope even though most runs never touch it -- ZeroGPU deletes the checkpoint once it has packed the weights, so a first build inside a GPU call would find nothing to read, which is the same trap that produced a FileNotFoundError earlier in this Space's history. Extracting the tail is ffmpeg work and happens in the handler, before the slot opens. `-sseof` seeks from the end so a long previous segment is not decoded in full, and the audio comes out already at the VAE's own 16 kHz rather than being resampled inside AudioProcessor. A file with no audio track logs and is ignored rather than failing the run. The UI no…4722183 -
Pick the crossfade law from the measured correlation, not by assumption
Alexander +1: Claude Opus 5 (1M context)Reported: at Video -> Audio scale 1.0 the sound is loud and hard to describe. Scale 1.0 means no guidance at all, so rescale cannot be involved -- the prediction is exactly cond and the factor is 1. The run points elsewhere: all eight seam gains above 1.0, one pinned at the 1.4 clamp, and a peak of 1.390, the highest yet. The crossfade was equal-power, cos and sin. That holds the level for *uncorrelated* material, and the two sides of a seam stopped being uncorrelated the moment each window was conditioned on the previous one's tail -- that is what the conditioning is for. Summed, cos + sin reaches 1.41 in the middle of the fade for identical signals: a +3 dB bump at every one of nine seams, periodic, with no name of its own, which is about how it was described. Now the two overlap regions are correlated and the blend divided by the true summed amplitude, sqrt(a^2 + b^2 + 2r*a*b). At r=0 that is 1 and the fade stays equal-power; at r=1 it is a+b and the fade becomes linear; between t…796da88 -
Raise the guidance rescale default; 0.7 was still leaving overload in
Alexander +1: Claude Opus 5 (1M context)Reported: Video -> Audio scale 1.3 with rescale 0.7 is audibly overloaded. The run backs it up -- rms 0.300 against a peak of 1.234, a crest factor of 12.3 dB. Clean Foley is transients on near-silence and runs far higher than that; peaks sitting that close to the body of the signal is what saturation looks like. 0.7 is the value image diffusion settled on, where a full variance match can wash a picture out. That trade does not carry over: 0.7 leaves 30% of the inflation in place, and nothing about a flat-sounding track is as bad as a distorted one. Default is now 0.9, and 1.0 is on the slider. Worth being clear about what this cannot do. The correction matches the prediction's overall standard deviation, so an overshoot confined to part of the signal survives it, and the peak scaling added in the previous commit only prevents the file from clipping -- it cannot undo saturation that happened in latent space before the vocoder ran. If 1.0 still overloads, the guidance is simply too st…fc15930 -
Stop clipping the output, and rebalance the seam reference
Alexander +1: Claude Opus 5 (1M context)The [audio] line from a good-sounding run reported peak=1.286. The model cannot produce that -- the BWE stage ends in clamp(-1, 1) -- so the overshoot was the seam level correction, and np.clip in the mux was flattening every peak above full scale. Foley is transients, which is precisely what hard clipping destroys, so some of the "poor quality" complaint may have been this. The track is now scaled to fit instead: 2.2 dB quieter on that run and otherwise unchanged. _audio_report still runs first so the logged peak stays honest, and an "exceeds full scale" line is now a signal that the correction is straining. What made it strain: 0.65 / 0.5 s stopped the chain compounding but went too far. The seam gains on that run were 2.00 0.59 1.75 0.88 1.16 1.34 1.72 1.04 -- 44% mean deviation against 5.6% at 1.0 / 1.0 s, one seam pinned at the 2.0 clamp, 6 dB worst case. The windows had stopped agreeing on level at all. So: strength back to 0.8 and 0.7 s, which the longer window pays for -- 9 h…85a1881 -
Lengthen the window to 121 frames to shorten the reference chain
Alexander +1: Claude Opus 5 (1M context)Residual noise still accumulates along a clip. Accumulation goes roughly as (carry per hop) ^ (number of hops); SEAM_STRENGTH and SEAM_SECONDS already cut the base, and the window length is the only lever on the exponent. 97 -> 121 frames takes a 35 s clip from 12 windows to 9, and a 60 s clip from 20 to 15. Cheaper than the frame count suggests. What the transformer sees is tokens, and the video VAE takes 8 pixel frames to one latent frame, so this is 13 -> 16 latent frames, ~1.23x -- and at roughly 2000 tokens a 22B model spends far more in its linear layers than in attention, so the per-window cost should scale close to linearly rather than quadratically. _per_chunk_cost now scales on latent frames, keeping the 97-frame calibration as its reference point. Net reserve for a 60 s PRO clip is 122 s against 125 s before, for five fewer hops. The real risk is not cost but that the Foley LoRA was trained at ~3.7 s (~89 frames): 97 was already slightly outside that and 121 is further out…3f52346 -
Turn on guidance rescale; it was off and that is why guidance crunched
Alexander +1: Claude Opus 5 (1M context)Raising Video -> Audio scale made the audio worse, and the cause was a default I copied without thinking. The guider computes pred = cond + (cfg-1)*(cond-uncond_text) + stg*(cond-uncond_perturbed) + (modality_scale-1)*(cond-uncond_modality) which *adds* to the prediction rather than mixing, so its variance grows with the scale -- and past the range the audio decoder was trained on, that is audible as a digital crunch. ltx_core has the standard correction right there, pulling pred back toward cond's standard deviation, and app.py passed rescale_scale=0.0. It was harmless while modality_scale stayed at 1.0, because then pred == cond and the factor is exactly 1; it stopped being harmless the moment the slider moved. Default 0.7. Exposed as a slider rather than only an env var because it has to be judged by ear against the Video -> Audio scale it is correcting, and an env change costs a Space restart and a 42 GB re-pack per value. Free in both direct…edd1ffe -
Give Video -> Audio scale a usable step, and a range that matches reality
Alexander +1: Claude Opus 5 (1M context)Reported: 1.0 is not enough, 1.2-1.3 is what is wanted, and by 2.5 digital artefacts appear. A 0.5 step could not express any of that. Step is now 0.1. The top of the range comes down from 6.0 to 3.0 as well -- not to remove a capability but because it was never one: everything above 2.5 was already past the point where the guidance starts breaking the audio, and spreading 0.1 steps over an unusable 5-wide range makes the part that matters fiddly to hit. The useful zone is now most of the slider instead of its first fifth. The info text says what the values do and that anything above 1.0 costs an extra forward pass per step. That cost is real -- it roughly doubles GPU time per window -- but it does not shorten the clip that fits: free stays at its full 30 s cap and PRO at 60 s, both still inside their reserve ceilings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U6SVxp9XWW8tngHpN7uaw51b59cd7 -
Stop the seam reference compounding along the clip
Alexander +1: Claude Opus 5 (1M context)Reported symptom: the sound gets progressively more monotonous and mushy the further into a clip it goes. That is the signature of the chain I built, and the risk I flagged when shipping it -- window N references N-1, which referenced N-2, so whatever the reference carries is carried forward. Two things made it carry rather than fade. strength=1.0 holds the reference perfectly clean, which makes it something the model must reproduce rather than reinterpret, so an artifact propagates verbatim and compounds. And the reference ran the full overlap, 25 latent frames of a ~101-frame window: a quarter of every window was dictated by the one before it, leaving much less room for the video to drive the sound -- which is the same complaint from the other side. Now strength 0.65 and 0.5 s. Below 1.0 each hop partially re-denoises the reference, so a carried artifact decays geometrically along the chain instead of accumulating. Both are env-tunable (FOLEY_SEAM_STRENGTH, FOLEY_SEAM_SECONDS) so t…df885b6 -
Log what the audio track actually is
Alexander +1: Claude Opus 5 (1M context)"Sounds mono and low quality" is checkable rather than arguable, and the code does not settle it: the audio VAE is in_channels/out_ch 2, and both the vocoder and the BWE stage declare stereo: true, so the pipeline is stereo end to end and should be running 16 kHz -> 48 kHz bandwidth extension. The BWE weights are in the checkpoint (vocoder.bwe_generator.*, 557 tensors), the key filter keeps them, and VocoderConfigurator returns VocoderWithBWE whenever the config carries a "bwe" block -- which this one does. So on paper none of it is missing. Three measurements settle it instead. L/R correlation: 1.000 means the model puts the same signal in both channels and it is mono no matter what the plumbing says. Energy above 8 kHz: the mel front end stops at mel_fmax 8000, so anything above that can only have come from the BWE, and near-zero means we are shipping band-limited audio. Sample rate and peak catch a plumbing mistake in the mux. Runs on the CPU in the handler, so it costs no quota. …2879587 -
Stop the seam correction drifting, and stop re-encoding the video
Alexander +1: Claude Opus 5 (1M context)A 35 s run (12 windows) says the reference conditioning works: the two sides of a seam disagree on level by 5.6% on average and 1.7 dB at worst. It also says my level matching had become the bigger artifact. Each window was scaled as a whole to match the already-corrected window before it, which makes the correction a random walk with nothing to pull it back -- the twelve gains compounded into a 4.1 dB swell and fade across the clip, considerably louder than the seams being fixed and not present in the video at all. The correction now applies only across the overlap and decays to 1.0 by the end of it: the seam is still levelled, window bodies keep their own level, nothing accumulates. Worst case is now a 1.1 dB step spread over a 1 s fade. Second, the mux took 59.3 s on that clip -- as long as all twelve windows of diffusion together -- because it decoded the normalized frames into a float tensor and ran them back through libx264. Those frames are already what we ship, so it is now a…7379687 -
Condition each window on the previous window's closing audio
Alexander +1: Claude Opus 5 (1M context)A crossfade hides a seam; it cannot make the two sides agree. One seed for the whole clip and RMS level matching helped, but the windows still drift, because nothing told window N what window N-1 actually ended up sounding like -- same prompt, same noise, still an independent take. ltx_core ships the piece for this. AudioConditionByReferenceLatent appends a reference latent as clean, non-denoised tokens, and AudioPatchifier documents a `shift` argument for exactly this case: "constructing overlapping windows from the same latent sequence". The previous window's tail goes in with default patchifier settings and shift=0, so its RoPE timestamps land precisely where this window's own opening tokens sit -- the overlap becomes a continuation rather than a coincidence. strength=1.0 keeps the reference clean. Cost is ~25 tokens on a ~2000-token sequence, so the duration model is unaffected. The window latent is trimmed to the real audio length before being handed on, so the reference is the …9fe0688 -
Widen the worker entry-lag margin; it is the one unstable constant
Alexander +1: Claude Opus 5 (1M context)The verification run put the entry lag at 4.8 s against a constant of 5.0 -- 4% of headroom, where everything else has ~30%. The first run measured 3.2 s for the same work, so fork-and-restore of ~42 GB clearly moves with machine load and cannot be pinned as tightly as the rest. Raised to 8.0. The other constants held. Per-window came back 4.52/4.54 s against 4.53/4.55 s on the first run and a predicted 5.7 -- stable to within 0.02 s across runs. The run reserved 31 s and was billed 22.8 s. Both changes from the previous commit are confirmed working: 8.2 s of ffmpeg now sits outside the billed slot, and no `429` has appeared since the deploy. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U6SVxp9XWW8tngHpN7uaw5a17cd40 -
Calibrate the cost model, and stop billing ffmpeg as GPU time
Alexander +1: Claude Opus 5 (1M context)The instrumented run came back with numbers that change the picture. The cost model was ~3x too high: 4.55 s per window measured against 16.8 s predicted, 2.1 s for a cold prompt against 50 s. That is not harmless caution. The logs from that run show two real visitors -- one signed in, one anonymous -- taking `429 Too Many Requests` from ZeroGPU on requests for 207 s, because the Space was asking for roughly three times what it uses. Over-reserving does not protect the quota, it refuses the slot to someone who had it. Every constant is now calibrated with ~30% headroom over measurement, and _duration receives the planned window list rather than re-probing the container, so the count is exact. The run we measured would have reserved 100 s; it now reserves 31 s against 20.7 s actually used. Second, _normalize and the mux are ffmpeg work and were costing 1.7 s and 6.2 s of a 25.4 s call -- nearly a third of the budget spent on the CPU with a GPU slot held open. They move either side of …ace3e55 -
Instrument a run end to end to measure ZeroGPU's per-call overhead
Alexander +1: Claude Opus 5 (1M context)The long-video question turns on one unmeasured number: what a @spaces.GPU call costs before it does any work. If forking a worker and restoring the packed resident tensors is ~2 s, windows can be spread over many calls and a job can use a whole day's quota; if it is ~30 s, they have to be batched. The cost model (1.6*steps+4 per window) has never been checked against a real run either, and on 19 windows an error in it multiplies by 19. Splits the Gradio handler from the GPU function so the boundary can be stamped from both sides. The handler resolves the visitor's tier and passes a plain bool, which is better anyway: OAuth objects no longer have to survive the trip into the worker. Wall-clock time.time() rather than perf_counter, which has no common origin across processes. One SUMMARY line per run reports: measured seconds per window against what the model predicts, the entry lag, and the cost of each stage. _normalize and the final mux are stamped separately because both are pure …eddb059 -
Drop references to the deleted TASKS.md
Alexander +1: Claude Opus 5 (1M context)Four comments pointed at a file that is no longer in the tree; each now says the thing it was pointing at. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U6SVxp9XWW8tngHpN7uaw5c85d8d4 -
Size limits to the visitor's own ZeroGPU tier, not the free one
Alexander +1: Claude Opus 5 (1M context)ZeroGPU bills the visitor's quota, not the Space owner's, so a PRO visitor really does have a larger budget on this Space -- and the hardcoded free-tier caps were spending none of it. The Space now signs visitors in (hf_oauth), reads isPro from the OIDC userinfo, and sizes both the accepted clip length and the reserved @spaces.GPU duration per tier: 16 s / 150 GPU-s for free and signed-out, 60 s / 600 GPU-s for PRO. All four numbers are env-overridable. Also fixes a real hazard the old ceiling had: min(cost, 150) *under-reserved* for a long clip rather than shortening it, and ZeroGPU stops a call at its reserved duration -- so an over-long clip would have died mid-generation with every one of those seconds already billed. The ceiling now trims the work instead. _longest_clip inverts the cost model over the actual settings, so raising Audio CFG or STG shortens the clip that fits rather than overrunning the slot. Verified: no combination of tier x CFG x STG exceeds its ceiling. The PRO…95aa841
-
Reduce audible differences between chunks
Alexander +1: Claude Opus 5 (1M context)The first end-to-end run works, and the windows are audibly different from each other. Three changes, cheapest first: One seed for the whole clip instead of seed + index. Varying it gave each window an independent take -- different timbre and level. The windows already differ because the video under them differs; the noise need not add to that. This was my mistake: I varied the seed to avoid identical windows, which was never a risk. A per-window gain matched on the overlap. Both windows describe the same frames there, so they ought to be equally loud, which gives a principled correction without flattening real dynamics the way normalizing whole chunks would. Clamped to 2x, and compounding along the clip since each window matches the corrected signal before it. Crossfade lengthened from 0.67 s to 1.0 s, which costs no extra windows at typical clip lengths (281 frames stays at 4, 361 stays at 5). None of this carries the identity of a sound across a boundary -- each window is still …22218b4 -
Cache safetensors metadata so reads survive ZeroGPU's blob cleanup
Alexander +1: Claude Opus 5 (1M context)Prompt encoding now completes -- inference_mode was the bug -- and the run gets as far as the first diffusion chunk, where it dies on: FileNotFoundError: .../ltx-2.3-22b-distilled-fp8.safetensors ZeroGPU deletes source files once their tensors are packed ("Cleaned 27.50GB of tensor files ... after packing"), so the checkpoint is gone by call time. That is fine for the weights, which are already resident, but DiffusionStage reaches for the file again mid-call: supports_generated_keyframes reads the header to look up the transformer config. read_model_metadata is now memoized by path and primed at module scope while the file still exists. read_model_config resolves it by module global, so callers that imported read_model_config directly are covered too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VNp8YiVRcjWBNqPguTw85hda763c4 -
Run generation under torch.inference_mode()
Alexander +1: Claude Opus 5 (1M context)This is the actual bug behind all four failures, and it was mine: generate() had no inference_mode and no no_grad. Both reference Spaces have it; I dropped it when restructuring the pipeline and never noticed. With autograd live, the prompt encode builds a graph. Fp8CastLinear upcasts a bf16 copy of its weight on every forward, and F.linear saves that copy for backward, so all 336 quantized Gemma projections are retained -- tens of GB on top of a resident set that leaves 6.8 GB free. The same mechanism explains the bitsandbytes failure at 21.7 GB free: the dequantized weights were retained too. That also explains why the probe passed. 64 MB, 512 MB and 2 GB all allocate fine because a bare torch.empty is not part of any graph; the exhaustion only happens once activations and saved weights accumulate across 48 layers. And it explains why the crash site kept moving between bitsandbytes, an activation and a matmul: it OOMs wherever it happens to cross the line, and the NVML assert is wh…278142c -
Probe plain CUDA allocations at the start of the GPU call
Alexander +1: Claude Opus 5 (1M context)Three fixes have targeted three suspects -- bitsandbytes, emulated CUDA at module scope, the allocator config -- and the same assert survived all of them. Every failure has been the first genuine allocation inside the worker, so ask that question directly: allocate 64 MB, 512 MB and 2 GB and log the outcome. A 64 MB failure with gigabytes reportedly free would mean the residency budget is the problem and both 'allocated' and mem_get_info are blind to whatever ZeroGPU holds to stage the restored weights. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VNp8YiVRcjWBNqPguTw85he30c68c -
Stop setting expandable_segments; it is the last shared suspect
Alexander +1: Claude Opus 5 (1M context)The same NVML assert now fires with bitsandbytes gone entirely -- this time from plain Gemma attention at modeling_gemma3.py:149. That weakens the previous conclusion that bitsandbytes was at fault: what the two failures actually share is that each is the first fresh CUDA allocation inside the ZeroGPU worker. Weights restored by ZeroGPU never travel the caching allocator's segment path, so the first real allocation is precisely where an expandable-segments problem would surface, and PYTORCH_CUDA_ALLOC_CONF is the only allocator-level setting this app makes. Flipping the default to off changes one variable. If it still fails, the next variable is headroom: 42.3 GB of a 50.9 GB slice is resident, leaving 8.1 GB, so dropping text_encoder from FOLEY_RESIDENT is the follow-up test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VNp8YiVRcjWBNqPguTw85head6e57 -
Quantize the text encoder with plain-PyTorch fp8 instead of bitsandbytes
Alexander +1: Claude Opus 5 (1M context)bitsandbytes' 4-bit CUDA backend does not work on this ZeroGPU stack. The telemetry ruled out both earlier theories: the failure is identical inside the GPU call where CUDA is real, and 21.7 GB of 50.9 GB was free when a 118 MB allocation failed. Landing in _dequant_linear_fallback means bnb found no usable kernel at all, consistent with a build without sm_120 support. Gemma's decoder projections are now stored as float8_e4m3fn and upcast in forward, reusing Fp8CastLinear from ltx_core.quantization.fp8_cast -- the same mechanism LTX applies to its own transformer, and no third-party kernels. At ~12 GB the encoder fits beside the resident transformer (28.7 + 12 of 50.9 GB), so it goes back into the resident set and ZeroGPU packs it at startup. bf16 was not an option: 24.4 GB against 22.2 GB free is a real OOM. Weights come from the gated QAT repo when HF_TOKEN is set, since those are what LTX-2.3 was trained against, and from an ungated mirror of the vanilla weights otherwise so the S…48e01e6 -
Record the open text-encoder task: bitsandbytes 4-bit is unusable on ZeroGPU
Alexander +1: Claude Opus 5 (1M context)Both hypotheses behind the previous fix are disproved by the VRAM telemetry: the failure is identical inside the GPU call where CUDA is real, and 21.7 GB of 50.9 GB is free when a 118 MB allocation fails. bitsandbytes' 4-bit CUDA backend is simply broken on this ZeroGPU/PyTorch/Blackwell combination -- it never reaches its fused kernel, landing in _dequant_linear_fallback. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VNp8YiVRcjWBNqPguTw85h9fd7ff2 -
Build the text encoder inside the GPU call, and cache prompt embeddings
Alexander +1: Claude Opus 5 (1M context)A run died on the first forward of the NF4 Gemma inside the ZeroGPU worker: bitsandbytes .../cuda/ops.py _dequant_linear_fallback torch.empty(shapeB, dtype=A.dtype, device=A.device) RuntimeError: NVML_SUCCESS == r INTERNAL ASSERT FAILED at CUDACachingAllocator.cpp:1165 That assert is what PyTorch raises when an allocation fails and it then cannot reach NVML to report memory stats, so it is a failed allocation, not an NVML bug. The likely cause is that bitsandbytes quantizes parameters during the move to CUDA, and warmup performed that move at module scope where ZeroGPU only has emulated CUDA. So the text encoder comes out of the resident set and is built inside the GPU call, where CUDA is real. To keep that from costing a Gemma load on every run, encoded prompt embeddings are now cached to disk: ZeroGPU forks a worker per call so process memory cannot carry a cache, but the container filesystem is shared, and the tensors are only a few MB. duration= consults the cache so…922bc19 -
Shorten short_description to the 60-char Space metadata limit
Alexander +1: Claude Opus 5 (1M context)Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VNp8YiVRcjWBNqPguTw85h2b8e6af -
Tidy: drop unused warmup import and write-only flag, hoist Audio import
Alexander +1: Claude Opus 5 (1M context)Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VNp8YiVRcjWBNqPguTw85ha7b1404 -
Rebuild the Foley Space to fit a free ZeroGPU quota
Alexander +1: Claude Opus 5 (1M context)The reference Space (hugging-apps/ltx-foley-v2a) cannot run on a free account at all: it declares duration=310 against a 300 s/day quota, and downloads ~70 GB of weights. Three changes bring a run down to an estimated 30-50 s of GPU time. Resident weights. Almost all of the reference's ~265 s per run is weight streaming, not diffusion -- every class in ltx_pipelines.utils.blocks builds its model inside __call__ and frees it via gpu_model() on exit. Each model is now built once at module scope (also what ZeroGPU wants, since it forks a worker per call) with dispose() shadowed by a no-op. Distilled + fp8. ltx-2.3-22b-distilled-fp8 runs 8 steps at CFG=1 instead of 30 steps with CFG+STG, and loads 29.5 GB instead of 46.1 GB. Uses fp8-scaled-mm rather than fp8-cast so the checkpoint's per-tensor scales survive and torch._scaled_mm runs natively on Blackwell. Smaller inputs. Gemma-3-12B comes from a pre-quantized NF4 repo (7.8 GB vs 24.4 GB) through PromptEncoder's text_encoder_builder ho…bbd4bcf -
initial commit
Verified signaturee33f85f