ACTION NEEDED @WenyanCong: free-tier ZeroGPU 450s blocks all non-Pro submissions
Summary
/submit on this Space fails for free-tier HF users with:
AppError: The requested GPU duration (450s) is larger than the maximum allowed
Repro
from gradio_client import Client, handle_file
c = Client("nvidia/PhysicalAI-OOD-Leaderboard", token=HF_TOKEN)
c.predict(handle_file("submission.json"), "model", "org", "", "track2", True, api_name="/submit")
Context
- Free ZeroGPU daily budget is 300s < 450s request → free accounts cannot score at all.
- Format validation works (Track2 349 keys / Track1 284 keys). Failure is only at private ZeroGPU evaluator.
- Pro accounts still score (new LB rows 2026-07-22). Historical free rows exist from ~2026-07-11 only.
- Related: challenge discussions #2, #3; this Space discussion #1.
Ask
Please lower private evaluator @spaces.GPU(duration=…) to ≤300s (ideally ≤120), or state that HF Pro is required for submission.
@WenyanCong — format-valid packs ready offline under HF VeigaPunk; can resubmit immediately after a fix. Contact: jpveigao10@gmail.com
R11: offline packs published for organizer scoring
Because free-tier still cannot call the private ZeroGPU evaluator (duration=450s), I published format-validated submission JSON here:
https://huggingface.co/datasets/VeigaPunk/physicalai-ood-submissions-pending
| File | Track | Keys |
|---|---|---|
submission_track2_val_gold.json |
track2 | 349 |
submission_track1_goldstyle_v3.json |
track1 | 284 × 6 |
@WenyanCong — please either:
- Lower private evaluator
@spaces.GPUduration to ≤300s so free users can/submit, or - Score these packs under HF user VeigaPunk / org VeigaPunk and publish if metrics warrant.
Contact: jpveigao10@gmail.com. Ready to resubmit instantly once ZeroGPU allows 450s or duration is reduced.
Confirming this is still broken on 2026-08-07, and adding a data point that changes the diagnosis: an HF Pro account fails too. This is not only a free-tier issue. Account Nurymanau, isPro=true, canPay=true, verified via whoami-v2 immediately before the attempts. Track 1 submit returns an immediate opaque failure. Tried with the same instant error: the Gradio UI form while logged in as Pro with the file loaded and fields typed by hand; gradio_client; the raw REST path (/gradio_api/upload then /gradio_api/call/submit), authenticated and anonymous; the full 284-key file and a reduced 5-key probe; several model names, with and without organization and contact. Format is not the cause: 214 clips / 284 events, keys "clip_id_eventidx" with 0-based per-clip indices, exactly 6 non-empty rollouts each, matching the documented spec and the announced 214/284 counts. Two asks, either of which unblocks everyone. First, lower the private evaluator's GPU duration below the ZeroGPU ceiling as already requested in #1 — since Pro also fails, 450s appears to exceed more than just the free allowance. Second, and cheaply, please set show_error=True on the Space: right now the failure surfaces as an empty error with no message, so participants cannot tell a quota problem from a format problem and lose days guessing. Happy to resubmit within minutes of a fix, or to have a published pack scored offline if that is easier on your side.