VibeGame-8B / code /docs /demo.md
khursheed's picture
Release pipeline source, tests, notebook, and licenses
ae36cfb verified
|
Raw History Blame Contribute Delete
9.63 kB

Building the 60-second walkthrough

The launch video is an edited walkthrough of measured model output. It never pretends that a 60-second edit proves a game was generated in 60 seconds. Actual generation and prompt-to-launch times appear on screen, even if they exceed the hook. There is no simulated typing or hidden repair. The original fixed-sample mode retains every selected failure. A separate success-only showcase is available below; it changes display selection, never benchmark results.

The default builder needs a completed evaluation of all 500 frozen prompts. Its three sample slots are fixed at indices 0, 200, and 400 before the results are known, spanning three distinct gameplay families. Each successful recording must match the original generation file, project contents, evaluation manifest, prompt ID, and recorded video hash. The compositor recomputes the aggregate counts from all 500 results and rejects partial runs or infrastructure failures.

Record the fixed samples

On the free Colab Linux runtime, install ffmpeg, xvfb, bubblewrap, Mesa software rendering, fonts-dejavu-core, and the Python package Pillow. No paid API or GPU is used for video editing. Each capture below runs the original unmodified generated project in the isolated recorder. The notebook installs DejaVu so arrows, em dashes, and bullets render correctly; Pillow's fallback font can leave missing glyphs in otherwise valid footage.

python scripts/record_game.py --evaluation /path/to/evaluation --index 0 \
  --godot /path/to/Godot --seconds 15 \
  --output /path/to/evaluation/recordings/sample-000.mp4
python scripts/record_game.py --evaluation /path/to/evaluation --index 200 \
  --godot /path/to/Godot --seconds 15 \
  --output /path/to/evaluation/recordings/sample-200.mp4
python scripts/record_game.py --evaluation /path/to/evaluation --index 400 \
  --godot /path/to/Godot --seconds 15 \
  --output /path/to/evaluation/recordings/sample-400.mp4

A sample that fails the clean-run test cannot be recorded as a successful game. Leave its slot in place. The compositor displays its failure as a diagnostic card for the same amount of time; it does not choose a different prompt. A successful sample with missing footage also gets a diagnostic card. At least one of the three fixed slots must have a genuine verified capture before a walkthrough can be built.

Compose and verify

python scripts/build_demo.py \
  --evaluation /path/to/evaluation \
  --output /path/to/VibeGame-8B-launch.mp4

The exact timeline is:

Time Content
00:00–00:08 Actual first sample prompt and its measured latency
00:08–00:14 Filenames, line counts, and source excerpt from its saved generation
00:14–00:28 Sample 0 capture, or its explicit diagnostic card
00:28–00:42 Sample 200 capture, or its explicit diagnostic card
00:42–00:56 Sample 400 capture, or its explicit diagnostic card
00:56–01:00 Compile and clean-run counts over all 500 prompts

Captures use fixed-timestep Godot Movie Maker rendering. They contain no human or simulated player input. This caveat remains visible during footage. The edit uses the first 14 seconds of each 15-second capture at its original simulation speed; it does not loop, speed up, or slow down gameplay. There is no music or voiceover. Gameplay is fitted inside a dedicated region so explanatory overlays do not cover the captured game image.

The builder uses FFprobe to verify every source/segment duration and the final 1280×720, 30 fps, 60-second video. It writes:

  • The requested .mp4.
  • A matching .demo.json with source hashes, aggregate counts, editing disclosures, capture metadata, and final duration evidence.
  • A matching .assets/ folder with the generated title cards and edit segments.

Existing videos or evidence are never silently overwritten. Choose a new output name to build another edit. Review the finished video before posting: a clean headless run cannot tell us whether a game is enjoyable or even meaningfully interactive. If sample failures are present, the accompanying release language must acknowledge them.

Optional success-only showcase

This display policy was declared at 2026-10-04 14:31:55 UTC, after the A100 experiment began. It was not a pre-blind benchmark sample plan. The benchmark's fixed indices remain 0, 200, and 400, and the default recorder/compositor behavior above is unchanged.

The explicit showcase mode waits for a complete, infrastructure-clean evaluation of all 500 prompts. It chooses exactly the first three clean-run successes in ascending frozen prompt index, from that same experiment. It does not choose the prettiest games or pool T4 and A100 results. It freezes a receipt binding all 500 attempts/results, the summary and manifest, and each selected output, project, and validation log. Changed evidence stops the build.

python scripts/showcase.py --evaluation /path/to/evaluation-a100-v1

Read showcase/selection.json for the three indices. For each exact index N, run:

python scripts/record_game.py --evaluation /path/to/evaluation-a100-v1 \
  --showcase-selection /path/to/evaluation-a100-v1/showcase/selection.json \
  --index N --godot /path/to/Godot --seconds 15 --fps 30 \
  --output /path/to/evaluation-a100-v1/showcase/recordings/sample-NNN.mp4
python scripts/build_demo.py --evaluation /path/to/evaluation-a100-v1 \
  --showcase-selection /path/to/evaluation-a100-v1/showcase/selection.json \
  --output /path/to/run/VibeGame-launch-60s.mp4

The output filename uses the zero-padded index. All three real captures are required. If one selected capture fails, it cannot be replaced by a later success; if the benchmark contains fewer than three clean results, three genuine recordings are unavailable. Source repair, generation retries and teacher-seed substitution are not allowed.

Every showcase frame says SELECTED SUCCESSES — NOT REPRESENTATIVE, shows the full compile/clean-run counts and rates over 500, and identifies the video as a 60-second edit rather than generation time. The intro and gameplay overlays show actual whole-batch generation and prompt-to-launch times, or explicitly report unavailable timing. The existing 8+6+14+14+14+4-second timeline is retained. The game image is fitted above the disclosure panel without covering its content. No player input is simulated. These are captures of untouched clean-running outputs, not evidence of responsive controls or correct game mechanics.

For notebook cell 18, set EVAL = RUN / "evaluation-a100-v1" and DEMO_MODE = "showcase" before running it. Its default remains "fixed". Publication recomputes the deterministic selection, verifies all three recordings and their hashes, and retains the original 500-result release requirements. The model card states the success-only selection and puts the actual pass counts and prototype limitations near the top.

Actual A100 walkthrough review

The completed 500-prompt A100 pass produced 6 compile successes (1.2%) and 5 clean 60-second runs (1.0%). Its measured median prompt-to-launch time among successful launches was 282.8 seconds; none launched within 60 seconds. These results describe the evaluated 4-bit base plus adapter, not separate merged-fp16 or GGUF benchmarks.

The deterministic showcase selected original prompt indices 56, 436, and 443. Review of nine frames from the actual edit showed moving Pong footage, a pinball game waiting for Space, and a mostly stationary rally game with a running timer. The captures contain no player input. They do not demonstrate responsive controls, working scoring, or enjoyable gameplay, and the sparse footage should not be presented as a polished game-generation result. Installing DejaVu and recomposing the overlays fixes missing text glyphs without changing the selected generations, captures, timings, or benchmark results. Preserve the initial video and its sidecar before making that typography-only edit, then inspect the new video.

Optional scaffold preview, clearly separate from model output

The repository's original teacher seed examples can be shown before training is finished, but only through an explicit preview mode. Record three seed project JSON files with record_game.py --project-json, then pass the matching examples and recordings in the same order:

python scripts/build_demo.py --scaffold-preview \
  --example examples/meteor_dodge --recording /path/to/seed-dodge.mp4 \
  --example examples/brick_bounce --recording /path/to/seed-bricks.mp4 \
  --example examples/star_patrol --recording /path/to/seed-patrol.mp4 \
  --output /path/to/VibeGame-8B-seed-preview.mp4

The flag cannot be combined with --evaluation. Every frame carries the permanent watermark TEACHER SEED PREVIEW — NOT TRAINED MODEL. The title cards make no inference-latency or trained-model performance claim. Sources must match the recorder's project digest. When a seed matches data/synthetic_candidates.jsonl, its original instruction is displayed; otherwise the missing original prompt is disclosed. A preview is not the requested model launch asset and must never be described as one.

Run evidence/layout tests with:

python -m unittest discover -s tests -p test_demo.py -v

For the optional CPU-only encoding integration test, set VIBEGAME_VIDEO_INTEGRATION=1. That test uses a temporary colored test source, verifies the complete 60-second encoding path, and deletes the fixture afterward. It produces no purported model demo or benchmark evidence.