# Image for the ARChitects ARC Prize 2024 reproduction (SCRUM-80) on HuggingFace Jobs. # # NOTE: there is intentionally NO ENTRYPOINT/CMD. HF Jobs issues `python ` # (IMAGE_SCRIPT_PATH, default /app/train.py), so an entrypoint here would conflict. # # BASE IMAGE — why a bare CUDA image and not pytorch/pytorch: # An earlier revision used `pytorch/pytorch:2.5.1-cuda12.4-cudnn9-runtime`, which ships its # own conda torch. `pip install unsloth` then installed a *second*, newer torch over the top # into the same site-packages. torch._inductor got loaded twice and `import unsloth` died with # `AssertionError: duplicate template name` before any of our code ran. # Fix: start from a CUDA image with no torch at all and let unsloth pull the torch it wants. # One torch, no collision. # # PYTHON — why the distro's stock python3 (3.10) and not python3.11: # A later revision installed `python3.11`, but Ubuntu 22.04's universe only carries # `3.11.0~rc1` (a release candidate), while `python3-pip` remains pip for 3.10. Symlinking # python->3.11 therefore ran 3.10's pip under a 3.11 interpreter, and the build failed. # Stock `python3` gives one interpreter with its matching pip. unsloth supports >=3.9,<3.15. # # GUARD — do NOT `import unsloth` at build time: # `import unsloth` probes for a GPU on import (unsloth_zoo/device_type.py calls # get_device_type() at module scope) and raises # `NotImplementedError: Unsloth cannot find any torch accelerator? You need a GPU.` # The HF Space builder is a CPU machine (SPACE_HARDWARE: cpu-basic), so such a guard can # never pass there — it made the build fail unconditionally. Import torch (CPU-safe) and # assert unsloth is *installed* via importlib.metadata, which reads metadata without # importing the package. # # `-base` (not `-runtime`/`-cudnn-runtime`): torch's wheel bundles its own CUDA libraries as # nvidia-* packages (cuDNN, cuBLAS, NCCL, cuSPARSE, … ~2.9 GB of wheels). A cudnn-runtime base # ships those libs too, so we were installing CUDA twice and the image ballooned to ~11.6 GB. # The base tag provides just the driver interface, which is all torch needs. # unsloth/bitsandbytes ship prebuilt wheels, so no nvcc (`-devel`) is required. # # Build time is substantial (torch + CUDA wheels + unsloth). The HF Space builds this image, so # SPACE_BUILD_TIMEOUT_SECONDS must be generous (>= 3600). FROM nvidia/cuda:12.4.1-base-ubuntu22.04 ENV PYTHONUNBUFFERED=1 \ PYTHONDONTWRITEBYTECODE=1 \ PYTHONPATH=/app:/app/vendor/architects \ HF_HUB_DISABLE_TELEMETRY=1 \ HF_HOME=/app/.hf_cache \ DEBIAN_FRONTEND=noninteractive WORKDIR /app # The CUDA base ships no Python, so install the distro's stock one. # # Use `python3` (Ubuntu 22.04's stable 3.10), NOT `python3.11`. Jammy's universe repo only # carries Python `3.11.0~rc1` — a release candidate — and `python3-pip` is pip for 3.10, so # symlinking python->3.11 made `python -m pip` run 3.10's pip under a 3.11 interpreter. That # mismatch broke the previous build. unsloth supports >=3.9,<3.15 and cp310 wheels exist for # torch/triton/bitsandbytes, so 3.10 needs no source builds. # # `gcc` is required at RUN TIME, not build time: unsloth's kernels are Triton, and Triton # JIT-compiles a small C shim (its `driver.c`) on the first kernel launch. Without a compiler the # job dies mid-inference with # RuntimeError: Failed to find C compiler. Please specify via CC environment variable. # (triton/runtime/build.py looks for `gcc`, then `clang`.) Triton ships its own `cuda.h` and # dlopen()s `libcuda.so.1` from the driver, so no CUDA toolkit or `-devel` base image is needed — # just a compiler. # # git is needed by some unsloth/transformers install paths. RUN apt-get update && apt-get install -y --no-install-recommends \ python3 python3-dev python3-pip git gcc \ && rm -rf /var/lib/apt/lists/* \ && ln -sf /usr/bin/python3 /usr/bin/python # One interpreter, one pip. Fail loudly here rather than debug a mismatch later. RUN python -c "import sys; assert sys.version_info[:2] == (3, 10), sys.version" \ && python -m pip --version # Assert the C compiler Triton needs, so a missing one fails this free CPU build instead of # surfacing mid-inference on a paid GPU (which is exactly how it was found). RUN command -v gcc >/dev/null || { echo "gcc missing: Triton JIT will fail at kernel launch"; exit 1; } COPY requirements.txt . # unsloth resolves and installs its own torch build here — do not pre-install torch. RUN python -m pip install --no-cache-dir --upgrade pip \ && python -m pip install --no-cache-dir -r requirements.txt \ && python -m pip install --no-cache-dir huggingface_hub # Sanity-check the install on the (CPU) builder, so a broken environment fails the free Space # build rather than a paid GPU job. `import torch` is safe without a GPU and would still catch # the torch._inductor double-install that killed the first run. unsloth is only checked for # presence — importing it here would raise "You need a GPU" on the CPU builder. RUN python -c "\ import torch, importlib.metadata as md; \ print('torch', torch.__version__, '| unsloth', md.version('unsloth'), '| cuda available:', torch.cuda.is_available())" COPY . . # Import the vendored modules we actually use. None of them import unsloth at module scope # (model_tools defers it into a function), so this is CPU-safe — and it catches library-API # drift for free. The 2024 code is resolved against current libraries, so imports break as # upstream APIs move: `from trl import DataCollatorForCompletionOnlyLM` (removed in trl 0.20.0) # failed here only after a paid GPU job had already started. Now it fails the Space build. RUN python -c "\ import arc_loader, model_tools, inference_tools, selection, arc_downloader; \ import importlib.metadata as md; \ print('vendored modules import OK | trl', md.version('trl'), '| transformers', md.version('transformers'))"