Spaces:
Running
Bare-metal C99 UEFI runtime running Llama 3.2 1B on real x86_64 hardware (zero OS, AVX2 + SMP, live web search)
Hi everyone,
For the past ~2 years, I have been building and iterating layer by layer on a freestanding C99 bare-metal x86_64 UEFI runtime (OO / LLAMAX) that boots directly from a USB stick on real PC hardware—with zero OS kernel (no Linux, no Windows, no glibc/malloc) and zero GPU/CUDA.
What started nearly two years ago as low-level UEFI Ring-0 experiments (framebuffer graphics, physical page allocation, and raw hardware initialization) gradually evolved—through countless iterations, debugging sessions on real silicon, and invaluable low-level systems advice from mentors and the pure-C / llm.c communities—into a self-contained inference runtime and minimal Ring-0 environment.
What makes it stand out on real hardware is the raw response speed and low latency without any OS scheduler or background noise:
- Direct Physical Memory & Multi-Core SMP (AVX2 + FMA3): Weights (Llama 3.2 1B Instruct, Q8_0 / packed Q4_0) are mapped directly into contiguous UEFI physical pages. CPU 0 handles UI/IRQs/network on a 1000 Hz Local APIC timer while discovered AP helper cores split matrix rows across 64-byte aligned lockless barriers (32 int8 elements per AVX2 cycle) with an 8-token micro-batch prefill.
- Zero-Waste KV Caching & Speculative Decoding: Multi-turn LCP prefix diffing skips recomputing prior conversation turns in RAM, combined with lossless n-gram speculative verification (up to 5 tokens per forward pass) and real-time RDTSC tok/s telemetry rendered directly on screen.
- Self-Contained Ring-0 Hardware & Web Loop: Includes custom bare-metal drivers for UEFI GOP framebuffer, external USB, audio, and Intel/Ethernet networking—allowing the local Llama model to autonomously query the live web, retrieve search results, and persist memory across cold USB reboots.
Live bootable USB image, real-hardware demo video, and QEMU/OVMF instructions:
https://huggingface.co/LLM-Baremetal/OO
Would love technical feedback from anyone pushing pure-C inference on bare metal!