Instructions to use Renesas/GR00T with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Renesas/GR00T with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Renesas/GR00T:F16 # Run inference directly in the terminal: llama cli -hf Renesas/GR00T:F16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Renesas/GR00T:F16 # Run inference directly in the terminal: llama cli -hf Renesas/GR00T:F16
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Renesas/GR00T:F16 # Run inference directly in the terminal: ./build/bin/llama cli -hf Renesas/GR00T:F16
Use Docker
docker model run hf.co/Renesas/GR00T:F16
- Llama new
- LM Studio
- Jan
- Ollama
How to use Renesas/GR00T with Ollama:
ollama run hf.co/Renesas/GR00T:F16
- Unsloth Desktop
- Docker Model Runner
How to use Renesas/GR00T with Docker Model Runner:
docker model run hf.co/Renesas/GR00T:F16
- Lemonade
How to use Renesas/GR00T with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Renesas/GR00T:F16
Run and chat with the model
lemonade run user.GR00T-F16
List all available models
lemonade list
- Atomic Chat
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Renesas/GR00T:F16# Run inference directly in the terminal:
llama cli -hf Renesas/GR00T:F16Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf Renesas/GR00T:F16# Run inference directly in the terminal:
./build/bin/llama cli -hf Renesas/GR00T:F16Use Docker
docker model run hf.co/Renesas/GR00T:F16GR00T β Renesas X5H
Source: NVIDIA's official Isaac-GR00T repository
Introduction
GR00T is NVIDIA's generalist foundation model for humanoid robots β a vision-language-action (VLA) model that maps camera input and language instructions to robot actions. Renesas is preparing an optimized deployment of this model for the R-Car Gen5 platform.
GR00T-N1.7-3B on the Renesas X5H β v0.2.0
Runner
groot-n1.7-3b-runner runs GR00T-N1.7-3B on NPU1 of the X5H board. It takes two camera
images, an instruction and the robot state, and returns an action chunk: 40 time steps of the
robot's next actions. The vision encoder, the language model (W4A16: 4-bit weights, 16-bit
activations) and the action head all run on the NPU. The runner is prebuilt for the board
(aarch64 Linux) and needs no toolchain.
Benchmark
Time per observation, by stage
| Stage | NPU time |
|---|---|
| Vision encoder | 75.4 ms |
| LLM backbone | 52.1 ms |
| Action head | 196.8 ms |
| Total | 324.3 ms |
Vision encoder, by graph
| Graph | Calls | ms per call | ms |
|---|---|---|---|
| Patch embedding | 1 | 1.41 | 1.4 |
| Transformer layer | 24 | 2.758 | 66.2 |
| DeepStack merger (layers 5, 11, 17) | 3 | 1.885 | 5.7 |
| Patch merger | 1 | 1.74 | 1.7 |
| Read back | 1 | 0.41 | 0.4 |
| Total | 75.4 |
LLM backbone, by graph
| Graph | Calls | ms per call | ms |
|---|---|---|---|
| Transformer layer, W4A16 | 16 | 3.079 | 49.3 |
| DeepStack inject (layers 0, 1, 2) | 3 | 0.568 | 1.7 |
| Input preparation | 1 | 0.79 | 0.8 |
| Final norm | 1 | 0.17 | 0.2 |
| Read back | 1 | 0.15 | 0.1 |
| Total | 52.1 |
Action head, by graph
| Graph | Calls | ms per call | ms |
|---|---|---|---|
| DiT block (16 cross-attention + 16 self-attention) Γ 4 steps | 128 | 1.319 | 168.8 |
| VL adapter (self-attention over the 156 VL tokens) | 1 | 16.28 | 16.3 |
| Cross-attention K/V of the 16 cross blocks | 1 | 7.01 | 7.0 |
| State and action encoding | 4 | 0.804 | 3.2 |
| Output projection and action decoder | 4 | 0.333 | 1.3 |
| Cross-attention masks, read back, Euler update | β | β | 0.1 |
| Total | 196.8 |
Contents of this release
Contents
groot-n1.7-3b-runner-0.2.0-Linux.sh is a self-extracting installer. It holds the runner, the
NPU graphs, the NPU firmware, the NPU setup scripts and a LIBERO test episode, already built for
the board. Nothing is compiled on the board.
groot-n1.7-3b-runner/
βββ groot-vla-q-runner # the runner; the vision encoder, action head and backbone
β # graphs are compiled into it
βββ run_demo.sh # runs the runner on the test episode
βββ setup_npu.sh # NPU bring-up
βββ scripts/ # npuN_power_on.sh, npuN_run.sh
βββ firmwares/npu0, npu1/ # NPU firmware (14 ELFs per NPU)
βββ groot-n1.7-3b-llm-q/ # the backbone's 16 int4 coefficient sections (~440 MB)
βββ test_data/libero_10_ep0/ # LIBERO test episode: one directory per step, and noise.bin
What you need
| Board | R-Car X5H (b001, dual NPX48K), root access |
| Free disk space | ~0.5 GB for the package and ~0.5 GB once it is installed, plus 6.3 GB for the GGUF files |
| Model | the three GGUF files of nvidia/GR00T-N1.7-LIBERO, suite libero_10, see below |
| GGUF file | size | what it holds |
|---|---|---|
mmproj-GR00T-N1.7-LIBERO-vision-F16.gguf |
819 394 752 B (0.8 GB) | vision encoder weights |
groot-n1.7-libero-llm-qwen3vl-f16.gguf |
2 239 173 984 B (2.2 GB) | backbone vocabulary and token embedding table |
groot-n1.7-libero-libero_10-action-head-f16.gguf |
3 242 914 880 B (3.2 GB) | action head weights |
The int4 weights of the backbone layers are already in the package, in
groot-n1.7-3b-llm-q/.
Download the model files
The three GGUF files are the weights of the nvidia/GR00T-N1.7-LIBERO checkpoint, suite
libero_10, converted to GGUF. Download them from https://huggingface.co/Renesas/GR00T/tree/main
on any machine, check them, and copy them to the board:
sha256sum mmproj-GR00T-N1.7-LIBERO-vision-F16.gguf groot-n1.7-libero-llm-qwen3vl-f16.gguf \
groot-n1.7-libero-libero_10-action-head-f16.gguf
# 152440b75b3327b81886fd7b00922237cb358b46df81af4859b24144a959a9b1 mmproj-GR00T-N1.7-LIBERO-vision-F16.gguf
# f41b3a85ee4e0bd632d5270cb57688cabba7139e465c3e9fcd786b0187c9bf52 groot-n1.7-libero-llm-qwen3vl-f16.gguf
# e5fa4f6921f7f859721921b4118a869a46fdc5523cda19328e30bdc243414f25 groot-n1.7-libero-libero_10-action-head-f16.gguf
# (the files this release was measured with)
Install
bash ./groot-n1.7-3b-runner-0.2.0-Linux.sh --prefix=./ --exclude-subdir --skip-license
cd groot-n1.7-3b-runner
mkdir gguf # copy the three GGUF files here
Run
Bring up the NPU once after each boot (NPU0, the default):
./setup_npu.sh
To run on NPU1 instead, pass --npu-idx 1 to setup_npu.sh, run_demo.sh and the runner.
The test episode, first step only:
./run_demo.sh
Every step of the test episode:
./run_demo.sh --all
run_demo.sh --help lists its options: the GGUF directory, the NPU (--npu-idx, default 0) and
the output prefix.
Your own observation, with the runner directly:
./groot-vla-q-runner --npu-idx 0 -e 2 \
-v gguf/mmproj-GR00T-N1.7-LIBERO-vision-F16.gguf \
-m gguf/groot-n1.7-libero-llm-qwen3vl-f16.gguf \
-a gguf/groot-n1.7-libero-libero_10-action-head-f16.gguf \
-d <dir with image_0.png and image_1.png> \
-i "<instruction>" -s <normalized state, comma-separated>
image_0.png is the agent-view camera and image_1.png the wrist camera. -e 2 is the
libero_sim embodiment of the LIBERO checkpoint.
Output
For each step the runner writes, under the -o prefix and the step's name:
| file | |
|---|---|
<prefix>_<step>_actions.bin |
the action chunk: 40 timesteps Γ 132 action dims, fp16, action dim fastest |
<prefix>_<step>_hidden.bin |
the backbone's output, 192 tokens Γ 2048, fp16 |
<prefix>_<step>_pred_velocity_step0..3.bin |
the velocity of each flow-matching step, 40 Γ 132, fp16 |
Only the first 7 action dims are LIBERO's (x, y, z, roll, pitch, yaw, gripper); the other 125
are padding. The chunk is in normalized action space: the robot runtime denormalizes it with the
statistics of the checkpoint (statistics.json, libero_sim), and LIBERO executes the first 16
timesteps.
The runner prints the time of each stage and of each NPU graph ([timing]), and the wall time per
stage ([Metrics]).
Options
./groot-vla-q-runner -d <base_dir> -i <instruction> -s <state> [common options]
./groot-vla-q-runner --episode-dir <dir> [--episode ID] [common options]
One observation, from loose image files:
-d base directory holding the input images
-s normalized robot state: comma-separated real DoF, at most 132
--images comma-separated image file names (2 expected,
default image_0.png,image_1.png)
Every step of one extracted episode, in order:
--episode-dir D a directory of step subdirectories, each holding that
step's 2 images, its state.csv and its instruction.txt
--episode ID which episode in D to run (ep0_f016 -> "ep0", or "0")
--step-start N skip the first N steps of the episode (default 0)
--step-stride N run every Nth step (default 1)
--max-steps N stop after N steps (default: to the end of the episode)
Common:
-i the instruction to condition on; with --episode it
overrides the episode's own instruction.txt
-m backbone GGUF path
-v mmproj (vision encoder) GGUF path
-a action-head GGUF path
-e embodiment id, in [0, 32) (default 0); 2 for LIBERO
-o output dump prefix (default: groot_vla)
--npu-idx N NPU device index (0 or 1, default 0)
--seed N seed for the starting noise (default 0)
--noise FILE take the starting noise from a file instead: raw fp16,
40 x 132 = 5280 values (10560 bytes), action dim fastest;
over an episode every step starts from this same noise
--llm-coef-dir DIR
where the 16 backbone coefficient sections are, relative
to the working directory (default: groot-n1.7-3b-llm-q)
- Downloads last month
- 38
16-bit
Model tree for Renesas/GR00T
Base model
nvidia/GR00T-N1.7-3B
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf Renesas/GR00T:F16# Run inference directly in the terminal: llama cli -hf Renesas/GR00T:F16