GR00T – Renesas X5H

NVIDIA Isaac GR00T

Isaac GR00T controlling a Unitree G1 humanoid robot

Source: NVIDIA's official Isaac-GR00T repository

Introduction

GR00T is NVIDIA's generalist foundation model for humanoid robots β€” a vision-language-action (VLA) model that maps camera input and language instructions to robot actions. Renesas is preparing an optimized deployment of this model for the R-Car Gen5 platform.

GR00T-N1.7-3B on the Renesas X5H β€” v0.2.0

Runner

groot-n1.7-3b-runner runs GR00T-N1.7-3B on NPU1 of the X5H board. It takes two camera images, an instruction and the robot state, and returns an action chunk: 40 time steps of the robot's next actions. The vision encoder, the language model (W4A16: 4-bit weights, 16-bit activations) and the action head all run on the NPU. The runner is prebuilt for the board (aarch64 Linux) and needs no toolchain.

Benchmark

Time per observation, by stage

Stage NPU time
Vision encoder 75.4 ms
LLM backbone 52.1 ms
Action head 196.8 ms
Total 324.3 ms

Vision encoder, by graph

Graph Calls ms per call ms
Patch embedding 1 1.41 1.4
Transformer layer 24 2.758 66.2
DeepStack merger (layers 5, 11, 17) 3 1.885 5.7
Patch merger 1 1.74 1.7
Read back 1 0.41 0.4
Total 75.4

LLM backbone, by graph

Graph Calls ms per call ms
Transformer layer, W4A16 16 3.079 49.3
DeepStack inject (layers 0, 1, 2) 3 0.568 1.7
Input preparation 1 0.79 0.8
Final norm 1 0.17 0.2
Read back 1 0.15 0.1
Total 52.1

Action head, by graph

Graph Calls ms per call ms
DiT block (16 cross-attention + 16 self-attention) Γ— 4 steps 128 1.319 168.8
VL adapter (self-attention over the 156 VL tokens) 1 16.28 16.3
Cross-attention K/V of the 16 cross blocks 1 7.01 7.0
State and action encoding 4 0.804 3.2
Output projection and action decoder 4 0.333 1.3
Cross-attention masks, read back, Euler update β€” β€” 0.1
Total 196.8

Contents of this release

Contents

groot-n1.7-3b-runner-0.2.0-Linux.sh is a self-extracting installer. It holds the runner, the NPU graphs, the NPU firmware, the NPU setup scripts and a LIBERO test episode, already built for the board. Nothing is compiled on the board.

groot-n1.7-3b-runner/
β”œβ”€β”€ groot-vla-q-runner         # the runner; the vision encoder, action head and backbone
β”‚                              # graphs are compiled into it
β”œβ”€β”€ run_demo.sh                # runs the runner on the test episode
β”œβ”€β”€ setup_npu.sh               # NPU bring-up
β”œβ”€β”€ scripts/                   # npuN_power_on.sh, npuN_run.sh
β”œβ”€β”€ firmwares/npu0, npu1/      # NPU firmware (14 ELFs per NPU)
β”œβ”€β”€ groot-n1.7-3b-llm-q/       # the backbone's 16 int4 coefficient sections (~440 MB)
└── test_data/libero_10_ep0/   # LIBERO test episode: one directory per step, and noise.bin

What you need

Board R-Car X5H (b001, dual NPX48K), root access
Free disk space ~0.5 GB for the package and ~0.5 GB once it is installed, plus 6.3 GB for the GGUF files
Model the three GGUF files of nvidia/GR00T-N1.7-LIBERO, suite libero_10, see below
GGUF file size what it holds
mmproj-GR00T-N1.7-LIBERO-vision-F16.gguf 819 394 752 B (0.8 GB) vision encoder weights
groot-n1.7-libero-llm-qwen3vl-f16.gguf 2 239 173 984 B (2.2 GB) backbone vocabulary and token embedding table
groot-n1.7-libero-libero_10-action-head-f16.gguf 3 242 914 880 B (3.2 GB) action head weights

The int4 weights of the backbone layers are already in the package, in groot-n1.7-3b-llm-q/.

Download the model files

The three GGUF files are the weights of the nvidia/GR00T-N1.7-LIBERO checkpoint, suite libero_10, converted to GGUF. Download them from https://huggingface.co/Renesas/GR00T/tree/main on any machine, check them, and copy them to the board:

sha256sum mmproj-GR00T-N1.7-LIBERO-vision-F16.gguf groot-n1.7-libero-llm-qwen3vl-f16.gguf \
    groot-n1.7-libero-libero_10-action-head-f16.gguf
# 152440b75b3327b81886fd7b00922237cb358b46df81af4859b24144a959a9b1  mmproj-GR00T-N1.7-LIBERO-vision-F16.gguf
# f41b3a85ee4e0bd632d5270cb57688cabba7139e465c3e9fcd786b0187c9bf52  groot-n1.7-libero-llm-qwen3vl-f16.gguf
# e5fa4f6921f7f859721921b4118a869a46fdc5523cda19328e30bdc243414f25  groot-n1.7-libero-libero_10-action-head-f16.gguf
# (the files this release was measured with)

Install

bash ./groot-n1.7-3b-runner-0.2.0-Linux.sh --prefix=./ --exclude-subdir --skip-license
cd groot-n1.7-3b-runner
mkdir gguf    # copy the three GGUF files here

Run

Bring up the NPU once after each boot (NPU0, the default):

./setup_npu.sh

To run on NPU1 instead, pass --npu-idx 1 to setup_npu.sh, run_demo.sh and the runner.

The test episode, first step only:

./run_demo.sh

Every step of the test episode:

./run_demo.sh --all

run_demo.sh --help lists its options: the GGUF directory, the NPU (--npu-idx, default 0) and the output prefix.

Your own observation, with the runner directly:

./groot-vla-q-runner --npu-idx 0 -e 2 \
    -v gguf/mmproj-GR00T-N1.7-LIBERO-vision-F16.gguf \
    -m gguf/groot-n1.7-libero-llm-qwen3vl-f16.gguf \
    -a gguf/groot-n1.7-libero-libero_10-action-head-f16.gguf \
    -d <dir with image_0.png and image_1.png> \
    -i "<instruction>" -s <normalized state, comma-separated>

image_0.png is the agent-view camera and image_1.png the wrist camera. -e 2 is the libero_sim embodiment of the LIBERO checkpoint.

Output

For each step the runner writes, under the -o prefix and the step's name:

file
<prefix>_<step>_actions.bin the action chunk: 40 timesteps Γ— 132 action dims, fp16, action dim fastest
<prefix>_<step>_hidden.bin the backbone's output, 192 tokens Γ— 2048, fp16
<prefix>_<step>_pred_velocity_step0..3.bin the velocity of each flow-matching step, 40 Γ— 132, fp16

Only the first 7 action dims are LIBERO's (x, y, z, roll, pitch, yaw, gripper); the other 125 are padding. The chunk is in normalized action space: the robot runtime denormalizes it with the statistics of the checkpoint (statistics.json, libero_sim), and LIBERO executes the first 16 timesteps.

The runner prints the time of each stage and of each NPU graph ([timing]), and the wall time per stage ([Metrics]).

Options

./groot-vla-q-runner -d <base_dir> -i <instruction> -s <state>   [common options]
./groot-vla-q-runner --episode-dir <dir> [--episode ID]          [common options]

One observation, from loose image files:
-d              base directory holding the input images
-s              normalized robot state: comma-separated real DoF, at most 132
--images        comma-separated image file names (2 expected,
                default image_0.png,image_1.png)

Every step of one extracted episode, in order:
--episode-dir D a directory of step subdirectories, each holding that
                step's 2 images, its state.csv and its instruction.txt
--episode ID    which episode in D to run (ep0_f016 -> "ep0", or "0")
--step-start N  skip the first N steps of the episode (default 0)
--step-stride N run every Nth step (default 1)
--max-steps N   stop after N steps (default: to the end of the episode)

Common:
-i              the instruction to condition on; with --episode it
                overrides the episode's own instruction.txt
-m              backbone GGUF path
-v              mmproj (vision encoder) GGUF path
-a              action-head GGUF path
-e              embodiment id, in [0, 32) (default 0); 2 for LIBERO
-o              output dump prefix (default: groot_vla)
--npu-idx N     NPU device index (0 or 1, default 0)
--seed N        seed for the starting noise (default 0)
--noise FILE    take the starting noise from a file instead: raw fp16,
                40 x 132 = 5280 values (10560 bytes), action dim fastest;
                over an episode every step starts from this same noise
--llm-coef-dir DIR
                where the 16 backbone coefficient sections are, relative
                to the working directory (default: groot-n1.7-3b-llm-q)
Downloads last month
38
GGUF
Model size
2B params
Architecture
groot-action-head
Hardware compatibility
Log In to add your hardware

16-bit

Video Preview
loading

Model tree for Renesas/GR00T

Quantized
(5)
this model