RACE-Net — amortized receiver-anchored truth inference
A 214,145-parameter axial set transformer that fuses the reports of a swarm of agents when some of them lie. Part of RoboRACE · showcase Space · code: github.com/Hisernberg/robotics

What it does
A robot (the receiver) gets, for each of T recent episodes, a score in [0, 1] from each of J agents (for example P(task succeeded) from J camera + VLM pairs). Some agents may be inverted, stuck, random, colluding or replicated. RACE-Net reads the whole unlabelled window at once and returns the log-odds of success for every episode. It assumes only one thing: the receiver's own column is honest (the anchor of RACE). Consistent liars are not discarded: their reports are effectively inverted and used.
- Input:
Pwith shape[T, J], any T and J (the model has no positional encodings; it is permutation-equivariant over episodes and peers). - Pre-processing: per-agent rank → probit within the window (label-free, invariant to any monotone miscalibration).
- Blocks (4×): attention across agents within an episode → attention across episodes within an agent → MLP (d = 64, 4 heads).
- Output: one logit per episode; decide
> 0.
Use
import sys, numpy as np
from huggingface_hub import snapshot_download
d = snapshot_download("Nabidnur/RoboRACE-Net"); sys.path.insert(0, d)
import racenet
P = np.random.rand(100, 5) # [episodes, agents] scores in [0, 1]; column `own` is the receiver
log_odds, thr = racenet.race_net(None, P, own=0, path=d)
success = log_odds > thr
Runs on CPU in a few milliseconds per window. Dependencies: torch, numpy, safetensors.
Training
Synthetic swarms only (racenet.sample_batch): heterogeneous agent skill, correlated honest errors, yes-bias, and an attack zoo with up to
J−1 liars. No benchmark data, labels or agent outputs were used. 6,000 AdamW steps, batch 24,
one-cycle LR 0.002, CPU only. Held-out synthetic accuracy at the end: 98.1%.
Evaluation (zero-shot on real VLM camera swarms)
Agents are five cameras scored by Cosmos-Reason2-2B; ground truth is the final physics state of a simulated robot arm. Each cell is one (attack, liar count, liar subset, receiver); numbers are mean accuracy in %.
| method (label-free unless noted) | A0 procedural (development) | SO-101 (test) | Franka Panda (fresh test) | SO-101, liars ≥ 50% | Panda, liars ≥ 50% |
|---|---|---|---|---|---|
| majority vote | 68.5 | 59.8 | 62.2 | 46.7 | 48.8 |
| Dawid–Skene (crowd-kit) | 69.4 | 60.1 | 65.0 | 43.5 | 45.1 |
| receiver alone | 81.4 | 71.3 | 74.5 | 71.3 | 74.5 |
| RACE online (EM) | 90.3 | 77.7 | 83.2 | 73.9 | 78.0 |
| RACE-S online (EM) | 91.2 | 82.7 | 87.0 | 79.8 | 83.6 |
| RACE-Net (this model) | 91.5 | 83.4 | 84.2 | 80.0 | 81.5 |
| labelled stacker (reference) | 84.8 | 82.6 | 84.9 | 78.1 | 80.7 |

Checkpoints and selection. Rule: highest mean accuracy of RACE-Net-V over three development sets (A0 evaluation window, SO-101 history split, Panda history split; equal weight); ties within 0.1 point go to the higher synthetic accuracy. The history splits are the benchmark's unlabelled calibration data; no label-free method ever used their labels, so they serve as development data for the two arms, and the evaluation splits stay test sets. Test columns were computed after the choice and are shown for transparency. On the same development sets RACE-S online scores 84.9 and RACE online 83.8.
| checkpoint | generator · type | steps | params | synthetic (v3) | dev: RACE-Net-V (selects) | dev: RACE-Pool | test SO-101 / Panda: RACE-Net-V | test: RACE-Pool |
|---|---|---|---|---|---|---|---|---|
| v3 refine · local · 6k (released) | v3 · EM-refinement | 6,000 | 214,145 | 97.8 | 84.0 | 85.6 | 83.4 / 84.2 | 82.0 / 85.9 |
| v3 large · Kaggle · 9k | v3 · plain | 9,000 | 589,729 | 97.9 | 83.8 | 85.6 | 83.6 / 84.1 | 82.2 / 85.9 |
| v3 · Kaggle · 15k · seed 1 | v3 · plain | 15,000 | 213,889 | 97.9 | 83.7 | 85.5 | 83.6 / 83.6 | 82.0 / 85.9 |
| v2 · local · 7k | v2 · plain | 7,000 | 213,889 | 96.7 | 83.4 | 85.6 | 82.6 / 83.5 | 81.9 / 86.1 |
| v3 · Kaggle · 15k · seed 0 | v3 · plain | 15,000 | 213,889 | 97.9 | 83.3 | 85.6 | 83.8 / 84.0 | 81.9 / 86.1 |
| v1 · local · 6k | v1 · plain | 6,000 | 213,889 | 97.6 | 82.2 | 85.0 | 81.6 / 81.8 | 81.7 / 85.5 |
| v1 · Kaggle · 15k | v1 · plain | 15,000 | 213,889 | 97.8 | 82.0 | 85.0 | 80.9 / 82.3 | 81.4 / 85.4 |
Checkpoints marked Kaggle were trained in Kaggle CPU notebooks (4 vCPU; logs in results/kaggle/). All checkpoints are in models/racenet/candidates/.
Scaling study

Scaling study (scripts/racenet_analysis.py; an independent synthetic generator, not the training distribution; 20 seeds per point):
- Liar majority. With 3 of 5 agents lying: RACE-Net-V 80.1, RACE-S online 77.6, RACE online 76.8, majority 45.5. With 15 of 16 lying: 86.7 / 83.4 / 78.7 / 21.9.
- Short windows. With only 8 unlabelled episodes: RACE-Net-V 71.7, RACE-S online 72.8, receiver alone 70.0; no label-free method beats the receiver before about 32 episodes.
- Large swarms. At 24 agents (larger than any training swarm): RACE-Net-V 93.8, RACE online 93.4, RACE-S online 93.2.
- Weak receiver (open weakness). If the receiver is only 55% accurate and nobody lies, RACE-Net-V reaches 65.8 and RACE-S online 54.4, while majority reaches 81.8 and discrete RACE 82.4. Score-level anchoring trusts a weak receiver too much; from a 70% receiver upwards the gap is gone.
- Against the first release. Weak receivers: 65.8 at 55% and 87.3 at 70% receiver accuracy (first release 57.1 and 77.4). 24-agent swarms: 93.8 (first release 89.1).
- RACE-Pool on the same sweeps: 79.2 with 3 of 5 lying, 94.9 at 24 agents, 68.2 with a 55% receiver.
Limitations
- It matches the EM-based methods but does not beat the best of them (RACE-S online) on the real camera swarms (see the table).
- Trained on a hand-written simulator of swarms. Where real agent errors differ from it, the EM-based RACE-S can be better.
- The receiver must be honest and better than chance; with a very weak receiver and a liar majority no label-free method can identify the truth.
- Fits on the current window (transductive); a sleeper that changes behaviour inside the window is treated as one mixed channel.
- Binary truth only. Evaluated in simulation with one VLM behind all cameras.
Files
model.safetensors · config.json · training_log.json · selection.json + candidates/ (all three checkpoints) · racenet.py (model, synthetic
generators, inference) · train_racenet.py · select_racenet.py · racenet_analysis.py + analysis.json (scaling study)
- Downloads last month
- 24
Dataset used to train Nabidnur/RoboRACE-Net
Spaces using Nabidnur/RoboRACE-Net 2
Evaluation results
- mean accuracy over all attack cells on RoboRACE-Bench Track A SO-101self-reported83.400
- mean accuracy over all attack cells on RoboRACE-Bench Track A-Pandaself-reported84.200