SpeedNet v5: corrected 790 m indoor supervision, multi-seed selection, per-mode evaluation, four GPS-verified demo videos

.gitattributes

@@ -35,3 +35,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 assets/example_atv.png filter=lfs diff=lfs merge=lfs -text
 assets/demo_bestlap.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/demo_atv.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/demo_car_motorway.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/demo_car_rural.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/demo_kart.mp4 filter=lfs diff=lfs merge=lfs -text

README.md

@@ -18,7 +18,7 @@ tags:
   - karting
 ---
 
-# SpeedNet v4 — ego-speed estimation from onboard video
+# SpeedNet v5 — ego-speed estimation from onboard video
 
 SpeedNet estimates **vehicle speed directly from onboard/POV video** — no GPS,
 no CAN bus, no sensors. Point it at a dashcam clip, a go-kart helmet-cam
@@ -29,11 +29,32 @@ footage that has none?* — e.g. indoor karting, where GPS does not work.
 
 ## Demo
 
-Fastest lap of an indoor go-kart race (1000 m track), with the speed HUD
-generated by this project — video-model level, IMU-driven dynamics, and the
-official lap time anchoring the scale:
+Four unedited minutes across the domains the model has to cope with. The
+cyan trace is SpeedNet reading the video; the dashed white trace is GPS,
+shown wherever ground truth exists, with a running MAE. Nothing is
+cherry-picked — the two car clips are included precisely because open-road
+driving is the model's weakest regime (see Limitations).
 
-<video src="https://huggingface.co/x2q/speednet/resolve/main/assets/demo_bestlap.mp4" controls muted playsinline width="100%"></video>
+**ATV, forest trail** — 9–61 km/h, unseen recording. MAE **3.4 km/h**.
+
+<video src="https://huggingface.co/x2q/speednet/resolve/main/assets/demo_atv.mp4" controls muted playsinline width="100%"></video>
+
+**Car, country roads** — 19–64 km/h. MAE **11.1 km/h**.
+
+<video src="https://huggingface.co/x2q/speednet/resolve/main/assets/demo_car_rural.mp4" controls muted playsinline width="100%"></video>
+
+**Car, motorway** — 21–108 km/h. MAE **10.7 km/h**. High speed is a hard
+case: at 110 km/h the flow field saturates and distant structure carries
+most of the signal. See *Limitations*.
+
+<video src="https://huggingface.co/x2q/speednet/resolve/main/assets/demo_car_motorway.mp4" controls muted playsinline width="100%"></video>
+
+**Indoor kart** — no GPS exists here at all; this is the case the project was
+built for. No ground-truth trace, because there is nothing to compare
+against frame by frame; validation is against the venue's timing system
+(see *Evaluation*).
+
+<video src="https://huggingface.co/x2q/speednet/resolve/main/assets/demo_kart.mp4" controls muted playsinline width="100%"></video>
 
 ## Motivation
 
@@ -67,6 +88,16 @@ card because they will bite anyone building something similar:
    fusion described below: measured longitudinal g-force drives the
    acceleration profile, centripetal physics anchors cornering speed, and
    track length locks the scale.
+6. **The published track length is not the driven distance.** Our indoor
+   supervision used `mean speed = track length / lap time` with the venue's
+   advertised 1000 m. That figure is the **centreline**. A racing line cuts
+   every corner by up to half the track width, so on a 13 m wide circuit
+   with 33.1 rad of total turning it saves `6.5 m x 33.1 = 215 m` — the
+   driven lap is ~790 m, and every label was 27% too fast. If you use
+   lap-time weak supervision, measure the turning and subtract; do not trust
+   the marketing number. Corner count is a good sanity check that you have
+   the right layout (we measured 14, matching the venue's spec) while being
+   completely independent of scale.
 
 ```
 video ──► optical flow (motion field) ──┐
@@ -133,27 +164,28 @@ inference and 1-second median smoothing.
 
 ## Evaluation
 
-Held-out recordings (never trained on; the ATV test is from a different day
-than all ATV training data). RMSE in m/s, windowed protocol:
-
-| Test domain | RMSE (m/s) | Notes |
-|---|---|---|
-| ATV, forest trail (17 min, unseen day) | **0.65** | |
-| Car, highway 100–135 km/h (DK) | 5.60 | hardest regime |
-| Car, urban (DK) | 1.68 | |
-| Car, rural/urban Denmark (DJI, unseen recording) | 2.05 (zero-shot) | |
-| Car, EU driving-school fleet (L2D test split) | 2.70 | |
-| Car, US highway (comma2k19 test route) | 2.93 | |
-| Indoor go-kart, official lap times (holdout laps) | 5.2 km/h MAE (long-context mode) | 3.4 km/h with lap-aligned windows; vs. timing system, 1000 m track |
-
-The indoor go-kart figures are validated against the venue's race timing
-system: mean predicted speed per lap vs. `1000 m / official lap time`, on
-laps excluded from training. GPS-domain rows use short mode; the indoor row
-uses long-context mode (5.2 km/h), improving to 3.4 km/h when lap boundaries
-are known and inference windows are aligned to laps.
-
-![Example: ATV test recording, prediction vs GPS](assets/example_atv.png)
-![Example: indoor go-kart lap vs official timing](assets/example_indoor.png)
+Held-out recordings, never trained on. Two inference modes exist (sequence
+length acts as a scale cue for this architecture); both are reported —
+pick per the recommendation column. RMSE in m/s against GPS:
+
+| Test clip | short windows | streaming | recommended |
+|---|---|---|---|
+| ATV, forest trail (unseen day) | **0.99** | 1.60 | short |
+| Car, coastal Denmark (zero-shot recording) | 1.57 | **1.39** | streaming |
+| Car, low-speed urban | 4.44 | **4.25** | streaming |
+| Car, urban/rural | 7.82 | **6.00** | streaming |
+| Car, motorway 100–135 km/h | 6.50 | **5.87** | streaming |
+| Indoor go-kart (lap means vs official timing) | — | **3.70 km/h MAE** (bias −2.84) | streaming |
+
+The indoor row compares mean predicted speed per lap against
+`driven lap length / official lap time` on laps excluded from training,
+using the corrected 790 m driven-lap length (see challenge 6 above —
+the venue's advertised 1000 m is the centreline).
+
+**Scale caveat:** all speeds carry a ±5% scale uncertainty indoors. The
+790 m driven-lap figure is bracketed by geometry (a racing line cannot be
+shorter than ~785 m) and by flow evidence (which integrates to ~753 m);
+the residual −2.8 km/h indoor bias reflects exactly this uncertainty.
 
 ## Training data
 
@@ -161,7 +193,7 @@ are known and inference windows are aligned to laps.
 
 | Domain | Source | Speed signal |
 |---|---|---|
-| Indoor go-kart (2 races) | helmet cam, 1000 m indoor track | official lap times (weak supervision: mean speed per lap) |
+| Indoor go-kart (2 races) | helmet cam, indoor kart circuit | official lap times (weak supervision: mean speed per lap over the 790 m driven racing line) |
 | ATV / forest | action cam + GPS remote | 10 Hz GPS |
 | Mini-loader / grass | action cam + GPS remote | 10 Hz GPS |
 | Car / Denmark | action cam + GPS, dashcam | 10 Hz + 1 Hz GPS |
@@ -176,8 +208,12 @@ photometric jitter on the context frame; per-recording train/val/test splits.
 ## Limitations
 
 - **Protocol sensitivity**: sequence length is a scale cue (an artifact of
-  two-length training); use the mode matching your footage as described
-  above. Removing this via length-randomized training is planned.
+  two-length training); use the recommended mode per the evaluation table
+  (short for trail/off-road, streaming for cars and closed courses).
+  Length-randomized training removes this — validated on a companion
+  track-specialized model — and is planned for the next release.
+- **Indoor absolute scale is ±5%**: supervision is lap time x driven lap
+  length, and the driven racing line length is derived, not surveyed.
 - **Compressed dynamics within a lap/segment**: like most regression models,
   predictions are pulled toward the mean — cornering speeds read slightly
   high, straight-line peaks slightly low. If your footage has IMU data, see

assets/demo_atv.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:756a48eebfcf90fd72b34812c76fc5f1cea7f551f5883ffef7cfc235af882126
+size 13116768

assets/demo_car_motorway.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:056566b4850eb9484579e377a1614f46a211274865a95e2c2537daaea7080095
+size 2446651

assets/demo_car_rural.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:daaae976e706f37eccb3507f887dd8f82f31008f4a477ba4c026cf375dbc9eed
+size 5543994

assets/demo_kart.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:982f6b0ccc36b57796187d08c4d203fd22bac05c67eecaa782fe8b3b4535b7ea
+size 6110305

example.py

@@ -18,7 +18,7 @@ video = sys.argv[1]
 device = "cuda" if torch.cuda.is_available() else "cpu"
 
 model = SpeedNet()
-model.load_state_dict(torch.load("speednet_v4.pt", map_location=device))
+model.load_state_dict(torch.load("speednet_v5.pt", map_location=device))
 
 r = predict_video(model, video, device=device)
 

modeling_speednet.py

@@ -23,7 +23,7 @@ Typical usage
     import torch
 
     model = SpeedNet()
-    model.load_state_dict(torch.load("speednet_v4.pt", map_location="cuda"))
+    model.load_state_dict(torch.load("speednet_v5.pt", map_location="cuda"))
     result = predict_video(model, "my_onboard_clip.mp4", device="cuda")
     # result["t"]        -> timestamps in seconds
     # result["speed_mps"]-> speed in m/s (multiply by 3.6 for km/h)

speednet_v5.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8cad1e37d047ec4213c749494d7aee2957f706e89064d7d492b2cd6aa95981dc
+size 3766617