riita10069 commited on
Commit
c77e366
·
verified ·
1 Parent(s): 9f03a9f

Publish four AutoE2E trajectory checkpoints

Browse files

Publish MLflow model versions 64-67 with immutable checkpoint hashes, per-model metadata, and KITScenes external overlay replay reports.

README.md CHANGED
@@ -1,3 +1,57 @@
1
  ---
2
- license: apache-2.0
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: other
3
+ library_name: pytorch
4
+ tags:
5
+ - autonomous-driving
6
+ - bevformer
7
+ - trajectory-planning
8
+ - nuplan
9
+ - kitscenes
10
  ---
11
+
12
+ # AutoE2E checkpoints
13
+
14
+ This repository contains four trajectory-planning checkpoints from the AutoE2E project. These are the exact PyTorch training checkpoints registered in MLflow, including model, optimizer, scheduler, configuration, metrics, and training state.
15
+
16
+ ## Checkpoints
17
+
18
+ | Path | Training stage | Epoch | MLflow version | SHA-256 |
19
+ |---|---|---:|---:|---|
20
+ | `models/nuplan-epoch-4/checkpoint.pt` | nuPlan trajectory and route | 4 | 64 | `ed00e072471aae51b4cf48fa54c3fd9c03c61e0fd6e2978829cd9068dfd6da68` |
21
+ | `models/nuplan-epoch-5/checkpoint.pt` | nuPlan trajectory and route | 5 | 65 | `ca8b43d7a777d6fd9195bb253d1452f3cd645f7aab3bd0e36b1f74ffb31df29b` |
22
+ | `models/kitscenes-epoch-5/checkpoint.pt` | KITScenes fine-tuning | 5 | 66 | `120a21639d9767d512eea645be0a83b01476c9e0366a8b957ac51b7ead1bd762` |
23
+ | `models/kitscenes-epoch-7/checkpoint.pt` | KITScenes fine-tuning | 7 | 67 | `a1e6b1621018740485b3e1e43dda23713d913667eb39a65b77fcbf895dcbcefd` |
24
+
25
+ All checkpoints use the `bevformer_v2_t8_split_navigation_v5` configuration. The camera BEV starts from the official BEVFormer V2 R50 T8 initialization and is frozen during trajectory stages. World Model and Reasoning branches are disabled.
26
+
27
+ ## External overlay replay
28
+
29
+ Each value is ADE / FDE in meters. KITScenes Val v3.5 contains 11,035 samples with Camera + HD Map + oracle post-hoc Route. KITScenes Test v1.0 contains 23,690 camera-only samples from a different scene population.
30
+
31
+ | Checkpoint | Val 1 s | Val 3 s | Val 5 s | Test 1 s | Test 3 s | Test 5 s |
32
+ |---|---:|---:|---:|---:|---:|---:|
33
+ | nuPlan Epoch 4 | 0.2030 / 0.4066 | 0.9503 / 2.4508 | 2.2919 / 6.2272 | 0.1916 / 0.3899 | 0.9181 / 2.3895 | 2.2840 / 6.4266 |
34
+ | nuPlan Epoch 5 | 0.1951 / 0.3779 | 0.9147 / 2.4226 | 2.2951 / 6.3885 | 0.1602 / 0.3134 | 0.7924 / 2.1621 | 2.1120 / 6.1751 |
35
+ | KITScenes Epoch 5 | 0.1472 / 0.2850 | 0.7449 / 2.0196 | 1.9405 / 5.5645 | 0.1463 / 0.3107 | 0.8263 / 2.2199 | 2.1042 / 5.9422 |
36
+ | KITScenes Epoch 7 | 0.1347 / 0.2831 | 0.8035 / 2.2127 | 2.0952 / 5.9509 | 0.1680 / 0.3754 | 0.9831 / 2.6242 | 2.4699 / 6.8964 |
37
+
38
+ The external evaluation is a deterministic replay of published control overlays. It does not re-run the model. Val and Test are different populations, so their difference is not evidence of a causal benefit from map or route inputs. The complete index and eight reports are under `evaluations/overlay-replay-v1/`.
39
+
40
+ ## Loading
41
+
42
+ ```python
43
+ import torch
44
+
45
+ checkpoint = torch.load("checkpoint.pt", map_location="cpu", weights_only=True)
46
+ model.load_state_dict(checkpoint["model_state_dict"], strict=True)
47
+ ```
48
+
49
+ Use an AutoE2E source revision compatible with `bevformer_v2_t8_split_navigation_v5`. Each model directory contains `metadata.json`. `SHA256SUMS` verifies all published files.
50
+
51
+ ## Intended use and limitations
52
+
53
+ These checkpoints are research artifacts for open-loop trajectory-planning evaluation and continued training. They are not validated for safety-critical deployment. The KITScenes Val route is reconstructed retrospectively from the logged future trajectory and contains oracle maneuver information. Distribution shift remains relevant.
54
+
55
+ ## License notice
56
+
57
+ The AutoE2E source code is Apache-2.0. These checkpoint files include parameters initialized from the official BEVFormer V2 R50 T8 checkpoint. The upstream weight license was recorded as `NOASSERTION`, and the pretrained component was trained on nuScenes. Users are responsible for complying with the terms of the upstream weights and datasets. No broader checkpoint license is asserted here.
SHA256SUMS ADDED
@@ -0,0 +1,19 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ee3c2f95176c285f3649281a986849a872b0920f4d1c67cad627d4e6424214e0 README.md
2
+ be1cec5f77a7a12326070503dbcd1004c63f54450cf1151c763a215816d838bd evaluations/overlay-replay-v1/index.json
3
+ 582dc108ce030756d5d1758dbe198d62b3355bf32a123af54be4e66704c50b8a evaluations/overlay-replay-v1/kitscenes-test/kitscenes-epoch-5/report.json
4
+ d01169d68266d4b7203e92a4d25ade2a3277357ff89cc26cdb202051a26016fc evaluations/overlay-replay-v1/kitscenes-test/kitscenes-epoch-7/report.json
5
+ 5eeb68a4013a9777ba927030b73877a1789f5b4eb9e14f6da760bef7f9b42808 evaluations/overlay-replay-v1/kitscenes-test/nuplan-epoch-4/report.json
6
+ fc5b19beea416706461b535314a497d6a2b929d97668e8475638ae04a65221b7 evaluations/overlay-replay-v1/kitscenes-test/nuplan-epoch-5/report.json
7
+ a8f8777da510b832f49503d959f057df6ab298057542e3ec10fd26bd9cb0bf91 evaluations/overlay-replay-v1/kitscenes-val/kitscenes-epoch-5/report.json
8
+ 832b9ec0648b997f5b1f825296269905e5ee3b13ae75918e0fdee5733dbd1818 evaluations/overlay-replay-v1/kitscenes-val/kitscenes-epoch-7/report.json
9
+ 1ffde86560b8d9701c23c98b6b05b0de2611446a8d92f0fb90a08ccb884d1acb evaluations/overlay-replay-v1/kitscenes-val/nuplan-epoch-4/report.json
10
+ e45996448f95f193acfc225c37ff169fab64bb24b5ea7717928a20c426e7303b evaluations/overlay-replay-v1/kitscenes-val/nuplan-epoch-5/report.json
11
+ 120a21639d9767d512eea645be0a83b01476c9e0366a8b957ac51b7ead1bd762 models/kitscenes-epoch-5/checkpoint.pt
12
+ 7e576d547291522dd1be153408cea0cdaa280b18db340622b7f57688835a6d0c models/kitscenes-epoch-5/metadata.json
13
+ a1e6b1621018740485b3e1e43dda23713d913667eb39a65b77fcbf895dcbcefd models/kitscenes-epoch-7/checkpoint.pt
14
+ f6b4670c41462689c0a7b54c22acbea0c4cbd935d6e369feaea7a963921afb5b models/kitscenes-epoch-7/metadata.json
15
+ ed00e072471aae51b4cf48fa54c3fd9c03c61e0fd6e2978829cd9068dfd6da68 models/nuplan-epoch-4/checkpoint.pt
16
+ 1199152e6c4493cfad16c5dff5053b8edc70db221f70cb57949862b4436eb640 models/nuplan-epoch-4/metadata.json
17
+ ca8b43d7a777d6fd9195bb253d1452f3cd645f7aab3bd0e36b1f74ffb31df29b models/nuplan-epoch-5/checkpoint.pt
18
+ feaa5ae17652c33bad0a4ec286efe9b487f621ca19e4c4ce428cd9d506967569 models/nuplan-epoch-5/metadata.json
19
+ c1c61990ed5cb0d66b36337d7d5ff67a46c37ddff41bd3266950507d747718ee release_manifest.json
evaluations/overlay-replay-v1/index.json ADDED
@@ -0,0 +1,320 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "created_at": "2026-09-14T09:13:53Z",
3
+ "reports": [
4
+ {
5
+ "dataset": "kitscenes-val",
6
+ "dataset_version": "v3.5",
7
+ "model_artifact_id": "120a21639d9767d512eea645be0a83b01476c9e0366a8b957ac51b7ead1bd762",
8
+ "model_name": "KITScenes Trajectory Epoch 5",
9
+ "report_sha256": "a8f8777da510b832f49503d959f057df6ab298057542e3ec10fd26bd9cb0bf91",
10
+ "report_uri": "s3://auto-e2e-platform-artifacts-381491877296/direct-evaluations/overlay-replay-v1/dataset=kitscenes-val/version=v3.5/model=120a21639d9767d512eea645be0a83b01476c9e0366a8b957ac51b7ead1bd762/report.json",
11
+ "trajectory": {
12
+ "ade_1s_m": 0.14720280881581088,
13
+ "ade_1s_sample_count": 11035,
14
+ "ade_2s_m": 0.3729208092788606,
15
+ "ade_2s_sample_count": 11035,
16
+ "ade_3s_m": 0.744916487594762,
17
+ "ade_3s_sample_count": 11035,
18
+ "ade_5s_m": 1.9405047441156529,
19
+ "ade_5s_sample_count": 11035,
20
+ "ade_6p4s_m": null,
21
+ "ade_6p4s_sample_count": 0,
22
+ "fde_1s_m": 0.28501037315538935,
23
+ "fde_1s_sample_count": 11035,
24
+ "fde_2s_m": 0.9239302700556777,
25
+ "fde_2s_sample_count": 11035,
26
+ "fde_3s_m": 2.0195565863793923,
27
+ "fde_3s_sample_count": 11035,
28
+ "fde_5s_m": 5.564463390564478,
29
+ "fde_5s_sample_count": 11035,
30
+ "fde_6p4s_m": null,
31
+ "fde_6p4s_sample_count": 0,
32
+ "mean_abs_lateral_error_m": 0.9844258828682012,
33
+ "mean_abs_longitudinal_error_m": 1.402497202309407,
34
+ "nonfinite_prediction_count": 0,
35
+ "nonfinite_prediction_rate": 0.0,
36
+ "total_timestep_count": 706240,
37
+ "valid_horizon_coverage": 0.78125,
38
+ "valid_timestep_count": 551750
39
+ }
40
+ },
41
+ {
42
+ "dataset": "kitscenes-val",
43
+ "dataset_version": "v3.5",
44
+ "model_artifact_id": "a1e6b1621018740485b3e1e43dda23713d913667eb39a65b77fcbf895dcbcefd",
45
+ "model_name": "KITScenes Trajectory Epoch 7",
46
+ "report_sha256": "832b9ec0648b997f5b1f825296269905e5ee3b13ae75918e0fdee5733dbd1818",
47
+ "report_uri": "s3://auto-e2e-platform-artifacts-381491877296/direct-evaluations/overlay-replay-v1/dataset=kitscenes-val/version=v3.5/model=a1e6b1621018740485b3e1e43dda23713d913667eb39a65b77fcbf895dcbcefd/report.json",
48
+ "trajectory": {
49
+ "ade_1s_m": 0.13467869638959368,
50
+ "ade_1s_sample_count": 11035,
51
+ "ade_2s_m": 0.3886152812418257,
52
+ "ade_2s_sample_count": 11035,
53
+ "ade_3s_m": 0.8034680158738138,
54
+ "ade_3s_sample_count": 11035,
55
+ "ade_5s_m": 2.0951617386340757,
56
+ "ade_5s_sample_count": 11035,
57
+ "ade_6p4s_m": null,
58
+ "ade_6p4s_sample_count": 0,
59
+ "fde_1s_m": 0.2830596284286022,
60
+ "fde_1s_sample_count": 11035,
61
+ "fde_2s_m": 1.0101467772020263,
62
+ "fde_2s_sample_count": 11035,
63
+ "fde_3s_m": 2.212711069790632,
64
+ "fde_3s_sample_count": 11035,
65
+ "fde_5s_m": 5.9508777011937015,
66
+ "fde_5s_sample_count": 11035,
67
+ "fde_6p4s_m": null,
68
+ "fde_6p4s_sample_count": 0,
69
+ "mean_abs_lateral_error_m": 1.0219872821653013,
70
+ "mean_abs_longitudinal_error_m": 1.565857085189425,
71
+ "nonfinite_prediction_count": 0,
72
+ "nonfinite_prediction_rate": 0.0,
73
+ "total_timestep_count": 706240,
74
+ "valid_horizon_coverage": 0.78125,
75
+ "valid_timestep_count": 551750
76
+ }
77
+ },
78
+ {
79
+ "dataset": "kitscenes-val",
80
+ "dataset_version": "v3.5",
81
+ "model_artifact_id": "ca8b43d7a777d6fd9195bb253d1452f3cd645f7aab3bd0e36b1f74ffb31df29b",
82
+ "model_name": "NuPlan Trajectory Epoch 5",
83
+ "report_sha256": "e45996448f95f193acfc225c37ff169fab64bb24b5ea7717928a20c426e7303b",
84
+ "report_uri": "s3://auto-e2e-platform-artifacts-381491877296/direct-evaluations/overlay-replay-v1/dataset=kitscenes-val/version=v3.5/model=ca8b43d7a777d6fd9195bb253d1452f3cd645f7aab3bd0e36b1f74ffb31df29b/report.json",
85
+ "trajectory": {
86
+ "ade_1s_m": 0.19511089635939421,
87
+ "ade_1s_sample_count": 11035,
88
+ "ade_2s_m": 0.4722640826416731,
89
+ "ade_2s_sample_count": 11035,
90
+ "ade_3s_m": 0.9147207671270287,
91
+ "ade_3s_sample_count": 11035,
92
+ "ade_5s_m": 2.295116252818514,
93
+ "ade_5s_sample_count": 11035,
94
+ "ade_6p4s_m": null,
95
+ "ade_6p4s_sample_count": 0,
96
+ "fde_1s_m": 0.37794148166995123,
97
+ "fde_1s_sample_count": 11035,
98
+ "fde_2s_m": 1.1331487579302755,
99
+ "fde_2s_sample_count": 11035,
100
+ "fde_3s_m": 2.422637476977514,
101
+ "fde_3s_sample_count": 11035,
102
+ "fde_5s_m": 6.388465638260406,
103
+ "fde_5s_sample_count": 11035,
104
+ "fde_6p4s_m": null,
105
+ "fde_6p4s_sample_count": 0,
106
+ "mean_abs_lateral_error_m": 1.19114795667851,
107
+ "mean_abs_longitudinal_error_m": 1.6633371994012054,
108
+ "nonfinite_prediction_count": 0,
109
+ "nonfinite_prediction_rate": 0.0,
110
+ "total_timestep_count": 706240,
111
+ "valid_horizon_coverage": 0.78125,
112
+ "valid_timestep_count": 551750
113
+ }
114
+ },
115
+ {
116
+ "dataset": "kitscenes-val",
117
+ "dataset_version": "v3.5",
118
+ "model_artifact_id": "ed00e072471aae51b4cf48fa54c3fd9c03c61e0fd6e2978829cd9068dfd6da68",
119
+ "model_name": "NuPlan Trajectory Epoch 4",
120
+ "report_sha256": "1ffde86560b8d9701c23c98b6b05b0de2611446a8d92f0fb90a08ccb884d1acb",
121
+ "report_uri": "s3://auto-e2e-platform-artifacts-381491877296/direct-evaluations/overlay-replay-v1/dataset=kitscenes-val/version=v3.5/model=ed00e072471aae51b4cf48fa54c3fd9c03c61e0fd6e2978829cd9068dfd6da68/report.json",
122
+ "trajectory": {
123
+ "ade_1s_m": 0.20299996935773157,
124
+ "ade_1s_sample_count": 11035,
125
+ "ade_2s_m": 0.5018003104886625,
126
+ "ade_2s_sample_count": 11035,
127
+ "ade_3s_m": 0.950258079868937,
128
+ "ade_3s_sample_count": 11035,
129
+ "ade_5s_m": 2.291857949000488,
130
+ "ade_5s_sample_count": 11035,
131
+ "ade_6p4s_m": null,
132
+ "ade_6p4s_sample_count": 0,
133
+ "fde_1s_m": 0.40659580015215774,
134
+ "fde_1s_sample_count": 11035,
135
+ "fde_2s_m": 1.1934783774143614,
136
+ "fde_2s_sample_count": 11035,
137
+ "fde_3s_m": 2.450842261185858,
138
+ "fde_3s_sample_count": 11035,
139
+ "fde_5s_m": 6.22721162418439,
140
+ "fde_5s_sample_count": 11035,
141
+ "fde_6p4s_m": null,
142
+ "fde_6p4s_sample_count": 0,
143
+ "mean_abs_lateral_error_m": 1.2082210980047907,
144
+ "mean_abs_longitudinal_error_m": 1.6447808719447803,
145
+ "nonfinite_prediction_count": 0,
146
+ "nonfinite_prediction_rate": 0.0,
147
+ "total_timestep_count": 706240,
148
+ "valid_horizon_coverage": 0.78125,
149
+ "valid_timestep_count": 551750
150
+ }
151
+ },
152
+ {
153
+ "dataset": "kitscenes-test",
154
+ "dataset_version": "v1.0",
155
+ "model_artifact_id": "120a21639d9767d512eea645be0a83b01476c9e0366a8b957ac51b7ead1bd762",
156
+ "model_name": "KITScenes Trajectory Epoch 5",
157
+ "report_sha256": "582dc108ce030756d5d1758dbe198d62b3355bf32a123af54be4e66704c50b8a",
158
+ "report_uri": "s3://auto-e2e-platform-artifacts-381491877296/direct-evaluations/overlay-replay-v1/dataset=kitscenes-test/version=v1.0/model=120a21639d9767d512eea645be0a83b01476c9e0366a8b957ac51b7ead1bd762/report.json",
159
+ "trajectory": {
160
+ "ade_1s_m": 0.14632053498252978,
161
+ "ade_1s_sample_count": 23690,
162
+ "ade_2s_m": 0.4121187056069301,
163
+ "ade_2s_sample_count": 23690,
164
+ "ade_3s_m": 0.8263108995057848,
165
+ "ade_3s_sample_count": 23690,
166
+ "ade_5s_m": 2.104176420295984,
167
+ "ade_5s_sample_count": 23690,
168
+ "ade_6p4s_m": null,
169
+ "ade_6p4s_sample_count": 0,
170
+ "fde_1s_m": 0.3106528805343601,
171
+ "fde_1s_sample_count": 23690,
172
+ "fde_2s_m": 1.0453379525636344,
173
+ "fde_2s_sample_count": 23690,
174
+ "fde_3s_m": 2.219851167527027,
175
+ "fde_3s_sample_count": 23690,
176
+ "fde_5s_m": 5.942222944848886,
177
+ "fde_5s_sample_count": 23690,
178
+ "fde_6p4s_m": null,
179
+ "fde_6p4s_sample_count": 0,
180
+ "mean_abs_lateral_error_m": 1.2043875392879961,
181
+ "mean_abs_longitudinal_error_m": 1.4050545911622678,
182
+ "nonfinite_prediction_count": 0,
183
+ "nonfinite_prediction_rate": 0.0,
184
+ "total_timestep_count": 1516160,
185
+ "valid_horizon_coverage": 0.78125,
186
+ "valid_timestep_count": 1184500
187
+ }
188
+ },
189
+ {
190
+ "dataset": "kitscenes-test",
191
+ "dataset_version": "v1.0",
192
+ "model_artifact_id": "a1e6b1621018740485b3e1e43dda23713d913667eb39a65b77fcbf895dcbcefd",
193
+ "model_name": "KITScenes Trajectory Epoch 7",
194
+ "report_sha256": "d01169d68266d4b7203e92a4d25ade2a3277357ff89cc26cdb202051a26016fc",
195
+ "report_uri": "s3://auto-e2e-platform-artifacts-381491877296/direct-evaluations/overlay-replay-v1/dataset=kitscenes-test/version=v1.0/model=a1e6b1621018740485b3e1e43dda23713d913667eb39a65b77fcbf895dcbcefd/report.json",
196
+ "trajectory": {
197
+ "ade_1s_m": 0.1680068780791157,
198
+ "ade_1s_sample_count": 23690,
199
+ "ade_2s_m": 0.4923481277999561,
200
+ "ade_2s_sample_count": 23690,
201
+ "ade_3s_m": 0.9830703987751742,
202
+ "ade_3s_sample_count": 23690,
203
+ "ade_5s_m": 2.469892014634966,
204
+ "ade_5s_sample_count": 23690,
205
+ "ade_6p4s_m": null,
206
+ "ade_6p4s_sample_count": 0,
207
+ "fde_1s_m": 0.37537789674250205,
208
+ "fde_1s_sample_count": 23690,
209
+ "fde_2s_m": 1.2511962416817588,
210
+ "fde_2s_sample_count": 23690,
211
+ "fde_3s_m": 2.624177628524085,
212
+ "fde_3s_sample_count": 23690,
213
+ "fde_5s_m": 6.896411433552511,
214
+ "fde_5s_sample_count": 23690,
215
+ "fde_6p4s_m": null,
216
+ "fde_6p4s_sample_count": 0,
217
+ "mean_abs_lateral_error_m": 1.4730429718483722,
218
+ "mean_abs_longitudinal_error_m": 1.6598486258851333,
219
+ "nonfinite_prediction_count": 0,
220
+ "nonfinite_prediction_rate": 0.0,
221
+ "total_timestep_count": 1516160,
222
+ "valid_horizon_coverage": 0.78125,
223
+ "valid_timestep_count": 1184500
224
+ }
225
+ },
226
+ {
227
+ "dataset": "kitscenes-test",
228
+ "dataset_version": "v1.0",
229
+ "model_artifact_id": "ca8b43d7a777d6fd9195bb253d1452f3cd645f7aab3bd0e36b1f74ffb31df29b",
230
+ "model_name": "NuPlan Trajectory Epoch 5",
231
+ "report_sha256": "fc5b19beea416706461b535314a497d6a2b929d97668e8475638ae04a65221b7",
232
+ "report_uri": "s3://auto-e2e-platform-artifacts-381491877296/direct-evaluations/overlay-replay-v1/dataset=kitscenes-test/version=v1.0/model=ca8b43d7a777d6fd9195bb253d1452f3cd645f7aab3bd0e36b1f74ffb31df29b/report.json",
233
+ "trajectory": {
234
+ "ade_1s_m": 0.16017068059452302,
235
+ "ade_1s_sample_count": 23690,
236
+ "ade_2s_m": 0.3979342750750205,
237
+ "ade_2s_sample_count": 23690,
238
+ "ade_3s_m": 0.7923739682724033,
239
+ "ade_3s_sample_count": 23690,
240
+ "ade_5s_m": 2.1119916453019565,
241
+ "ade_5s_sample_count": 23690,
242
+ "ade_6p4s_m": null,
243
+ "ade_6p4s_sample_count": 0,
244
+ "fde_1s_m": 0.3134281782098752,
245
+ "fde_1s_sample_count": 23690,
246
+ "fde_2s_m": 0.9727075050905802,
247
+ "fde_2s_sample_count": 23690,
248
+ "fde_3s_m": 2.1621481640709304,
249
+ "fde_3s_sample_count": 23690,
250
+ "fde_5s_m": 6.175121158406895,
251
+ "fde_5s_sample_count": 23690,
252
+ "fde_6p4s_m": null,
253
+ "fde_6p4s_sample_count": 0,
254
+ "mean_abs_lateral_error_m": 1.0584339709055983,
255
+ "mean_abs_longitudinal_error_m": 1.5495691251398722,
256
+ "nonfinite_prediction_count": 0,
257
+ "nonfinite_prediction_rate": 0.0,
258
+ "total_timestep_count": 1516160,
259
+ "valid_horizon_coverage": 0.78125,
260
+ "valid_timestep_count": 1184500
261
+ }
262
+ },
263
+ {
264
+ "dataset": "kitscenes-test",
265
+ "dataset_version": "v1.0",
266
+ "model_artifact_id": "ed00e072471aae51b4cf48fa54c3fd9c03c61e0fd6e2978829cd9068dfd6da68",
267
+ "model_name": "NuPlan Trajectory Epoch 4",
268
+ "report_sha256": "5eeb68a4013a9777ba927030b73877a1789f5b4eb9e14f6da760bef7f9b42808",
269
+ "report_uri": "s3://auto-e2e-platform-artifacts-381491877296/direct-evaluations/overlay-replay-v1/dataset=kitscenes-test/version=v1.0/model=ed00e072471aae51b4cf48fa54c3fd9c03c61e0fd6e2978829cd9068dfd6da68/report.json",
270
+ "trajectory": {
271
+ "ade_1s_m": 0.1915564095529401,
272
+ "ade_1s_sample_count": 23690,
273
+ "ade_2s_m": 0.4816506897179045,
274
+ "ade_2s_sample_count": 23690,
275
+ "ade_3s_m": 0.9181210775291644,
276
+ "ade_3s_sample_count": 23690,
277
+ "ade_5s_m": 2.2840408150913345,
278
+ "ade_5s_sample_count": 23690,
279
+ "ade_6p4s_m": null,
280
+ "ade_6p4s_sample_count": 0,
281
+ "fde_1s_m": 0.3898781367591937,
282
+ "fde_1s_sample_count": 23690,
283
+ "fde_2s_m": 1.1514432707206264,
284
+ "fde_2s_sample_count": 23690,
285
+ "fde_3s_m": 2.38952477957317,
286
+ "fde_3s_sample_count": 23690,
287
+ "fde_5s_m": 6.42657723546563,
288
+ "fde_5s_sample_count": 23690,
289
+ "fde_6p4s_m": null,
290
+ "fde_6p4s_sample_count": 0,
291
+ "mean_abs_lateral_error_m": 1.2167123846906904,
292
+ "mean_abs_longitudinal_error_m": 1.6287253737215717,
293
+ "nonfinite_prediction_count": 0,
294
+ "nonfinite_prediction_rate": 0.0,
295
+ "total_timestep_count": 1516160,
296
+ "valid_horizon_coverage": 0.78125,
297
+ "valid_timestep_count": 1184500
298
+ }
299
+ }
300
+ ],
301
+ "schema_version": "kitscenes_val_test_overlay_evaluation_index_v1",
302
+ "target_identities": {
303
+ "kitscenes-test": {
304
+ "sample_count": 23690,
305
+ "sample_tar_bytes_streamed": 12241868800,
306
+ "sample_uid_sha256": "9f7e5f0eae0582c3b439d29914cefb88e79b944ce1349125d8e9cdbc8bd891db",
307
+ "source_manifest_set_sha256": "ac291affe2c174d8c506f5564fc22989d8d627e01c90f2d17bed2c877a28528a",
308
+ "source_partition_count": 206,
309
+ "source_partition_list_sha256": "5aec3df6d997b94a6d782c9cd3ac7f704fc43cd5d8406b4cd7eb00d1f286b476"
310
+ },
311
+ "kitscenes-val": {
312
+ "sample_count": 11035,
313
+ "sample_tar_bytes_streamed": 7770572800,
314
+ "sample_uid_sha256": "00f824202c97b47ec74561d247e8e30686e5505eef2ad2bc0acba97e41ce9925",
315
+ "source_manifest_set_sha256": "5d4c0ffc7b0706cd917f1b1546cc557b63372839bd67677f8baeef59ad0981e2",
316
+ "source_partition_count": 117,
317
+ "source_partition_list_sha256": "a978e6a373677a15998f41032a850de26cc321271918094237dd71c6e0a51870"
318
+ }
319
+ }
320
+ }
evaluations/overlay-replay-v1/kitscenes-test/kitscenes-epoch-5/report.json ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "created_at": "2026-09-14T10:07:41Z",
3
+ "dataset": "kitscenes-test",
4
+ "dataset_manifest_sha256": "b5dbe72c8d2c5cf49f8d55c9b24a4bde9dca130bdb1816480f299a82b43f1a24",
5
+ "dataset_version": "v1.0",
6
+ "evaluation_role": "official_test_camera_only_missing_map_route",
7
+ "input_track": "camera_only_missing_map_route",
8
+ "limitations": [
9
+ "Trajectory metrics replay the published deterministic control overlay and do not rerun the model.",
10
+ "Route reconstruction, route counterfactual, and route input gradient metrics are not stored in trajectory overlay v5."
11
+ ],
12
+ "metrics": {
13
+ "bev_segmentation": {
14
+ "available": false,
15
+ "reason": "not_replayed_from_trajectory_overlay"
16
+ },
17
+ "route": {
18
+ "available": false,
19
+ "reason": "route_counterfactuals_not_in_overlay_artifact"
20
+ },
21
+ "sample_count": 23690,
22
+ "schema_version": "reactive_multitask_evaluation_v1",
23
+ "trajectory": {
24
+ "ade_1s_m": 0.14632053498252978,
25
+ "ade_1s_sample_count": 23690,
26
+ "ade_2s_m": 0.4121187056069301,
27
+ "ade_2s_sample_count": 23690,
28
+ "ade_3s_m": 0.8263108995057848,
29
+ "ade_3s_sample_count": 23690,
30
+ "ade_5s_m": 2.104176420295984,
31
+ "ade_5s_sample_count": 23690,
32
+ "ade_6p4s_m": null,
33
+ "ade_6p4s_sample_count": 0,
34
+ "fde_1s_m": 0.3106528805343601,
35
+ "fde_1s_sample_count": 23690,
36
+ "fde_2s_m": 1.0453379525636344,
37
+ "fde_2s_sample_count": 23690,
38
+ "fde_3s_m": 2.219851167527027,
39
+ "fde_3s_sample_count": 23690,
40
+ "fde_5s_m": 5.942222944848886,
41
+ "fde_5s_sample_count": 23690,
42
+ "fde_6p4s_m": null,
43
+ "fde_6p4s_sample_count": 0,
44
+ "mean_abs_lateral_error_m": 1.2043875392879961,
45
+ "mean_abs_longitudinal_error_m": 1.4050545911622678,
46
+ "nonfinite_prediction_count": 0,
47
+ "nonfinite_prediction_rate": 0.0,
48
+ "total_timestep_count": 1516160,
49
+ "valid_horizon_coverage": 0.78125,
50
+ "valid_timestep_count": 1184500
51
+ }
52
+ },
53
+ "model_artifact_id": "120a21639d9767d512eea645be0a83b01476c9e0366a8b957ac51b7ead1bd762",
54
+ "model_name": "KITScenes Trajectory Epoch 5",
55
+ "overlay_object_count": 206,
56
+ "overlay_sample_count": 23690,
57
+ "overlay_schema": "v5",
58
+ "overlay_set_sha256": "a14fd102ff9f0ae19b6f6e50141cc3b7318ba5ad2d2e6093de38f470560ce32c",
59
+ "schema_version": "reactive_kitscenes_overlay_replay_evaluation_v1",
60
+ "source_split": "test",
61
+ "target_identity": {
62
+ "sample_count": 23690,
63
+ "sample_tar_bytes_streamed": 12241868800,
64
+ "sample_uid_sha256": "9f7e5f0eae0582c3b439d29914cefb88e79b944ce1349125d8e9cdbc8bd891db",
65
+ "source_manifest_set_sha256": "ac291affe2c174d8c506f5564fc22989d8d627e01c90f2d17bed2c877a28528a",
66
+ "source_partition_count": 206,
67
+ "source_partition_list_sha256": "5aec3df6d997b94a6d782c9cd3ac7f704fc43cd5d8406b4cd7eb00d1f286b476"
68
+ },
69
+ "target_sample_count": 23690,
70
+ "trajectory_inference_policy": {
71
+ "control_rollout": "semi_implicit_unicycle_v1",
72
+ "dt_seconds": 0.1,
73
+ "planner": "gru",
74
+ "stochastic_noise": false
75
+ }
76
+ }
evaluations/overlay-replay-v1/kitscenes-test/kitscenes-epoch-7/report.json ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "created_at": "2026-09-14T10:08:08Z",
3
+ "dataset": "kitscenes-test",
4
+ "dataset_manifest_sha256": "b5dbe72c8d2c5cf49f8d55c9b24a4bde9dca130bdb1816480f299a82b43f1a24",
5
+ "dataset_version": "v1.0",
6
+ "evaluation_role": "official_test_camera_only_missing_map_route",
7
+ "input_track": "camera_only_missing_map_route",
8
+ "limitations": [
9
+ "Trajectory metrics replay the published deterministic control overlay and do not rerun the model.",
10
+ "Route reconstruction, route counterfactual, and route input gradient metrics are not stored in trajectory overlay v5."
11
+ ],
12
+ "metrics": {
13
+ "bev_segmentation": {
14
+ "available": false,
15
+ "reason": "not_replayed_from_trajectory_overlay"
16
+ },
17
+ "route": {
18
+ "available": false,
19
+ "reason": "route_counterfactuals_not_in_overlay_artifact"
20
+ },
21
+ "sample_count": 23690,
22
+ "schema_version": "reactive_multitask_evaluation_v1",
23
+ "trajectory": {
24
+ "ade_1s_m": 0.1680068780791157,
25
+ "ade_1s_sample_count": 23690,
26
+ "ade_2s_m": 0.4923481277999561,
27
+ "ade_2s_sample_count": 23690,
28
+ "ade_3s_m": 0.9830703987751742,
29
+ "ade_3s_sample_count": 23690,
30
+ "ade_5s_m": 2.469892014634966,
31
+ "ade_5s_sample_count": 23690,
32
+ "ade_6p4s_m": null,
33
+ "ade_6p4s_sample_count": 0,
34
+ "fde_1s_m": 0.37537789674250205,
35
+ "fde_1s_sample_count": 23690,
36
+ "fde_2s_m": 1.2511962416817588,
37
+ "fde_2s_sample_count": 23690,
38
+ "fde_3s_m": 2.624177628524085,
39
+ "fde_3s_sample_count": 23690,
40
+ "fde_5s_m": 6.896411433552511,
41
+ "fde_5s_sample_count": 23690,
42
+ "fde_6p4s_m": null,
43
+ "fde_6p4s_sample_count": 0,
44
+ "mean_abs_lateral_error_m": 1.4730429718483722,
45
+ "mean_abs_longitudinal_error_m": 1.6598486258851333,
46
+ "nonfinite_prediction_count": 0,
47
+ "nonfinite_prediction_rate": 0.0,
48
+ "total_timestep_count": 1516160,
49
+ "valid_horizon_coverage": 0.78125,
50
+ "valid_timestep_count": 1184500
51
+ }
52
+ },
53
+ "model_artifact_id": "a1e6b1621018740485b3e1e43dda23713d913667eb39a65b77fcbf895dcbcefd",
54
+ "model_name": "KITScenes Trajectory Epoch 7",
55
+ "overlay_object_count": 206,
56
+ "overlay_sample_count": 23690,
57
+ "overlay_schema": "v5",
58
+ "overlay_set_sha256": "850fda2cb17d89810a4ba9abadf5fb390583fce1b907861a3ec4de9ae23f3c38",
59
+ "schema_version": "reactive_kitscenes_overlay_replay_evaluation_v1",
60
+ "source_split": "test",
61
+ "target_identity": {
62
+ "sample_count": 23690,
63
+ "sample_tar_bytes_streamed": 12241868800,
64
+ "sample_uid_sha256": "9f7e5f0eae0582c3b439d29914cefb88e79b944ce1349125d8e9cdbc8bd891db",
65
+ "source_manifest_set_sha256": "ac291affe2c174d8c506f5564fc22989d8d627e01c90f2d17bed2c877a28528a",
66
+ "source_partition_count": 206,
67
+ "source_partition_list_sha256": "5aec3df6d997b94a6d782c9cd3ac7f704fc43cd5d8406b4cd7eb00d1f286b476"
68
+ },
69
+ "target_sample_count": 23690,
70
+ "trajectory_inference_policy": {
71
+ "control_rollout": "semi_implicit_unicycle_v1",
72
+ "dt_seconds": 0.1,
73
+ "planner": "gru",
74
+ "stochastic_noise": false
75
+ }
76
+ }
evaluations/overlay-replay-v1/kitscenes-test/nuplan-epoch-4/report.json ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "created_at": "2026-09-14T10:09:00Z",
3
+ "dataset": "kitscenes-test",
4
+ "dataset_manifest_sha256": "b5dbe72c8d2c5cf49f8d55c9b24a4bde9dca130bdb1816480f299a82b43f1a24",
5
+ "dataset_version": "v1.0",
6
+ "evaluation_role": "official_test_camera_only_missing_map_route",
7
+ "input_track": "camera_only_missing_map_route",
8
+ "limitations": [
9
+ "Trajectory metrics replay the published deterministic control overlay and do not rerun the model.",
10
+ "Route reconstruction, route counterfactual, and route input gradient metrics are not stored in trajectory overlay v5."
11
+ ],
12
+ "metrics": {
13
+ "bev_segmentation": {
14
+ "available": false,
15
+ "reason": "not_replayed_from_trajectory_overlay"
16
+ },
17
+ "route": {
18
+ "available": false,
19
+ "reason": "route_counterfactuals_not_in_overlay_artifact"
20
+ },
21
+ "sample_count": 23690,
22
+ "schema_version": "reactive_multitask_evaluation_v1",
23
+ "trajectory": {
24
+ "ade_1s_m": 0.1915564095529401,
25
+ "ade_1s_sample_count": 23690,
26
+ "ade_2s_m": 0.4816506897179045,
27
+ "ade_2s_sample_count": 23690,
28
+ "ade_3s_m": 0.9181210775291644,
29
+ "ade_3s_sample_count": 23690,
30
+ "ade_5s_m": 2.2840408150913345,
31
+ "ade_5s_sample_count": 23690,
32
+ "ade_6p4s_m": null,
33
+ "ade_6p4s_sample_count": 0,
34
+ "fde_1s_m": 0.3898781367591937,
35
+ "fde_1s_sample_count": 23690,
36
+ "fde_2s_m": 1.1514432707206264,
37
+ "fde_2s_sample_count": 23690,
38
+ "fde_3s_m": 2.38952477957317,
39
+ "fde_3s_sample_count": 23690,
40
+ "fde_5s_m": 6.42657723546563,
41
+ "fde_5s_sample_count": 23690,
42
+ "fde_6p4s_m": null,
43
+ "fde_6p4s_sample_count": 0,
44
+ "mean_abs_lateral_error_m": 1.2167123846906904,
45
+ "mean_abs_longitudinal_error_m": 1.6287253737215717,
46
+ "nonfinite_prediction_count": 0,
47
+ "nonfinite_prediction_rate": 0.0,
48
+ "total_timestep_count": 1516160,
49
+ "valid_horizon_coverage": 0.78125,
50
+ "valid_timestep_count": 1184500
51
+ }
52
+ },
53
+ "model_artifact_id": "ed00e072471aae51b4cf48fa54c3fd9c03c61e0fd6e2978829cd9068dfd6da68",
54
+ "model_name": "NuPlan Trajectory Epoch 4",
55
+ "overlay_object_count": 206,
56
+ "overlay_sample_count": 23690,
57
+ "overlay_schema": "v5",
58
+ "overlay_set_sha256": "fc2202e59c30aa6bf7a7ff7f3afec24b8b29516cc1eb6083a80eacb41f9427c1",
59
+ "schema_version": "reactive_kitscenes_overlay_replay_evaluation_v1",
60
+ "source_split": "test",
61
+ "target_identity": {
62
+ "sample_count": 23690,
63
+ "sample_tar_bytes_streamed": 12241868800,
64
+ "sample_uid_sha256": "9f7e5f0eae0582c3b439d29914cefb88e79b944ce1349125d8e9cdbc8bd891db",
65
+ "source_manifest_set_sha256": "ac291affe2c174d8c506f5564fc22989d8d627e01c90f2d17bed2c877a28528a",
66
+ "source_partition_count": 206,
67
+ "source_partition_list_sha256": "5aec3df6d997b94a6d782c9cd3ac7f704fc43cd5d8406b4cd7eb00d1f286b476"
68
+ },
69
+ "target_sample_count": 23690,
70
+ "trajectory_inference_policy": {
71
+ "control_rollout": "semi_implicit_unicycle_v1",
72
+ "dt_seconds": 0.1,
73
+ "planner": "gru",
74
+ "stochastic_noise": false
75
+ }
76
+ }
evaluations/overlay-replay-v1/kitscenes-test/nuplan-epoch-5/report.json ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "created_at": "2026-09-14T10:08:35Z",
3
+ "dataset": "kitscenes-test",
4
+ "dataset_manifest_sha256": "b5dbe72c8d2c5cf49f8d55c9b24a4bde9dca130bdb1816480f299a82b43f1a24",
5
+ "dataset_version": "v1.0",
6
+ "evaluation_role": "official_test_camera_only_missing_map_route",
7
+ "input_track": "camera_only_missing_map_route",
8
+ "limitations": [
9
+ "Trajectory metrics replay the published deterministic control overlay and do not rerun the model.",
10
+ "Route reconstruction, route counterfactual, and route input gradient metrics are not stored in trajectory overlay v5."
11
+ ],
12
+ "metrics": {
13
+ "bev_segmentation": {
14
+ "available": false,
15
+ "reason": "not_replayed_from_trajectory_overlay"
16
+ },
17
+ "route": {
18
+ "available": false,
19
+ "reason": "route_counterfactuals_not_in_overlay_artifact"
20
+ },
21
+ "sample_count": 23690,
22
+ "schema_version": "reactive_multitask_evaluation_v1",
23
+ "trajectory": {
24
+ "ade_1s_m": 0.16017068059452302,
25
+ "ade_1s_sample_count": 23690,
26
+ "ade_2s_m": 0.3979342750750205,
27
+ "ade_2s_sample_count": 23690,
28
+ "ade_3s_m": 0.7923739682724033,
29
+ "ade_3s_sample_count": 23690,
30
+ "ade_5s_m": 2.1119916453019565,
31
+ "ade_5s_sample_count": 23690,
32
+ "ade_6p4s_m": null,
33
+ "ade_6p4s_sample_count": 0,
34
+ "fde_1s_m": 0.3134281782098752,
35
+ "fde_1s_sample_count": 23690,
36
+ "fde_2s_m": 0.9727075050905802,
37
+ "fde_2s_sample_count": 23690,
38
+ "fde_3s_m": 2.1621481640709304,
39
+ "fde_3s_sample_count": 23690,
40
+ "fde_5s_m": 6.175121158406895,
41
+ "fde_5s_sample_count": 23690,
42
+ "fde_6p4s_m": null,
43
+ "fde_6p4s_sample_count": 0,
44
+ "mean_abs_lateral_error_m": 1.0584339709055983,
45
+ "mean_abs_longitudinal_error_m": 1.5495691251398722,
46
+ "nonfinite_prediction_count": 0,
47
+ "nonfinite_prediction_rate": 0.0,
48
+ "total_timestep_count": 1516160,
49
+ "valid_horizon_coverage": 0.78125,
50
+ "valid_timestep_count": 1184500
51
+ }
52
+ },
53
+ "model_artifact_id": "ca8b43d7a777d6fd9195bb253d1452f3cd645f7aab3bd0e36b1f74ffb31df29b",
54
+ "model_name": "NuPlan Trajectory Epoch 5",
55
+ "overlay_object_count": 206,
56
+ "overlay_sample_count": 23690,
57
+ "overlay_schema": "v5",
58
+ "overlay_set_sha256": "4e379136e08603ae8aece99e24e5269ed8283229c80b2844d120ec11aff09c36",
59
+ "schema_version": "reactive_kitscenes_overlay_replay_evaluation_v1",
60
+ "source_split": "test",
61
+ "target_identity": {
62
+ "sample_count": 23690,
63
+ "sample_tar_bytes_streamed": 12241868800,
64
+ "sample_uid_sha256": "9f7e5f0eae0582c3b439d29914cefb88e79b944ce1349125d8e9cdbc8bd891db",
65
+ "source_manifest_set_sha256": "ac291affe2c174d8c506f5564fc22989d8d627e01c90f2d17bed2c877a28528a",
66
+ "source_partition_count": 206,
67
+ "source_partition_list_sha256": "5aec3df6d997b94a6d782c9cd3ac7f704fc43cd5d8406b4cd7eb00d1f286b476"
68
+ },
69
+ "target_sample_count": 23690,
70
+ "trajectory_inference_policy": {
71
+ "control_rollout": "semi_implicit_unicycle_v1",
72
+ "dt_seconds": 0.1,
73
+ "planner": "gru",
74
+ "stochastic_noise": false
75
+ }
76
+ }
evaluations/overlay-replay-v1/kitscenes-val/kitscenes-epoch-5/report.json ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "created_at": "2026-09-14T09:16:51Z",
3
+ "dataset": "kitscenes-val",
4
+ "dataset_manifest_sha256": "4a12ede73949cf66fd01fdaee89766f1e534de705f783a2f65b372c142fa1d05",
5
+ "dataset_version": "v3.5",
6
+ "evaluation_role": "official_val_camera_map_route",
7
+ "input_track": "camera_map_route",
8
+ "limitations": [
9
+ "Trajectory metrics replay the published deterministic control overlay and do not rerun the model.",
10
+ "Route reconstruction, route counterfactual, and route input gradient metrics are not stored in trajectory overlay v5."
11
+ ],
12
+ "metrics": {
13
+ "bev_segmentation": {
14
+ "available": false,
15
+ "reason": "not_replayed_from_trajectory_overlay"
16
+ },
17
+ "route": {
18
+ "available": false,
19
+ "reason": "route_counterfactuals_not_in_overlay_artifact"
20
+ },
21
+ "sample_count": 11035,
22
+ "schema_version": "reactive_multitask_evaluation_v1",
23
+ "trajectory": {
24
+ "ade_1s_m": 0.14720280881581088,
25
+ "ade_1s_sample_count": 11035,
26
+ "ade_2s_m": 0.3729208092788606,
27
+ "ade_2s_sample_count": 11035,
28
+ "ade_3s_m": 0.744916487594762,
29
+ "ade_3s_sample_count": 11035,
30
+ "ade_5s_m": 1.9405047441156529,
31
+ "ade_5s_sample_count": 11035,
32
+ "ade_6p4s_m": null,
33
+ "ade_6p4s_sample_count": 0,
34
+ "fde_1s_m": 0.28501037315538935,
35
+ "fde_1s_sample_count": 11035,
36
+ "fde_2s_m": 0.9239302700556777,
37
+ "fde_2s_sample_count": 11035,
38
+ "fde_3s_m": 2.0195565863793923,
39
+ "fde_3s_sample_count": 11035,
40
+ "fde_5s_m": 5.564463390564478,
41
+ "fde_5s_sample_count": 11035,
42
+ "fde_6p4s_m": null,
43
+ "fde_6p4s_sample_count": 0,
44
+ "mean_abs_lateral_error_m": 0.9844258828682012,
45
+ "mean_abs_longitudinal_error_m": 1.402497202309407,
46
+ "nonfinite_prediction_count": 0,
47
+ "nonfinite_prediction_rate": 0.0,
48
+ "total_timestep_count": 706240,
49
+ "valid_horizon_coverage": 0.78125,
50
+ "valid_timestep_count": 551750
51
+ }
52
+ },
53
+ "model_artifact_id": "120a21639d9767d512eea645be0a83b01476c9e0366a8b957ac51b7ead1bd762",
54
+ "model_name": "KITScenes Trajectory Epoch 5",
55
+ "overlay_object_count": 140,
56
+ "overlay_sample_count": 13525,
57
+ "overlay_schema": "v5",
58
+ "overlay_set_sha256": "d38f6b4b23cbeb94ac00f4ed2dfbf10bf80a9b19d8f8ee1aeeb849731fc407e7",
59
+ "schema_version": "reactive_kitscenes_overlay_replay_evaluation_v1",
60
+ "source_split": "val",
61
+ "target_identity": {
62
+ "sample_count": 11035,
63
+ "sample_tar_bytes_streamed": 7770572800,
64
+ "sample_uid_sha256": "00f824202c97b47ec74561d247e8e30686e5505eef2ad2bc0acba97e41ce9925",
65
+ "source_manifest_set_sha256": "5d4c0ffc7b0706cd917f1b1546cc557b63372839bd67677f8baeef59ad0981e2",
66
+ "source_partition_count": 117,
67
+ "source_partition_list_sha256": "a978e6a373677a15998f41032a850de26cc321271918094237dd71c6e0a51870"
68
+ },
69
+ "target_sample_count": 11035,
70
+ "trajectory_inference_policy": {
71
+ "control_rollout": "semi_implicit_unicycle_v1",
72
+ "dt_seconds": 0.1,
73
+ "planner": "gru",
74
+ "stochastic_noise": false
75
+ }
76
+ }
evaluations/overlay-replay-v1/kitscenes-val/kitscenes-epoch-7/report.json ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "created_at": "2026-09-14T09:34:08Z",
3
+ "dataset": "kitscenes-val",
4
+ "dataset_manifest_sha256": "4a12ede73949cf66fd01fdaee89766f1e534de705f783a2f65b372c142fa1d05",
5
+ "dataset_version": "v3.5",
6
+ "evaluation_role": "official_val_camera_map_route",
7
+ "input_track": "camera_map_route",
8
+ "limitations": [
9
+ "Trajectory metrics replay the published deterministic control overlay and do not rerun the model.",
10
+ "Route reconstruction, route counterfactual, and route input gradient metrics are not stored in trajectory overlay v5."
11
+ ],
12
+ "metrics": {
13
+ "bev_segmentation": {
14
+ "available": false,
15
+ "reason": "not_replayed_from_trajectory_overlay"
16
+ },
17
+ "route": {
18
+ "available": false,
19
+ "reason": "route_counterfactuals_not_in_overlay_artifact"
20
+ },
21
+ "sample_count": 11035,
22
+ "schema_version": "reactive_multitask_evaluation_v1",
23
+ "trajectory": {
24
+ "ade_1s_m": 0.13467869638959368,
25
+ "ade_1s_sample_count": 11035,
26
+ "ade_2s_m": 0.3886152812418257,
27
+ "ade_2s_sample_count": 11035,
28
+ "ade_3s_m": 0.8034680158738138,
29
+ "ade_3s_sample_count": 11035,
30
+ "ade_5s_m": 2.0951617386340757,
31
+ "ade_5s_sample_count": 11035,
32
+ "ade_6p4s_m": null,
33
+ "ade_6p4s_sample_count": 0,
34
+ "fde_1s_m": 0.2830596284286022,
35
+ "fde_1s_sample_count": 11035,
36
+ "fde_2s_m": 1.0101467772020263,
37
+ "fde_2s_sample_count": 11035,
38
+ "fde_3s_m": 2.212711069790632,
39
+ "fde_3s_sample_count": 11035,
40
+ "fde_5s_m": 5.9508777011937015,
41
+ "fde_5s_sample_count": 11035,
42
+ "fde_6p4s_m": null,
43
+ "fde_6p4s_sample_count": 0,
44
+ "mean_abs_lateral_error_m": 1.0219872821653013,
45
+ "mean_abs_longitudinal_error_m": 1.565857085189425,
46
+ "nonfinite_prediction_count": 0,
47
+ "nonfinite_prediction_rate": 0.0,
48
+ "total_timestep_count": 706240,
49
+ "valid_horizon_coverage": 0.78125,
50
+ "valid_timestep_count": 551750
51
+ }
52
+ },
53
+ "model_artifact_id": "a1e6b1621018740485b3e1e43dda23713d913667eb39a65b77fcbf895dcbcefd",
54
+ "model_name": "KITScenes Trajectory Epoch 7",
55
+ "overlay_object_count": 140,
56
+ "overlay_sample_count": 13525,
57
+ "overlay_schema": "v5",
58
+ "overlay_set_sha256": "8886560702b7ba4cc22c3bb491bd814406029e9bae41d0a262f97ee8314d06b1",
59
+ "schema_version": "reactive_kitscenes_overlay_replay_evaluation_v1",
60
+ "source_split": "val",
61
+ "target_identity": {
62
+ "sample_count": 11035,
63
+ "sample_tar_bytes_streamed": 7770572800,
64
+ "sample_uid_sha256": "00f824202c97b47ec74561d247e8e30686e5505eef2ad2bc0acba97e41ce9925",
65
+ "source_manifest_set_sha256": "5d4c0ffc7b0706cd917f1b1546cc557b63372839bd67677f8baeef59ad0981e2",
66
+ "source_partition_count": 117,
67
+ "source_partition_list_sha256": "a978e6a373677a15998f41032a850de26cc321271918094237dd71c6e0a51870"
68
+ },
69
+ "target_sample_count": 11035,
70
+ "trajectory_inference_policy": {
71
+ "control_rollout": "semi_implicit_unicycle_v1",
72
+ "dt_seconds": 0.1,
73
+ "planner": "gru",
74
+ "stochastic_noise": false
75
+ }
76
+ }
evaluations/overlay-replay-v1/kitscenes-val/nuplan-epoch-4/report.json ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "created_at": "2026-09-14T10:02:44Z",
3
+ "dataset": "kitscenes-val",
4
+ "dataset_manifest_sha256": "4a12ede73949cf66fd01fdaee89766f1e534de705f783a2f65b372c142fa1d05",
5
+ "dataset_version": "v3.5",
6
+ "evaluation_role": "official_val_camera_map_route",
7
+ "input_track": "camera_map_route",
8
+ "limitations": [
9
+ "Trajectory metrics replay the published deterministic control overlay and do not rerun the model.",
10
+ "Route reconstruction, route counterfactual, and route input gradient metrics are not stored in trajectory overlay v5."
11
+ ],
12
+ "metrics": {
13
+ "bev_segmentation": {
14
+ "available": false,
15
+ "reason": "not_replayed_from_trajectory_overlay"
16
+ },
17
+ "route": {
18
+ "available": false,
19
+ "reason": "route_counterfactuals_not_in_overlay_artifact"
20
+ },
21
+ "sample_count": 11035,
22
+ "schema_version": "reactive_multitask_evaluation_v1",
23
+ "trajectory": {
24
+ "ade_1s_m": 0.20299996935773157,
25
+ "ade_1s_sample_count": 11035,
26
+ "ade_2s_m": 0.5018003104886625,
27
+ "ade_2s_sample_count": 11035,
28
+ "ade_3s_m": 0.950258079868937,
29
+ "ade_3s_sample_count": 11035,
30
+ "ade_5s_m": 2.291857949000488,
31
+ "ade_5s_sample_count": 11035,
32
+ "ade_6p4s_m": null,
33
+ "ade_6p4s_sample_count": 0,
34
+ "fde_1s_m": 0.40659580015215774,
35
+ "fde_1s_sample_count": 11035,
36
+ "fde_2s_m": 1.1934783774143614,
37
+ "fde_2s_sample_count": 11035,
38
+ "fde_3s_m": 2.450842261185858,
39
+ "fde_3s_sample_count": 11035,
40
+ "fde_5s_m": 6.22721162418439,
41
+ "fde_5s_sample_count": 11035,
42
+ "fde_6p4s_m": null,
43
+ "fde_6p4s_sample_count": 0,
44
+ "mean_abs_lateral_error_m": 1.2082210980047907,
45
+ "mean_abs_longitudinal_error_m": 1.6447808719447803,
46
+ "nonfinite_prediction_count": 0,
47
+ "nonfinite_prediction_rate": 0.0,
48
+ "total_timestep_count": 706240,
49
+ "valid_horizon_coverage": 0.78125,
50
+ "valid_timestep_count": 551750
51
+ }
52
+ },
53
+ "model_artifact_id": "ed00e072471aae51b4cf48fa54c3fd9c03c61e0fd6e2978829cd9068dfd6da68",
54
+ "model_name": "NuPlan Trajectory Epoch 4",
55
+ "overlay_object_count": 140,
56
+ "overlay_sample_count": 13525,
57
+ "overlay_schema": "v5",
58
+ "overlay_set_sha256": "4a3f10790e6ad66a7c2aef5bdb121ca86f361f3e1bb86b5a5bd0abb056942c3f",
59
+ "schema_version": "reactive_kitscenes_overlay_replay_evaluation_v1",
60
+ "source_split": "val",
61
+ "target_identity": {
62
+ "sample_count": 11035,
63
+ "sample_tar_bytes_streamed": 7770572800,
64
+ "sample_uid_sha256": "00f824202c97b47ec74561d247e8e30686e5505eef2ad2bc0acba97e41ce9925",
65
+ "source_manifest_set_sha256": "5d4c0ffc7b0706cd917f1b1546cc557b63372839bd67677f8baeef59ad0981e2",
66
+ "source_partition_count": 117,
67
+ "source_partition_list_sha256": "a978e6a373677a15998f41032a850de26cc321271918094237dd71c6e0a51870"
68
+ },
69
+ "target_sample_count": 11035,
70
+ "trajectory_inference_policy": {
71
+ "control_rollout": "semi_implicit_unicycle_v1",
72
+ "dt_seconds": 0.1,
73
+ "planner": "gru",
74
+ "stochastic_noise": false
75
+ }
76
+ }
evaluations/overlay-replay-v1/kitscenes-val/nuplan-epoch-5/report.json ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "created_at": "2026-09-14T10:00:27Z",
3
+ "dataset": "kitscenes-val",
4
+ "dataset_manifest_sha256": "4a12ede73949cf66fd01fdaee89766f1e534de705f783a2f65b372c142fa1d05",
5
+ "dataset_version": "v3.5",
6
+ "evaluation_role": "official_val_camera_map_route",
7
+ "input_track": "camera_map_route",
8
+ "limitations": [
9
+ "Trajectory metrics replay the published deterministic control overlay and do not rerun the model.",
10
+ "Route reconstruction, route counterfactual, and route input gradient metrics are not stored in trajectory overlay v5."
11
+ ],
12
+ "metrics": {
13
+ "bev_segmentation": {
14
+ "available": false,
15
+ "reason": "not_replayed_from_trajectory_overlay"
16
+ },
17
+ "route": {
18
+ "available": false,
19
+ "reason": "route_counterfactuals_not_in_overlay_artifact"
20
+ },
21
+ "sample_count": 11035,
22
+ "schema_version": "reactive_multitask_evaluation_v1",
23
+ "trajectory": {
24
+ "ade_1s_m": 0.19511089635939421,
25
+ "ade_1s_sample_count": 11035,
26
+ "ade_2s_m": 0.4722640826416731,
27
+ "ade_2s_sample_count": 11035,
28
+ "ade_3s_m": 0.9147207671270287,
29
+ "ade_3s_sample_count": 11035,
30
+ "ade_5s_m": 2.295116252818514,
31
+ "ade_5s_sample_count": 11035,
32
+ "ade_6p4s_m": null,
33
+ "ade_6p4s_sample_count": 0,
34
+ "fde_1s_m": 0.37794148166995123,
35
+ "fde_1s_sample_count": 11035,
36
+ "fde_2s_m": 1.1331487579302755,
37
+ "fde_2s_sample_count": 11035,
38
+ "fde_3s_m": 2.422637476977514,
39
+ "fde_3s_sample_count": 11035,
40
+ "fde_5s_m": 6.388465638260406,
41
+ "fde_5s_sample_count": 11035,
42
+ "fde_6p4s_m": null,
43
+ "fde_6p4s_sample_count": 0,
44
+ "mean_abs_lateral_error_m": 1.19114795667851,
45
+ "mean_abs_longitudinal_error_m": 1.6633371994012054,
46
+ "nonfinite_prediction_count": 0,
47
+ "nonfinite_prediction_rate": 0.0,
48
+ "total_timestep_count": 706240,
49
+ "valid_horizon_coverage": 0.78125,
50
+ "valid_timestep_count": 551750
51
+ }
52
+ },
53
+ "model_artifact_id": "ca8b43d7a777d6fd9195bb253d1452f3cd645f7aab3bd0e36b1f74ffb31df29b",
54
+ "model_name": "NuPlan Trajectory Epoch 5",
55
+ "overlay_object_count": 140,
56
+ "overlay_sample_count": 13525,
57
+ "overlay_schema": "v5",
58
+ "overlay_set_sha256": "036487860685027ef9e279b70417b47fa4165537daa50c6fde518e7ab6508f04",
59
+ "schema_version": "reactive_kitscenes_overlay_replay_evaluation_v1",
60
+ "source_split": "val",
61
+ "target_identity": {
62
+ "sample_count": 11035,
63
+ "sample_tar_bytes_streamed": 7770572800,
64
+ "sample_uid_sha256": "00f824202c97b47ec74561d247e8e30686e5505eef2ad2bc0acba97e41ce9925",
65
+ "source_manifest_set_sha256": "5d4c0ffc7b0706cd917f1b1546cc557b63372839bd67677f8baeef59ad0981e2",
66
+ "source_partition_count": 117,
67
+ "source_partition_list_sha256": "a978e6a373677a15998f41032a850de26cc321271918094237dd71c6e0a51870"
68
+ },
69
+ "target_sample_count": 11035,
70
+ "trajectory_inference_policy": {
71
+ "control_rollout": "semi_implicit_unicycle_v1",
72
+ "dt_seconds": 0.1,
73
+ "planner": "gru",
74
+ "stochastic_noise": false
75
+ }
76
+ }
models/kitscenes-epoch-5/checkpoint.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:120a21639d9767d512eea645be0a83b01476c9e0366a8b957ac51b7ead1bd762
3
+ size 334754024
models/kitscenes-epoch-5/metadata.json ADDED
@@ -0,0 +1,153 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architecture": {
3
+ "enable_bev_segmentation": true,
4
+ "enable_reasoning": false,
5
+ "enable_route_conditioning": true,
6
+ "enable_route_reconstruction": true,
7
+ "enable_world_model": false,
8
+ "freeze_bevformer": true,
9
+ "identifier": "bevformer_v2_t8_split_navigation_v5",
10
+ "num_views": 6
11
+ },
12
+ "checkpoint": {
13
+ "config_sha256": "03105a365c230c2d7c8e74f51b46afef3f626c79c417881661dd8c2267b9f2ee",
14
+ "contains": [
15
+ "config",
16
+ "config_sha256",
17
+ "epoch",
18
+ "metrics",
19
+ "metrics_sha256",
20
+ "model_state_dict",
21
+ "model_state_sha256",
22
+ "optimizer_state_dict",
23
+ "scheduler_state_dict",
24
+ "training_state"
25
+ ],
26
+ "epoch": 5,
27
+ "format": "PyTorch training checkpoint",
28
+ "metrics_sha256": "5df5b910163252a3eed6712ed3ca62b5de74006b52a7402a88186104c98a349e",
29
+ "model_state_sha256": "3ff76c0b70eb3220e0de24439f24589565504a728da0d95555f71548753d2e47",
30
+ "path": "models/kitscenes-epoch-5/checkpoint.pt",
31
+ "sha256": "120a21639d9767d512eea645be0a83b01476c9e0366a8b957ac51b7ead1bd762",
32
+ "size_bytes": 334754024
33
+ },
34
+ "display_name": "KITScenes Epoch 5",
35
+ "evaluations": {
36
+ "external_overlay_replay_v1": {
37
+ "test": {
38
+ "dataset": "kitscenes-test",
39
+ "dataset_version": "v1.0",
40
+ "input_track": "camera_only_missing_map_route",
41
+ "model_reexecuted": false,
42
+ "report_path": "evaluations/overlay-replay-v1/kitscenes-test/kitscenes-epoch-5/report.json",
43
+ "report_sha256": "582dc108ce030756d5d1758dbe198d62b3355bf32a123af54be4e66704c50b8a",
44
+ "sample_count": 23690,
45
+ "trajectory": {
46
+ "ade_1s_m": 0.14632053498252978,
47
+ "ade_1s_sample_count": 23690,
48
+ "ade_2s_m": 0.4121187056069301,
49
+ "ade_2s_sample_count": 23690,
50
+ "ade_3s_m": 0.8263108995057848,
51
+ "ade_3s_sample_count": 23690,
52
+ "ade_5s_m": 2.104176420295984,
53
+ "ade_5s_sample_count": 23690,
54
+ "ade_6p4s_m": null,
55
+ "ade_6p4s_sample_count": 0,
56
+ "fde_1s_m": 0.3106528805343601,
57
+ "fde_1s_sample_count": 23690,
58
+ "fde_2s_m": 1.0453379525636344,
59
+ "fde_2s_sample_count": 23690,
60
+ "fde_3s_m": 2.219851167527027,
61
+ "fde_3s_sample_count": 23690,
62
+ "fde_5s_m": 5.942222944848886,
63
+ "fde_5s_sample_count": 23690,
64
+ "fde_6p4s_m": null,
65
+ "fde_6p4s_sample_count": 0,
66
+ "mean_abs_lateral_error_m": 1.2043875392879961,
67
+ "mean_abs_longitudinal_error_m": 1.4050545911622678,
68
+ "nonfinite_prediction_count": 0,
69
+ "nonfinite_prediction_rate": 0.0,
70
+ "total_timestep_count": 1516160,
71
+ "valid_horizon_coverage": 0.78125,
72
+ "valid_timestep_count": 1184500
73
+ }
74
+ },
75
+ "val": {
76
+ "dataset": "kitscenes-val",
77
+ "dataset_version": "v3.5",
78
+ "input_track": "camera_map_route",
79
+ "model_reexecuted": false,
80
+ "report_path": "evaluations/overlay-replay-v1/kitscenes-val/kitscenes-epoch-5/report.json",
81
+ "report_sha256": "a8f8777da510b832f49503d959f057df6ab298057542e3ec10fd26bd9cb0bf91",
82
+ "sample_count": 11035,
83
+ "trajectory": {
84
+ "ade_1s_m": 0.14720280881581088,
85
+ "ade_1s_sample_count": 11035,
86
+ "ade_2s_m": 0.3729208092788606,
87
+ "ade_2s_sample_count": 11035,
88
+ "ade_3s_m": 0.744916487594762,
89
+ "ade_3s_sample_count": 11035,
90
+ "ade_5s_m": 1.9405047441156529,
91
+ "ade_5s_sample_count": 11035,
92
+ "ade_6p4s_m": null,
93
+ "ade_6p4s_sample_count": 0,
94
+ "fde_1s_m": 0.28501037315538935,
95
+ "fde_1s_sample_count": 11035,
96
+ "fde_2s_m": 0.9239302700556777,
97
+ "fde_2s_sample_count": 11035,
98
+ "fde_3s_m": 2.0195565863793923,
99
+ "fde_3s_sample_count": 11035,
100
+ "fde_5s_m": 5.564463390564478,
101
+ "fde_5s_sample_count": 11035,
102
+ "fde_6p4s_m": null,
103
+ "fde_6p4s_sample_count": 0,
104
+ "mean_abs_lateral_error_m": 0.9844258828682012,
105
+ "mean_abs_longitudinal_error_m": 1.402497202309407,
106
+ "nonfinite_prediction_count": 0,
107
+ "nonfinite_prediction_rate": 0.0,
108
+ "total_timestep_count": 706240,
109
+ "valid_horizon_coverage": 0.78125,
110
+ "valid_timestep_count": 551750
111
+ }
112
+ }
113
+ },
114
+ "internal_validation": {
115
+ "ade_6p4s_m": 2.632596758054813,
116
+ "comfortable_rate": 0.7541884816753927,
117
+ "drivable_area_compliance_rate": 0.7758017015706806,
118
+ "drivable_area_success_rate": 0.4442408376963351,
119
+ "fde_6p4s_m": 7.556681459112317,
120
+ "route_corridor_compliance_rate": 0.39594963071109585,
121
+ "route_corridor_iou": 0.9991168210595887,
122
+ "route_corridor_success_rate": 0.06080384747509447
123
+ }
124
+ },
125
+ "license_notice": {
126
+ "checkpoint": "Other; see the repository model card.",
127
+ "pretrained_component": "Official BEVFormer V2 R50 T8 initialization; weight license recorded as NOASSERTION.",
128
+ "source_code": "Apache-2.0",
129
+ "training_data_notice": "The pretrained component was trained on nuScenes, whose terms may impose additional restrictions."
130
+ },
131
+ "limitations": [
132
+ "KITScenes Val uses an oracle post-hoc route reconstructed from the logged future trajectory.",
133
+ "KITScenes Val and Test are different scene populations and are not a causal map or route ablation.",
134
+ "External overlay replay evaluates published controls and does not re-run the model.",
135
+ "Confidence intervals and per-sample route counterfactuals are not included.",
136
+ "These open-loop checkpoints are not validated for safety-critical deployment."
137
+ ],
138
+ "lineage": {
139
+ "parent": "nuPlan Epoch 5 checkpoint ca8b43d7a777",
140
+ "training_stage": "KITScenes fine-tuning with frozen camera BEV"
141
+ },
142
+ "mlflow_registry": {
143
+ "external_overlay_test_run_id": "6a6fe8ae954d495f9f0b4aab9f740e80",
144
+ "external_overlay_val_run_id": "dacc23b278a74932999333718e772fa5",
145
+ "internal_validation_run_id": "8e0c2d24d7094d6192eae58078728641",
146
+ "model_version": "66",
147
+ "registered_model_name": "auto-e2e-driving-policy"
148
+ },
149
+ "model_id": "kitscenes-epoch-5",
150
+ "published_at": "2026-09-16T02:26:06+00:00",
151
+ "repository": "AutowareFoundation/auto_e2e",
152
+ "schema_version": "auto_e2e_checkpoint_metadata_v1"
153
+ }
models/kitscenes-epoch-7/checkpoint.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a1e6b1621018740485b3e1e43dda23713d913667eb39a65b77fcbf895dcbcefd
3
+ size 334754024
models/kitscenes-epoch-7/metadata.json ADDED
@@ -0,0 +1,153 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architecture": {
3
+ "enable_bev_segmentation": true,
4
+ "enable_reasoning": false,
5
+ "enable_route_conditioning": true,
6
+ "enable_route_reconstruction": true,
7
+ "enable_world_model": false,
8
+ "freeze_bevformer": true,
9
+ "identifier": "bevformer_v2_t8_split_navigation_v5",
10
+ "num_views": 6
11
+ },
12
+ "checkpoint": {
13
+ "config_sha256": "03105a365c230c2d7c8e74f51b46afef3f626c79c417881661dd8c2267b9f2ee",
14
+ "contains": [
15
+ "config",
16
+ "config_sha256",
17
+ "epoch",
18
+ "metrics",
19
+ "metrics_sha256",
20
+ "model_state_dict",
21
+ "model_state_sha256",
22
+ "optimizer_state_dict",
23
+ "scheduler_state_dict",
24
+ "training_state"
25
+ ],
26
+ "epoch": 7,
27
+ "format": "PyTorch training checkpoint",
28
+ "metrics_sha256": "40842a9483e223ab86d81b830b1c7fb136ce4ba274420860befd64d5d33b1fe1",
29
+ "model_state_sha256": "422601516e530c2c5793cd00f997abd4b708074117bdcdef0053a3620c651c81",
30
+ "path": "models/kitscenes-epoch-7/checkpoint.pt",
31
+ "sha256": "a1e6b1621018740485b3e1e43dda23713d913667eb39a65b77fcbf895dcbcefd",
32
+ "size_bytes": 334754024
33
+ },
34
+ "display_name": "KITScenes Epoch 7",
35
+ "evaluations": {
36
+ "external_overlay_replay_v1": {
37
+ "test": {
38
+ "dataset": "kitscenes-test",
39
+ "dataset_version": "v1.0",
40
+ "input_track": "camera_only_missing_map_route",
41
+ "model_reexecuted": false,
42
+ "report_path": "evaluations/overlay-replay-v1/kitscenes-test/kitscenes-epoch-7/report.json",
43
+ "report_sha256": "d01169d68266d4b7203e92a4d25ade2a3277357ff89cc26cdb202051a26016fc",
44
+ "sample_count": 23690,
45
+ "trajectory": {
46
+ "ade_1s_m": 0.1680068780791157,
47
+ "ade_1s_sample_count": 23690,
48
+ "ade_2s_m": 0.4923481277999561,
49
+ "ade_2s_sample_count": 23690,
50
+ "ade_3s_m": 0.9830703987751742,
51
+ "ade_3s_sample_count": 23690,
52
+ "ade_5s_m": 2.469892014634966,
53
+ "ade_5s_sample_count": 23690,
54
+ "ade_6p4s_m": null,
55
+ "ade_6p4s_sample_count": 0,
56
+ "fde_1s_m": 0.37537789674250205,
57
+ "fde_1s_sample_count": 23690,
58
+ "fde_2s_m": 1.2511962416817588,
59
+ "fde_2s_sample_count": 23690,
60
+ "fde_3s_m": 2.624177628524085,
61
+ "fde_3s_sample_count": 23690,
62
+ "fde_5s_m": 6.896411433552511,
63
+ "fde_5s_sample_count": 23690,
64
+ "fde_6p4s_m": null,
65
+ "fde_6p4s_sample_count": 0,
66
+ "mean_abs_lateral_error_m": 1.4730429718483722,
67
+ "mean_abs_longitudinal_error_m": 1.6598486258851333,
68
+ "nonfinite_prediction_count": 0,
69
+ "nonfinite_prediction_rate": 0.0,
70
+ "total_timestep_count": 1516160,
71
+ "valid_horizon_coverage": 0.78125,
72
+ "valid_timestep_count": 1184500
73
+ }
74
+ },
75
+ "val": {
76
+ "dataset": "kitscenes-val",
77
+ "dataset_version": "v3.5",
78
+ "input_track": "camera_map_route",
79
+ "model_reexecuted": false,
80
+ "report_path": "evaluations/overlay-replay-v1/kitscenes-val/kitscenes-epoch-7/report.json",
81
+ "report_sha256": "832b9ec0648b997f5b1f825296269905e5ee3b13ae75918e0fdee5733dbd1818",
82
+ "sample_count": 11035,
83
+ "trajectory": {
84
+ "ade_1s_m": 0.13467869638959368,
85
+ "ade_1s_sample_count": 11035,
86
+ "ade_2s_m": 0.3886152812418257,
87
+ "ade_2s_sample_count": 11035,
88
+ "ade_3s_m": 0.8034680158738138,
89
+ "ade_3s_sample_count": 11035,
90
+ "ade_5s_m": 2.0951617386340757,
91
+ "ade_5s_sample_count": 11035,
92
+ "ade_6p4s_m": null,
93
+ "ade_6p4s_sample_count": 0,
94
+ "fde_1s_m": 0.2830596284286022,
95
+ "fde_1s_sample_count": 11035,
96
+ "fde_2s_m": 1.0101467772020263,
97
+ "fde_2s_sample_count": 11035,
98
+ "fde_3s_m": 2.212711069790632,
99
+ "fde_3s_sample_count": 11035,
100
+ "fde_5s_m": 5.9508777011937015,
101
+ "fde_5s_sample_count": 11035,
102
+ "fde_6p4s_m": null,
103
+ "fde_6p4s_sample_count": 0,
104
+ "mean_abs_lateral_error_m": 1.0219872821653013,
105
+ "mean_abs_longitudinal_error_m": 1.565857085189425,
106
+ "nonfinite_prediction_count": 0,
107
+ "nonfinite_prediction_rate": 0.0,
108
+ "total_timestep_count": 706240,
109
+ "valid_horizon_coverage": 0.78125,
110
+ "valid_timestep_count": 551750
111
+ }
112
+ }
113
+ },
114
+ "internal_validation": {
115
+ "ade_6p4s_m": 2.754791687592786,
116
+ "comfortable_rate": 0.7465968586387435,
117
+ "drivable_area_compliance_rate": 0.7988219895287958,
118
+ "drivable_area_success_rate": 0.41387434554973823,
119
+ "fde_6p4s_m": 7.857446600512373,
120
+ "route_corridor_compliance_rate": 0.42360335795259363,
121
+ "route_corridor_iou": 0.9990024728541969,
122
+ "route_corridor_success_rate": 0.07626245276537272
123
+ }
124
+ },
125
+ "license_notice": {
126
+ "checkpoint": "Other; see the repository model card.",
127
+ "pretrained_component": "Official BEVFormer V2 R50 T8 initialization; weight license recorded as NOASSERTION.",
128
+ "source_code": "Apache-2.0",
129
+ "training_data_notice": "The pretrained component was trained on nuScenes, whose terms may impose additional restrictions."
130
+ },
131
+ "limitations": [
132
+ "KITScenes Val uses an oracle post-hoc route reconstructed from the logged future trajectory.",
133
+ "KITScenes Val and Test are different scene populations and are not a causal map or route ablation.",
134
+ "External overlay replay evaluates published controls and does not re-run the model.",
135
+ "Confidence intervals and per-sample route counterfactuals are not included.",
136
+ "These open-loop checkpoints are not validated for safety-critical deployment."
137
+ ],
138
+ "lineage": {
139
+ "parent": "nuPlan Epoch 5 checkpoint ca8b43d7a777",
140
+ "training_stage": "KITScenes fine-tuning with frozen camera BEV"
141
+ },
142
+ "mlflow_registry": {
143
+ "external_overlay_test_run_id": "baa85b55d3cb46e9ba727df6979ca999",
144
+ "external_overlay_val_run_id": "730de8db5fb040f2bfb4c78c921913f1",
145
+ "internal_validation_run_id": "a313c95024b04cafa748832e5e532c5c",
146
+ "model_version": "67",
147
+ "registered_model_name": "auto-e2e-driving-policy"
148
+ },
149
+ "model_id": "kitscenes-epoch-7",
150
+ "published_at": "2026-09-16T02:26:06+00:00",
151
+ "repository": "AutowareFoundation/auto_e2e",
152
+ "schema_version": "auto_e2e_checkpoint_metadata_v1"
153
+ }
models/nuplan-epoch-4/checkpoint.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ed00e072471aae51b4cf48fa54c3fd9c03c61e0fd6e2978829cd9068dfd6da68
3
+ size 334907944
models/nuplan-epoch-4/metadata.json ADDED
@@ -0,0 +1,151 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architecture": {
3
+ "enable_bev_segmentation": true,
4
+ "enable_reasoning": false,
5
+ "enable_route_conditioning": true,
6
+ "enable_route_reconstruction": true,
7
+ "enable_world_model": false,
8
+ "freeze_bevformer": true,
9
+ "identifier": "bevformer_v2_t8_split_navigation_v5",
10
+ "num_views": 6
11
+ },
12
+ "checkpoint": {
13
+ "config_sha256": "e06fb3943e32bdab7c8b53f597ec216d5636591a73a7206045a465ffa49d1ea0",
14
+ "contains": [
15
+ "config",
16
+ "config_sha256",
17
+ "epoch",
18
+ "metrics",
19
+ "model_state_dict",
20
+ "model_state_sha256",
21
+ "optimizer_state_dict",
22
+ "scheduler_state_dict",
23
+ "training_state"
24
+ ],
25
+ "epoch": 4,
26
+ "format": "PyTorch training checkpoint",
27
+ "metrics_sha256": null,
28
+ "model_state_sha256": "1a3f159c864e6098d633f8c68596934b7f4678fcaee5ec3d40ae8eec14e650c4",
29
+ "path": "models/nuplan-epoch-4/checkpoint.pt",
30
+ "sha256": "ed00e072471aae51b4cf48fa54c3fd9c03c61e0fd6e2978829cd9068dfd6da68",
31
+ "size_bytes": 334907944
32
+ },
33
+ "display_name": "nuPlan Epoch 4",
34
+ "evaluations": {
35
+ "external_overlay_replay_v1": {
36
+ "test": {
37
+ "dataset": "kitscenes-test",
38
+ "dataset_version": "v1.0",
39
+ "input_track": "camera_only_missing_map_route",
40
+ "model_reexecuted": false,
41
+ "report_path": "evaluations/overlay-replay-v1/kitscenes-test/nuplan-epoch-4/report.json",
42
+ "report_sha256": "5eeb68a4013a9777ba927030b73877a1789f5b4eb9e14f6da760bef7f9b42808",
43
+ "sample_count": 23690,
44
+ "trajectory": {
45
+ "ade_1s_m": 0.1915564095529401,
46
+ "ade_1s_sample_count": 23690,
47
+ "ade_2s_m": 0.4816506897179045,
48
+ "ade_2s_sample_count": 23690,
49
+ "ade_3s_m": 0.9181210775291644,
50
+ "ade_3s_sample_count": 23690,
51
+ "ade_5s_m": 2.2840408150913345,
52
+ "ade_5s_sample_count": 23690,
53
+ "ade_6p4s_m": null,
54
+ "ade_6p4s_sample_count": 0,
55
+ "fde_1s_m": 0.3898781367591937,
56
+ "fde_1s_sample_count": 23690,
57
+ "fde_2s_m": 1.1514432707206264,
58
+ "fde_2s_sample_count": 23690,
59
+ "fde_3s_m": 2.38952477957317,
60
+ "fde_3s_sample_count": 23690,
61
+ "fde_5s_m": 6.42657723546563,
62
+ "fde_5s_sample_count": 23690,
63
+ "fde_6p4s_m": null,
64
+ "fde_6p4s_sample_count": 0,
65
+ "mean_abs_lateral_error_m": 1.2167123846906904,
66
+ "mean_abs_longitudinal_error_m": 1.6287253737215717,
67
+ "nonfinite_prediction_count": 0,
68
+ "nonfinite_prediction_rate": 0.0,
69
+ "total_timestep_count": 1516160,
70
+ "valid_horizon_coverage": 0.78125,
71
+ "valid_timestep_count": 1184500
72
+ }
73
+ },
74
+ "val": {
75
+ "dataset": "kitscenes-val",
76
+ "dataset_version": "v3.5",
77
+ "input_track": "camera_map_route",
78
+ "model_reexecuted": false,
79
+ "report_path": "evaluations/overlay-replay-v1/kitscenes-val/nuplan-epoch-4/report.json",
80
+ "report_sha256": "1ffde86560b8d9701c23c98b6b05b0de2611446a8d92f0fb90a08ccb884d1acb",
81
+ "sample_count": 11035,
82
+ "trajectory": {
83
+ "ade_1s_m": 0.20299996935773157,
84
+ "ade_1s_sample_count": 11035,
85
+ "ade_2s_m": 0.5018003104886625,
86
+ "ade_2s_sample_count": 11035,
87
+ "ade_3s_m": 0.950258079868937,
88
+ "ade_3s_sample_count": 11035,
89
+ "ade_5s_m": 2.291857949000488,
90
+ "ade_5s_sample_count": 11035,
91
+ "ade_6p4s_m": null,
92
+ "ade_6p4s_sample_count": 0,
93
+ "fde_1s_m": 0.40659580015215774,
94
+ "fde_1s_sample_count": 11035,
95
+ "fde_2s_m": 1.1934783774143614,
96
+ "fde_2s_sample_count": 11035,
97
+ "fde_3s_m": 2.450842261185858,
98
+ "fde_3s_sample_count": 11035,
99
+ "fde_5s_m": 6.22721162418439,
100
+ "fde_5s_sample_count": 11035,
101
+ "fde_6p4s_m": null,
102
+ "fde_6p4s_sample_count": 0,
103
+ "mean_abs_lateral_error_m": 1.2082210980047907,
104
+ "mean_abs_longitudinal_error_m": 1.6447808719447803,
105
+ "nonfinite_prediction_count": 0,
106
+ "nonfinite_prediction_rate": 0.0,
107
+ "total_timestep_count": 706240,
108
+ "valid_horizon_coverage": 0.78125,
109
+ "valid_timestep_count": 551750
110
+ }
111
+ }
112
+ },
113
+ "internal_validation": {
114
+ "ade_6p4s_m": 1.2578079247687128,
115
+ "bev_dynamic_macro_average_precision": 0.006642145206117095,
116
+ "bev_static_macro_average_precision": 0.14298715599094136,
117
+ "bev_vehicle_average_precision": 0.017910564961292577,
118
+ "bev_vulnerable_road_user_average_precision": 0.0011931259995499048,
119
+ "fde_6p4s_m": 3.845009753582417,
120
+ "route_corridor_iou": 0.9911965199680612
121
+ }
122
+ },
123
+ "license_notice": {
124
+ "checkpoint": "Other; see the repository model card.",
125
+ "pretrained_component": "Official BEVFormer V2 R50 T8 initialization; weight license recorded as NOASSERTION.",
126
+ "source_code": "Apache-2.0",
127
+ "training_data_notice": "The pretrained component was trained on nuScenes, whose terms may impose additional restrictions."
128
+ },
129
+ "limitations": [
130
+ "KITScenes Val uses an oracle post-hoc route reconstructed from the logged future trajectory.",
131
+ "KITScenes Val and Test are different scene populations and are not a causal map or route ablation.",
132
+ "External overlay replay evaluates published controls and does not re-run the model.",
133
+ "Confidence intervals and per-sample route counterfactuals are not included.",
134
+ "These open-loop checkpoints are not validated for safety-critical deployment."
135
+ ],
136
+ "lineage": {
137
+ "parent": "Official BEVFormer V2 R50 T8 initialization",
138
+ "training_stage": "nuPlan trajectory and route training with frozen camera BEV"
139
+ },
140
+ "mlflow_registry": {
141
+ "external_overlay_test_run_id": "cf984b0f7b8846a3919444ccdf3191cc",
142
+ "external_overlay_val_run_id": "ed3cf683849f4ddbaa55672502885eb1",
143
+ "internal_validation_run_id": "77a3ea44ea7440a682e6b2558c71f32f",
144
+ "model_version": "64",
145
+ "registered_model_name": "auto-e2e-driving-policy"
146
+ },
147
+ "model_id": "nuplan-epoch-4",
148
+ "published_at": "2026-09-16T02:26:06+00:00",
149
+ "repository": "AutowareFoundation/auto_e2e",
150
+ "schema_version": "auto_e2e_checkpoint_metadata_v1"
151
+ }
models/nuplan-epoch-5/checkpoint.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ca8b43d7a777d6fd9195bb253d1452f3cd645f7aab3bd0e36b1f74ffb31df29b
3
+ size 334907944
models/nuplan-epoch-5/metadata.json ADDED
@@ -0,0 +1,151 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architecture": {
3
+ "enable_bev_segmentation": true,
4
+ "enable_reasoning": false,
5
+ "enable_route_conditioning": true,
6
+ "enable_route_reconstruction": true,
7
+ "enable_world_model": false,
8
+ "freeze_bevformer": true,
9
+ "identifier": "bevformer_v2_t8_split_navigation_v5",
10
+ "num_views": 6
11
+ },
12
+ "checkpoint": {
13
+ "config_sha256": "e06fb3943e32bdab7c8b53f597ec216d5636591a73a7206045a465ffa49d1ea0",
14
+ "contains": [
15
+ "config",
16
+ "config_sha256",
17
+ "epoch",
18
+ "metrics",
19
+ "model_state_dict",
20
+ "model_state_sha256",
21
+ "optimizer_state_dict",
22
+ "scheduler_state_dict",
23
+ "training_state"
24
+ ],
25
+ "epoch": 5,
26
+ "format": "PyTorch training checkpoint",
27
+ "metrics_sha256": null,
28
+ "model_state_sha256": "30ad1af5ab1ddfa14699f95751358608f929fb8c006f8f7e1347008179c90d9e",
29
+ "path": "models/nuplan-epoch-5/checkpoint.pt",
30
+ "sha256": "ca8b43d7a777d6fd9195bb253d1452f3cd645f7aab3bd0e36b1f74ffb31df29b",
31
+ "size_bytes": 334907944
32
+ },
33
+ "display_name": "nuPlan Epoch 5",
34
+ "evaluations": {
35
+ "external_overlay_replay_v1": {
36
+ "test": {
37
+ "dataset": "kitscenes-test",
38
+ "dataset_version": "v1.0",
39
+ "input_track": "camera_only_missing_map_route",
40
+ "model_reexecuted": false,
41
+ "report_path": "evaluations/overlay-replay-v1/kitscenes-test/nuplan-epoch-5/report.json",
42
+ "report_sha256": "fc5b19beea416706461b535314a497d6a2b929d97668e8475638ae04a65221b7",
43
+ "sample_count": 23690,
44
+ "trajectory": {
45
+ "ade_1s_m": 0.16017068059452302,
46
+ "ade_1s_sample_count": 23690,
47
+ "ade_2s_m": 0.3979342750750205,
48
+ "ade_2s_sample_count": 23690,
49
+ "ade_3s_m": 0.7923739682724033,
50
+ "ade_3s_sample_count": 23690,
51
+ "ade_5s_m": 2.1119916453019565,
52
+ "ade_5s_sample_count": 23690,
53
+ "ade_6p4s_m": null,
54
+ "ade_6p4s_sample_count": 0,
55
+ "fde_1s_m": 0.3134281782098752,
56
+ "fde_1s_sample_count": 23690,
57
+ "fde_2s_m": 0.9727075050905802,
58
+ "fde_2s_sample_count": 23690,
59
+ "fde_3s_m": 2.1621481640709304,
60
+ "fde_3s_sample_count": 23690,
61
+ "fde_5s_m": 6.175121158406895,
62
+ "fde_5s_sample_count": 23690,
63
+ "fde_6p4s_m": null,
64
+ "fde_6p4s_sample_count": 0,
65
+ "mean_abs_lateral_error_m": 1.0584339709055983,
66
+ "mean_abs_longitudinal_error_m": 1.5495691251398722,
67
+ "nonfinite_prediction_count": 0,
68
+ "nonfinite_prediction_rate": 0.0,
69
+ "total_timestep_count": 1516160,
70
+ "valid_horizon_coverage": 0.78125,
71
+ "valid_timestep_count": 1184500
72
+ }
73
+ },
74
+ "val": {
75
+ "dataset": "kitscenes-val",
76
+ "dataset_version": "v3.5",
77
+ "input_track": "camera_map_route",
78
+ "model_reexecuted": false,
79
+ "report_path": "evaluations/overlay-replay-v1/kitscenes-val/nuplan-epoch-5/report.json",
80
+ "report_sha256": "e45996448f95f193acfc225c37ff169fab64bb24b5ea7717928a20c426e7303b",
81
+ "sample_count": 11035,
82
+ "trajectory": {
83
+ "ade_1s_m": 0.19511089635939421,
84
+ "ade_1s_sample_count": 11035,
85
+ "ade_2s_m": 0.4722640826416731,
86
+ "ade_2s_sample_count": 11035,
87
+ "ade_3s_m": 0.9147207671270287,
88
+ "ade_3s_sample_count": 11035,
89
+ "ade_5s_m": 2.295116252818514,
90
+ "ade_5s_sample_count": 11035,
91
+ "ade_6p4s_m": null,
92
+ "ade_6p4s_sample_count": 0,
93
+ "fde_1s_m": 0.37794148166995123,
94
+ "fde_1s_sample_count": 11035,
95
+ "fde_2s_m": 1.1331487579302755,
96
+ "fde_2s_sample_count": 11035,
97
+ "fde_3s_m": 2.422637476977514,
98
+ "fde_3s_sample_count": 11035,
99
+ "fde_5s_m": 6.388465638260406,
100
+ "fde_5s_sample_count": 11035,
101
+ "fde_6p4s_m": null,
102
+ "fde_6p4s_sample_count": 0,
103
+ "mean_abs_lateral_error_m": 1.19114795667851,
104
+ "mean_abs_longitudinal_error_m": 1.6633371994012054,
105
+ "nonfinite_prediction_count": 0,
106
+ "nonfinite_prediction_rate": 0.0,
107
+ "total_timestep_count": 706240,
108
+ "valid_horizon_coverage": 0.78125,
109
+ "valid_timestep_count": 551750
110
+ }
111
+ }
112
+ },
113
+ "internal_validation": {
114
+ "ade_6p4s_m": 1.2667837238223,
115
+ "bev_dynamic_macro_average_precision": 0.006642125301720789,
116
+ "bev_static_macro_average_precision": 0.14298718688799092,
117
+ "bev_vehicle_average_precision": 0.017910500586346273,
118
+ "bev_vulnerable_road_user_average_precision": 0.0011931349457863422,
119
+ "fde_6p4s_m": 3.8982392455945956,
120
+ "route_corridor_iou": 0.9980338574990057
121
+ }
122
+ },
123
+ "license_notice": {
124
+ "checkpoint": "Other; see the repository model card.",
125
+ "pretrained_component": "Official BEVFormer V2 R50 T8 initialization; weight license recorded as NOASSERTION.",
126
+ "source_code": "Apache-2.0",
127
+ "training_data_notice": "The pretrained component was trained on nuScenes, whose terms may impose additional restrictions."
128
+ },
129
+ "limitations": [
130
+ "KITScenes Val uses an oracle post-hoc route reconstructed from the logged future trajectory.",
131
+ "KITScenes Val and Test are different scene populations and are not a causal map or route ablation.",
132
+ "External overlay replay evaluates published controls and does not re-run the model.",
133
+ "Confidence intervals and per-sample route counterfactuals are not included.",
134
+ "These open-loop checkpoints are not validated for safety-critical deployment."
135
+ ],
136
+ "lineage": {
137
+ "parent": "Official BEVFormer V2 R50 T8 initialization",
138
+ "training_stage": "nuPlan trajectory and route training with frozen camera BEV"
139
+ },
140
+ "mlflow_registry": {
141
+ "external_overlay_test_run_id": "db91565f5e4741e48bf242e7b3e86a02",
142
+ "external_overlay_val_run_id": "0f9d11bd016448ac94d5095b7225e270",
143
+ "internal_validation_run_id": "3d419166635a44edacf8503fe94d4805",
144
+ "model_version": "65",
145
+ "registered_model_name": "auto-e2e-driving-policy"
146
+ },
147
+ "model_id": "nuplan-epoch-5",
148
+ "published_at": "2026-09-16T02:26:06+00:00",
149
+ "repository": "AutowareFoundation/auto_e2e",
150
+ "schema_version": "auto_e2e_checkpoint_metadata_v1"
151
+ }
release_manifest.json ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "external_overlay_replay_index_sha256": "be1cec5f77a7a12326070503dbcd1004c63f54450cf1151c763a215816d838bd",
3
+ "models": [
4
+ {
5
+ "checkpoint_path": "models/nuplan-epoch-4/checkpoint.pt",
6
+ "checkpoint_sha256": "ed00e072471aae51b4cf48fa54c3fd9c03c61e0fd6e2978829cd9068dfd6da68",
7
+ "checkpoint_size_bytes": 334907944,
8
+ "display_name": "nuPlan Epoch 4",
9
+ "mlflow_model_version": "64",
10
+ "model_id": "nuplan-epoch-4"
11
+ },
12
+ {
13
+ "checkpoint_path": "models/nuplan-epoch-5/checkpoint.pt",
14
+ "checkpoint_sha256": "ca8b43d7a777d6fd9195bb253d1452f3cd645f7aab3bd0e36b1f74ffb31df29b",
15
+ "checkpoint_size_bytes": 334907944,
16
+ "display_name": "nuPlan Epoch 5",
17
+ "mlflow_model_version": "65",
18
+ "model_id": "nuplan-epoch-5"
19
+ },
20
+ {
21
+ "checkpoint_path": "models/kitscenes-epoch-5/checkpoint.pt",
22
+ "checkpoint_sha256": "120a21639d9767d512eea645be0a83b01476c9e0366a8b957ac51b7ead1bd762",
23
+ "checkpoint_size_bytes": 334754024,
24
+ "display_name": "KITScenes Epoch 5",
25
+ "mlflow_model_version": "66",
26
+ "model_id": "kitscenes-epoch-5"
27
+ },
28
+ {
29
+ "checkpoint_path": "models/kitscenes-epoch-7/checkpoint.pt",
30
+ "checkpoint_sha256": "a1e6b1621018740485b3e1e43dda23713d913667eb39a65b77fcbf895dcbcefd",
31
+ "checkpoint_size_bytes": 334754024,
32
+ "display_name": "KITScenes Epoch 7",
33
+ "mlflow_model_version": "67",
34
+ "model_id": "kitscenes-epoch-7"
35
+ }
36
+ ],
37
+ "published_at": "2026-09-16T02:26:06+00:00",
38
+ "repository": "AutowareFoundation/auto_e2e",
39
+ "schema_version": "auto_e2e_hf_release_manifest_v1"
40
+ }