Spaces:
Running on Zero
Running on Zero
| # Data Dictionary — Member 4 | |
| Companion to `reports/phase2_data_preparation.md`. Column semantics from the | |
| Phase 1/2 audits; label semantics marked where unconfirmed. | |
| ## Part 1 — DTC-FM Near-Miss Dataset | |
| Processed file: `data/processed/near_miss/dtc_fm_combined.csv` (+ `.parquet` exact-float copy). | |
| | Column | Type | Unit | Meaning | Missing | Origin | Leakage concern | Deployment concern | | |
| |---|---|---|---|---|---|---|---| | |
| | sample_index | int64 | — | **Internal processing index 0..2507 (row position). NOT a real dataset ID.** | 0 | generated Phase 2 | none | none | | |
| | speed_object_1_kph | int64 | km/h | Speed of object 1 in the interaction | 0 | original | none | none | | |
| | speed_object_2_kph | int64 | km/h | Speed of object 2 in the interaction | 0 | original | none | none | | |
| | angle_degrees | float64 | degrees | Encounter angle between objects (0.01–89.98) | 0 | original | none | none | | |
| | PET | float64 | s | Post-Encroachment Time between object pairs (0.53–3.00) | 0 | original | none statistical | **POST-EVENT: only measurable after the interaction completes — unavailable for real-time decisions** | | |
| | acceleration_obj1_mps2 | float64 | m/s² | Longitudinal acceleration of object 1 (−5.0–5.0, quantized ~0.56 steps) | 0 | original | none | none | | |
| | acceleration_obj2_mps2 | float64 | m/s² | Longitudinal acceleration of object 2 (−5.0–4.44) | 0 | original | none | none | | |
| | min_dist_dual_check_m | float64 | m | Minimum separation distance under dual checking (0.5–25.71) | 0 | original | none | none | | |
| | object_count_at_frame1 | int64 | count | Objects detected in the reference frame (5–57) | 0 | original | none | none | | |
| | target_dist_px | float64 | px | Distance between objects in camera pixel space (0–613.4) | 0 | original | none | **camera-dependent: not transferable across camera geometries** | | |
| | FE_inv_PET | float64 | 1/s | Engineered, verified `1/PET` | 0 | derived (verified) | redundancy only | inherits PET post-event limitation | | |
| | FE_safety_index | float64 | — | Engineered, **formula UNIDENTIFIED**; ≈ `(1/PET)/dist_px × unknown term`; extremes clipped at round caps (max 2.85e7) | 0 | derived (unverified) | **MEDIUM — unverified provenance** | unknown | | |
| | FE_log_mesafe | float64 | ln(px) | Engineered, verified `ln(1 + target_dist_px)` ("mesafe" = distance) | 0 | derived (verified) | redundancy only | camera-dependent | | |
| | FE_dist_squared | float64 | px² | Engineered, verified `target_dist_px²` | 0 | derived (verified) | redundancy only | camera-dependent | | |
| | class_object_1 | str | — | Object 1 type ∈ {Car, VRU, Ped, LCV, HCV} | 0 | original categorical | none | none | | |
| | class_object_2 | str | — | Object 2 type ∈ {Car, VRU, Ped, LCV} (no HCV) | 0 | original categorical | none | none | | |
| | event_type | int64 | — | **Binary target. `0 = class_0`, `1 = class_1`. Semantics UNCONFIRMED** — no label map in the files; class-conditional statistics are *consistent with* 1 = conflict/near-miss but this is not documentation-confirmed | 0 | original label | — | — | | |
| ### Notes | |
| - Class counts: class_0 = 1,867 (74.44%), class_1 = 641 (25.56%) — moderately imbalanced. | |
| - Acceleration values are quantized (~0.56 m/s² steps, likely per-frame estimation). | |
| - `class_object_1 = HCV` occurs in only 2 rows; treat as rare category in Phase 3. | |
| - No train/val/test split exists in the source; splitting is deferred to a later phase. | |
| ## Part 2 — Accident CCTV Object-Detection Dataset | |
| Canonical source: the 4 Parquet shards in `dataset/` (never copied into | |
| processed outputs). Processed indexes: `data/processed/accident/{train,validation,test}_index.parquet`. | |
| | Field | Type | Meaning | Structure | | |
| |---|---|---|---| | |
| | image | struct<bytes: binary, path: string> | CCTV frame, embedded PNG | 640×640 px; `path` is always null (bytes only) | | |
| | image.bytes | binary | PNG-encoded image | verified decodable on a sampled basis; IHDR-validated on all 2,763 | | |
| | objects | struct<bbox, category> | annotation container | one struct per image | | |
| | objects.bbox | list<fixed_size_list<float32, 4>> | per-object box, **COCO-style [x, y, w, h] absolute pixels** | 1–12 boxes/image; 100% within image bounds (x+w ≤ 640, y+h ≤ 640) | | |
| | objects.category | list<int64> | per-object class id | `0 = accident`, `1 = non_accident` (per source dataset card; map not embedded in files) | | |
| ### Index fields (processed artifact) | |
| | Column | Type | Meaning | | |
| |---|---|---| | |
| | split | str | train / validation / test | | |
| | source_file | str | original parquet shard name | | |
| | row_index | int64 | 0-based row WITHIN source_file — internal reference is `(source_file, row_index)`; **not a dataset ID** | | |
| | image_sha256 | str | content hash of the embedded PNG (dup/integrity key) | | |
| | image_format, width, height | str/int32 | IHDR-derived; all PNG 640×640 | | |
| | n_objects | int32 | annotation count for the image | | |
| | categories | list<int64> | copy of objects.category | | |
| | bbox | list<list<float32[4]>> | copy of objects.bbox (unchanged representation) | | |
| ### Usage in future pipeline | |
| - Phase 5 (accident model) reads images by resolving `(source_file, row_index)` | |
| against the canonical shards; the index supplies annotations without loading bytes. | |
| - No bounding-box normalization was performed (Phase 2 policy: preserve | |
| representation unless a downstream model requires conversion). | |
| - Image-level accident prevalence is ~99% (nearly every frame contains ≥1 | |
| accident-labeled object), so image-level classification is trivially biased; | |
| the meaningful task is object-level detection/localization. |