File size: 3,110 Bytes
6342806
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
"""Next-Generation AV Perception Evaluation platform (Features 2-5).

This package adds four capabilities on top of the existing evaluation
substrate, plus the architecture decision deliverables under
docs/architecture/:

2. Counterfactual simulation with validity gating
   (:mod:`counterfactual`, :mod:`worldmodel`, :mod:`validity`)
3. Closed-loop behavioral evaluation + causal counterfactual replay
   (:mod:`closedloop`, :mod:`causal`)
4. Safety-informed metrics (Safety-Critical Recall, risk-weighted metrics)
   (:mod:`safety_metrics`)
5. Compute dedup + intelligent launch-eval scheduling
   (:mod:`cache`, :mod:`compute`, :mod:`scheduler`, :mod:`regression`,
   :mod:`lineage`)

Design principles enforced in code
----------------------------------
* Evidence before inference: every metric, gate and verdict here is
  deterministic math on data; nothing is decided by a model's opinion.
* Open-loop AND closed-loop metrics are complementary: closed-loop behavioral
  reports always carry the open-loop metrics (mAP/recall/IoU) alongside; no
  report replaces them.
* Simulation is evidence, not reality: every datum carries a
  :class:`sensorflow.nextgen.models.DataLabel`
  (REAL / REPLAYED / SIMULATED / GENERATED / COUNTERFACTUAL) which is carried
  through every downstream report.
* Deterministic safety policy: gate thresholds are explicit, versioned
  parameters; the same inputs always produce the same launch recommendation.
* Scale without linear recompute: the content-addressed feature cache
  (:mod:`cache`) and the anytime-valid early-stopping scheduler
  (:mod:`scheduler`, delegating to :mod:`sensorflow.seqeval`) together keep a
  100k-scenario gauntlet sublinear in full-inference cost.
* Design for deletion / reuse > extend > new. Reuse map (each documented at
  the point of use):

  - sensorflow.bevfusion   scenes/sensors/engines/evaluate: the scenario,
                           sensor-simulation and perception substrate.
                           Counterfactuals are transformations of its
                           SceneSequence; closed-loop runs its sensor models.
  - sensorflow.seqeval     ALL sequential statistics: anytime-valid e-process
                           early stopping, cluster units, paired deltas.
  - sensorflow.safety      ssam_ext TTC/DRAC/collision-probability/stopping
                           math; gate structuring conventions; ODD strata.
  - sensorflow.megaeval    distribution-shift machinery (train-vs-eval mix)
                           and metric-cube aggregation patterns.
  - sensorflow.rca.stats   PSI/JS divergence primitives (imported read-only
                           with a guarded fallback).
  - sensorflow.vitis       optional optical-flow temporal cross-check
                           (guarded import; absent -> skipped, reported).
  - sensorflow.agentic     Feature 1 (built concurrently by another agent):
                           guarded import for the AgenticSafetyScorecard
                           shape; a compatible local definition is used when
                           the package is not importable.
"""