Buckets:
| # Rp.py Optimization Campaign (GEPA "optimize anything" style) | |
| This folder contains an iterative, GEPA-style optimization campaign applied to | |
| **`Rp.py`** (the parallel telemetry-extraction variant) benchmarked against | |
| **`R.py`** (the sequential original). | |
| ## Hard requirement: byte-identical outputs | |
| The user constraint is that every variant must produce **byte-identical** | |
| output to `R.py`. This is **verified** (see `compare_outputs.py`) by running | |
| each variant into a fresh output directory and diffing every `*.json` file | |
| against the `R.py` reference byte-for-byte. | |
| **Key finding:** `Rp.py`'s per-lap path (`_lap_telemetry_or_none`) is a | |
| line-for-line mirror of FastF1's `Lap.get_telemetry()`, so the parallel | |
| baseline is already byte-identical to `R.py` (all 1368 JSON files match). | |
| ## Methodology (mirroring `optimize_anything`) | |
| This repository does not commit an API key or provider configuration, so the | |
| saved campaign is a **GEPA-style artifact/evaluator search**, not a claim that | |
| an LLM proposer was run in this checkout. Each mutation is retained as a | |
| separate text artifact, evaluated by the same executable benchmark, and | |
| accepted only when its diagnostic gate (byte identity) passes. The optional | |
| official `gepa` package can be installed when running an LLM-guided proposer; | |
| the deterministic benchmark and verification harness do not require it. | |
| 1. **Seed**: start from the existing implementations (`R.py` sequential, | |
| `Rp.py` parallel) as baseline candidates. | |
| 2. **Evaluate**: run each candidate end-to-end on the **2026 Austrian Grand | |
| Prix - Race** session, writing outputs into a *fresh* directory (`EXP_ROOT`) | |
| so the "file already exists => skip regeneration" guard never short-circuits | |
| the work. Reuse the warm FastF1 `cache/` so no network is needed. | |
| Metric = wall-clock seconds (lower is better), gated by **byte-identity vs | |
| R.py**. | |
| 3. **Propose / mutate**: apply a single, measurable optimization per experiment | |
| (each saved as its own file), re-benchmark, and verify byte-identity. | |
| 4. **Accept**: adopt the best *byte-identical* candidate as the base. | |
| Each experiment is a **separate, self-contained file** (no overwrite of `Rp.py`). | |
| ## Results (Austrian GP Race, 1,368 JSON files / 1,341 telemetry files each) | |
| | Experiment | File | Time (s) | vs R | vs Rp | Byte-identical? | | |
| |---|---|---:|---:|---:|---| | |
| | exp7_numpy_merge.py **[BYTE-IDENTICAL BEST]** | exp7_numpy_merge.py | 6.53 | 60.7x | 10.6x | YES | | |
| | exp6_numpy_drvhead.py | exp6_numpy_drvhead.py | 18.50 | 21.4x | 3.7x | YES | | |
| | exp5_saturate.py | exp5_saturate.py | 51.57 | 7.7x | 1.3x | YES | | |
| | Rp.py (parallel baseline) | run_par.py | 68.97 | 5.7x | 1.0x | YES | | |
| | R.py (sequential baseline) | run_seq.py | 396.13 | 1.0x | 0.2x | YES | | |
| | exp4_fastslice.py (NOT byte-identical) | exp4_fastslice.py | 11.11 | 35.7x | 6.2x | NO | | |
| | exp3_workers.py (NOT byte-identical) | exp3_workers.py | 27.94 | 14.2x | 2.5x | NO | | |
| | exp2_merge_once.py (NOT byte-identical) | exp2_merge_once.py | 38.94 | 10.2x | 1.8x | NO | | |
| Chart: `progress_chart.png` - raw data: `results.json` / `results.md`. | |
| ## What was optimized (and what is / is not byte-identical) | |
| The dominant per-lap cost in this pipeline is the FastF1 per-lap | |
| `get_telemetry` chain. Profiling showed its cost is concentrated in three | |
| places, each of which was replaced by an exact numpy/scipy replica in a | |
| separate byte-identical experiment: | |
| 1. **Parallelize (Rp vs R) - 5.7x**: process drivers in a `ProcessPoolExecutor` | |
| sharing the loaded session via `fork` (copy-on-write) instead of one driver | |
| at a time. Byte-identical. OK | |
| 2. **`exp5` saturate workers - 1.3x vs Rp**: keep the exact per-lap | |
| `get_telemetry` but raise the worker cap from 16 to 32 so all 22 drivers run | |
| in a single wave. Byte-identical. OK | |
| 3. **`exp6` numpy driver-ahead - 2.8x vs exp5**: FastF1's `add_driver_ahead` | |
| (~180-280 ms/lap, ~75% of per-lap time; it integrates distance for all ~19 | |
| other drivers and runs an outer join + argmin) is re-implemented in numpy. | |
| The distance integration reproduces pandas' exact float sequence (`/1e9` | |
| seconds on int64 ns, `Speed / 3.6`, float64 cumsum); the pandas "outer join | |
| on SessionTime" + per-row argmin matrix math is reproduced | |
| element-for-element. Byte-identical (all 1368 JSON files match R.py). OK | |
| 4. **`exp7` numpy merge+fill+slice - 2.4x vs exp6, the byte-identical | |
| champion**: the remaining per-lap pandas machinery - `add_distance` / | |
| `add_relative_distance`, the two `merge_channels(frequency='original')` | |
| calls (car+driver-ahead and pos+car) and the | |
| `slice_by_lap(interpolate_edges=True)` edge merge - is replaced with | |
| numpy/scipy replicas: | |
| * `add_distance`: the pandas dt sequence is reproduced exactly, including | |
| the scalar `Timedelta.total_seconds()` semantics for the first row | |
| (`days*86400 + seconds + microseconds/1e6` with floor-based components, | |
| nanoseconds dropped) which differs from the vectorized `.dt.total_seconds` | |
| used for the diff. | |
| * `merge_channels`: outer join on the int64-ns Date union with per-channel | |
| interpolation dispatch matching pandas exactly - `np.interp` for 'index' | |
| channels (Speed, RPM, Throttle, DistanceToDriverAhead), | |
| `scipy.interpolate.interp1d(kind='quadratic', fill_value='extrapolate')` | |
| with pandas' leading-NaN preservation for 'quadratic' channels (Distance, | |
| RelativeDistance, X, Y, Z), and `ffill().ffill().bfill()` for 'discrete' | |
| channels (nGear, Brake, DRS, DriverAhead, Status). | |
| * `slice_by_lap`: the 2-row edges merge plus the [start, end] mask and the | |
| `Time = SessionTime - start_time` shift. | |
| Byte-identical (all 1368 JSON files match R.py). OK | |
| The earlier `exp2`/`exp3`/`exp4` experiments tried to remove the repeated | |
| per-lap merge and are dramatically faster, but they are **NOT byte-identical** | |
| and are therefore rejected by the campaign's correctness gate: | |
| - `exp2` / `exp3` (merge-once) build the telemetry ONCE per driver and slice | |
| laps from it. The driver-wide merge timebase differs from the per-lap merge | |
| timebase, so the interpolated **`distance` / `X` / `Y` / `Z` / | |
| `RelativeDistance`** samples differ from `R.py`. NO | |
| - `exp4` (fast boolean-mask slice) additionally **drops the 2 interpolated | |
| boundary rows per lap**, so the `time` arrays have 2 fewer samples per lap. NO | |
| They are kept in this folder as exploration data (they show where the speed | |
| ceiling is if you *relax* the byte-identity requirement), but the **final | |
| accepted best is `exp7_numpy_merge.py`**, which is verified byte-identical. | |
| ## Reproducing | |
| ```bash | |
| # 1) (re)generate the experiment scripts from R.py/Rp.py if needed | |
| python experiments/builder.py # exp2..exp5 + run_seq/run_par | |
| python experiments/make_exp6.py # exp6_numpy_drvhead.py | |
| python experiments/make_exp7.py # exp7_numpy_merge.py | |
| # 2) run a single variant into a fresh output dir (warm ./cache) | |
| rm -rf /tmp/exp_out && mkdir -p /tmp/exp_out | |
| EXP_ROOT=/tmp/exp_out no_proxy='*' python experiments/exp7_numpy_merge.py | |
| # 3) benchmark accepted candidates in fresh, isolated output directories and | |
| # record timings + byte-identity results in one JSON report. The output root | |
| # is resolved to an absolute path before child processes start. | |
| .venv/bin/python experiments/benchmark.py --output-root /tmp/rp-benchmark | |
| # Include the intentionally rejected, non-byte-identical explorations too. | |
| .venv/bin/python experiments/benchmark.py --include-exploratory \ | |
| --output-root /tmp/rp-benchmark-exploratory | |
| # 4) or verify one candidate against an existing R.py-derived reference | |
| # (run_seq.py is the benchmark adapter generated from R.py) | |
| .venv/bin/python experiments/compare_outputs.py /tmp/ref /tmp/exp_out | |
| # 5) validate the fast replicas in-process (asserts every lap matches pandas) | |
| FF1_COMPARE_FAST=1 EXP_ROOT=/tmp/exp_out no_proxy='*' \ | |
| python experiments/exp6_numpy_drvhead.py | |
| FF1_COMPARE_FAST=1 EXP_ROOT=/tmp/exp_out2 no_proxy='*' \ | |
| python experiments/exp7_numpy_merge.py | |
| # 6) regenerate the checked-in summary + chart | |
| .venv/bin/python experiments/make_results.py | |
| ``` | |
| The benchmark checks that all generated candidates, including `Rf.py`, are | |
| freshly regenerated from the current `R.py`/`Rp.py` source chain before | |
| starting, then compares every candidate against the new `run_seq.py` output. By default it runs only accepted | |
| byte-identical candidates; `--include-exploratory` adds the deliberately | |
| rejected exp2/exp3/exp4 variants and returns nonzero if any candidate fails the | |
| byte-identity gate. | |
| Notes: environments without `os.fork` fall back to spawn workers (each worker | |
| reloads the session once from cache). Use the repo `.venv` (Python 3.14) which | |
| has fastf1, orjson, numpy, pandas, scipy, psutil, requests and matplotlib; all | |
| are declared in `requirements.txt`. Run | |
| one heavy variant at a time - running several 22-process experiments | |
| concurrently exhausts RAM and triggers OOM kills. Timings above are | |
| best-of/representative single-run figures from one shared session; absolute | |
| numbers vary with machine load, relative ordering does not. | |
Xet Storage Details
- Size:
- 9.09 kB
- Xet hash:
- e0c2ab8c21ef820d93a0cd01e2b5cd5c00d6cec221a5a32984428144f06ef74e
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.