| # Refactor Plan & Todo – SolarWine Baseline |
|
|
| This file tracks the refactor steps for Gemini utilities, phenology, time features, IMS coverage, and performance/caching improvements. |
|
|
| --- |
|
|
| ## High Priority |
|
|
| - [x] **Unify Gemini client & JSON handling** |
| - [x] `genai_utils`: verify `get_google_api_key`, `get_genai_client`, `extract_json_object` (already created). |
| - [x] `day_ahead_advisor`: |
| - [x] Use `extract_json_object` via local `_extract_json` wrapper. |
| - [x] Confirm `api_key` and `client` now call `genai_utils`. |
| - [x] `llm_data_engineer`: |
| - [x] Replace `api_key` property with `get_google_api_key(self._api_key)`. |
| - [x] Replace `client` property with `get_genai_client(self._api_key)`. |
| - [x] Swap local `_extract_json` to use `extract_json_object`. |
| - [x] `routing_agent`: |
| - [x] Replace `api_key` property with `get_google_api_key(self._api_key)`. |
| - [x] Replace `client` property with `get_genai_client(self._api_key)`. |
| - [x] `vineyard_chatbot`: |
| - [x] Replace `api_key` property with `get_google_api_key(self._api_key)`. |
| - [x] Replace `client` property with `get_genai_client(self._api_key)`. |
| - [x] Make `has_api_key` call `get_google_api_key` in a try/except instead of duplicating logic. |
|
|
| - [x] **Centralise phenology logic** |
| - [x] `phenology.py`: confirm `estimate_stage_for_date` and `stage_id_and_description_for_date` match the biology plan and skills. |
| - [x] `app.py` (Vineyard Advisor vine snapshot): |
| - [x] Replace inline month→stage mapping with `stage_id_and_description_for_date(today)`. |
| - [x] Any future modules that reason about "current stage" should import from `phenology` instead of re-implementing month logic. |
|
|
| - [x] **IMS cache coverage & observability** |
| - [ ] `IMSClient.fetch_and_cache`: |
| - [ ] Add lightweight logging or counters for skipped chunks (date ranges that failed). |
| - [x] `app.py` – Forecasting tab: |
| - [x] Add a small IMS coverage panel that shows: |
| - [x] First timestamp in cache |
| - [x] Last timestamp in cache |
| - [x] Number of days of coverage |
| - [x] Age (minutes/hours) of the latest record |
| - [ ] `app.py` – Vineyard Advisor tab: |
| - [ ] Reuse the cached vine snapshot to also display IMS coverage (not just last point) where useful. |
|
|
| --- |
|
|
| ## Medium Priority |
|
|
| - [x] **Shared time-feature utilities** |
| - [x] `time_features.py`: confirm `add_cyclical_time_features` API (timestamp column or index). |
| - [x] `preprocessor.create_time_features`: |
| - [x] Replace manual sin/cos code with `add_cyclical_time_features(df, timestamp_col="timestamp_utc")`. |
| - [x] Keep raw `month` and `day_of_year` features as now. |
| - [x] `chronos_forecaster.load_data`: |
| - [x] Replace inline `hour_sin/cos` and `doy_sin/cos` computations with `add_cyclical_time_features(resampled, index_is_timestamp=True)`. |
| - [x] `llm_data_engineer.engineer_features`: |
| - [x] Replace its own cyclical feature block with a call to `add_cyclical_time_features` on the resolved timestamp. |
|
|
| - [ ] **Cache heavy shading simulations** |
| - [ ] `tracker_optimizer.simulate_tilt_angles`: |
| - [ ] Add a simple in-module cache (e.g. dict keyed by `(len(df), tuple(sorted(angles)))` or `(date_min, date_max, angles)` for the sample dataset). |
| - [ ] Ensure caching does not break if `df` changes (avoid caching across different data sources). |
| - [ ] `vineyard_chatbot._tool_compare_angles`: |
| - [ ] Confirm it benefits from the new `simulate_tilt_angles` cache (repeated queries reuse results). |
| - [ ] Optionally expose precomputed angle summaries in the Shading tab for instant advisor-like responses. |
|
|
| - [ ] **Use phenology helper in future logic** |
| - [ ] When adding or refactoring: |
| - [ ] Day-ahead planner |
| - [ ] Energy budget planner |
| - [ ] Operational modes / real-time control |
| - Always derive phenological stage via `phenology.estimate_stage_for_date` or `stage_id_and_description_for_date` instead of hand-written month rules. |
|
|
| --- |
|
|
| ## Low Priority (later) |
|
|
| - [ ] **Vectorise `FarquharModel.compute_all`** if Stage 1 datasets grow significantly or are needed in near real-time. |
| - [ ] **Design and implement `thingsboard_client`** according to the `thingsboard-client` skill, then gradually wire real-time vine telemetry (soil moisture, irrigation, local temps) into Vineyard Advisor and future control logic. |
| - [ ] **Add focused tests** for: |
| - [ ] `FarquharModel` (known inputs, NaN handling). |
| - [ ] `Preprocessor` (merge, split, edge cases). |
| - [ ] `SolarGeometry/ShadowModel` (tracker tilt, mask shape). |
| - [ ] `ChronosForecaster.load_data` (grid coverage, growing-season filter, time features). |
|
|
