Spaces:
Running
Running
Commit History
-
Cache-bust CSS/JS on every container start
A browser cache miss between deploys served the old app.js with the new /api/bess shape, producing nonsense state-dropdown options (fetched_at, source, states, warnings). Render index.html at startup with the script and stylesheet URLs carrying a version query string derived from int(time.time()). Each container restart produces a fresh value, so the browser always refetches assets that paired with this build of the API. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>9f09117 -
Fix HF deployment serving stale BESS list with fictional DUIDs
The HuggingFace Space was returning a hand-written fallback bess_list.json on every /api/bess call because AEMO's WAF on www.aemo.com.au returns 403 to requests from the HF egress IP range (verified in the HF container logs — UA-spoofing alone doesn't help, the block is IP-based). The fallback had stale fictional DUIDs (NESBESS1, HORNSDALE_PWR1, etc.) and no capacity_mwh field, which caused every BESS to show "? MWh" in the dropdown and every SCADA/dispatch lookup to 404. Five fixes shipped together: 1. Send proper browser headers on the AEMO XLSX request (low-confidence shot — the block is IP-based, but no reason not to try). 2. Bake a fresh bess_list.json snapshot into the repo daily via a new GitHub Action (.github/workflows/refresh_bess_list.yml) running scripts/refresh_bess_list.py from GH runners (egress NOT on AEMO's blocklist). The bundled snapshot now has 38 real BESS units with real DUIDs and capacity_mwh values. 3. Mirror the raw XLSX through raw.githubuse…0d542b0
-
Fix era-aware file matching for pre-Apr 2025 SCADA data and add warnings system
Era 1 (before Apr 29 2025) FPPMW files use the settlement date (D) in filenames, not the publication date (D+1). The previous code always searched with D+1, finding incorrect or no data for March/April 2025. Now searches with [D, D+1] for Era 1 to cover the full 04:00–04:00 trading day from midnight-to-midnight files. Also adds a warnings system: FetchResult dataclass carries user-facing warnings (e.g. Era 2 dates with only 12h of data), surfaced as amber boxes in the UI and X-Data-Warnings headers on downloads. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>c6457f4
-
Replace inline expand with modal for data-type About panels
Claude- Remove click-to-expand inline panels that pushed page content down - Hover over ▶ About still shows a brief one-line tooltip (CSS only) - Click opens a centred modal with the full column definitions; closes via ✕ button, backdrop click, or Escape key - Modal content defined in JS (ABOUT map) to keep HTML clean https://claude.ai/code/session_01TrCrzZmWbQscJiAQaW7Rg843cb5cc -
Add hoverable/expandable About panels to data-type checkboxes
Claude- Remove sub-labels (Power, MW) and ⓘ info icon from checkbox rows - Add ▶ About toggle button next to each checkbox label; hover shows a compact tooltip, click expands an inline column-definition panel - Stack same-section result panels flush (card-group, no gap) while keeping clear spacing between top-level sections (Select Data, SCADA results, Energy results, About this app) https://claude.ai/code/session_01TrCrzZmWbQscJiAQaW7Rg8134bb77
-
Add SCADA data source & column reference (same component as 5-min energy)
ClaudeAdds a collapsible <details class="dataset-info"> accordion inside the SCADA Summary Statistics panel, matching the existing 5-minute energy component in structure and CSS classes (.col-defs / .col-def / .col-name / .col-desc — no new styles needed). The accordion is labelled "About this dataset — PUBLIC_NEXT_DAY_FPPMW" and expands to show: Column Descriptions: INTERVAL_DATETIME — target 4-second grid timestamp (YYYY/MM/DD HH:MM:SS) MEASUREMENT_DATETIME — actual AEMO telemetry recording time (may differ due to jitter) MEASURED_MW — instantaneous power output (MW); signed +/- for BESS MW_QUALITY_FLAG — numeric data-health code; live vs substituted reading MW_QUALITY_FLAG Values (from AEMO FPP Factor Calculation Guide): 0 Good — valid live telemetry (green) 1 Substituted/Estimated — previous known-good value held or interpolated (orange) 2 Bad — communication/hardware fault; exclude from analysis (red) 3 Manual …3af727f -
Fix chart width (Plotly hidden-container) and add section spacing
ClaudeChart width fix (app.js): Plotly.react() was being called while results-box, scada-results, and energy-results all had display:none. Plotly measured a zero-width container and locked in that dimension even after the divs were revealed. Fix: unhide resultsBox, then the section div, then call the render function — in that order — so Plotly measures the true container width. Section spacing (style.css): #results-box: display:flex / flex-direction:column / gap:3rem gives a clear visual break between the SCADA section and the Energy section when both are loaded. Hidden (.hidden = display:none) children produce no gap, so there is no unwanted whitespace when only one section is visible. #scada-results, #energy-results: same flex-column / gap:1.5rem so the panels within each section (stats, chart, download) are properly spaced. https://claude.ai/code/session_01TrCrzZmWbQscJiAQaW7Rg89845df1 -
Add inline help for dispatch energy data (tooltip icons + collapsible panel)
ClaudeOption 1 — ⓘ tooltip icons: - Added to the '5-minute Dispatch' checkbox label (tooltip appears below since it's near the top of the page, using .tip-below variant) - Added inline next to INITIALMW, INITIAL_ENERGY_STORAGE, and ENERGY_STORAGE in the Download Energy section; each tooltip carries a concise one-sentence description of the column's meaning Option 2 — collapsible 'About this dataset' panel: - <details>/<summary> block placed inside the Dispatch Energy Summary Statistics panel, above the stats grid - Expanded content includes: · Data source paragraph with hyperlinks to both NEMWEB Current and Archive directories for Next_Day_Dispatch · Three-row column definition list (col-name + col-desc in a CSS grid) for INITIALMW, INITIAL_ENERGY_STORAGE, ENERGY_STORAGE, using the full descriptions from AEMO's data model documentation - Collapsed by default; animated ▶ arrow rotates on open; summary text brightens on hover/open CSS additions (style.css)…1b5231e -
Add 5-minute dispatch energy data, data-type checkboxes, remove preview table
ClaudeNew data source — BESS Initial Energy (MWh): - config.py: add DISPATCH_CURRENT_URL, DISPATCH_ARCHIVE_URL, DISPATCH_START_DATE (2025-02-11), DISPATCH_COLUMNS, and TIMING_DEFAULT_DISPATCH_CURRENT/ARCHIVE_SEC constants - dispatch_fetcher.py: new service that fetches Next_Day_Dispatch ZIPs from NEMWEB Current (daily files: YYYYMMDD = settlement date) or Archive (monthly bundles, selected by largest start-date <= target); uses remotezip for archives; returns (csv_bytes, source_type) - dispatch_processor.py: new service that parses the AEMO C/I/D CSV format, extracts only the UNIT_SOLUTION table, filters by DUID and NEM trading-day window, and exposes compute_dispatch_summary(), to_csv_bytes(), to_parquet_bytes(), to_json_records() - analytics.py: extend _TIMING_DEFAULTS dict to cover dispatch_current and dispatch_archive; get_timing_estimate() now handles all 4 types - api.py: add GET /api/energy-data, /api/download/energy-csv, /api/download/energy-parquet; update /api/i…2f1129f -
Add learned wait-time estimates and stretch chart to fill frame
ClaudeTiming estimates (backend): - config.py: add FPPMW_CUTOVER_DATE, TIMING_MIN_SAMPLES, TIMING_DEFAULT_CURRENT_SEC / ARCHIVE_SEC as named constants - analytics.py: migrate schema to add duration_ms + source_type columns; extend log_request() to accept them; add get_timing_estimate() which returns p75 of the last 100 successful view durations per source_type, falling back to hardcoded defaults until >= TIMING_MIN_SAMPLES records exist - api.py: time fetch + process pipeline in all three data handlers; classify each request as 'current' (>= 2026-01-11) or 'archive'; pass duration_ms + source_type to log_request; expose get_timing_estimate results and cutover_date in /api/info Dynamic loading message (frontend): - app.js: store cutover_date and estimates from /api/info on boot; add formatEstimate() helper that converts seconds to human-readable bands and appends sample count when the estimate is learned (not default); replace hardcoded loading strings with the dynamic es…e6a4103
-
Catch all network errors; show HTTP status on frontend error
ClaudeTwo defensive fixes to expose the real failure cause: 1. aemo_fetcher.py: add except httpx.HTTPError fallback in both _list_directory and fetch_csv_for_date so that ConnectError, RemoteProtocolError, and other non-timeout/non-HTTP errors are converted to AEMOFetchError (JSON 503) rather than propagating as unhandled exceptions that could produce non-JSON responses. 2. app.js: replace the generic 'Unknown error' fallback with 'Server error <status> (no details available)' so the HTTP status code is always visible in the UI when the JSON body cannot be parsed (e.g. a 502/504 proxy timeout returning an HTML error page). https://claude.ai/code/session_01Vm5dTYdRf99LZ9MfB86vjxd707685 -
Improve loading time estimate and add disclaimer
Claude- Loading message is now date-aware: dates from 11 Jan 2026 onward show a '2–3 minutes' estimate; older dates warn of '5–10 minutes or more' due to large NEMWEB archive files, replacing the misleading '10–20 s'. - Added a legal disclaimer paragraph to the footer covering data accuracy, liability, and directing users to verify with AEMO directly. https://claude.ai/code/session_01Vm5dTYdRf99LZ9MfB86vjx12d7321 -
Unify hyperlink colour and add FPP source to footer
Claude- Extend the footer link style (var(--accent) sky-blue, no underline / underline on hover) to .panel a so the 'About This App' links match. - Add missing Frequency Performance Payments (FPP) scheme link to the footer alongside the existing FPPDAILY and NEM Generation Information sources. Also reword 'Data source:' to 'SCADA data:' for clarity. https://claude.ai/code/session_01Vm5dTYdRf99LZ9MfB86vjx0a43438 -
Disable all form controls while data is loading
ClaudeIntroduce setFormLocked(locked) which disables/re-enables selState, selBess, inpDate and btnLoad atomically. Called with true before the fetch and false in the finally block, so all controls are restored regardless of success or error. https://claude.ai/code/session_01Vm5dTYdRf99LZ9MfB86vjx95bc2d3 -
Fix BESS columns, dropdown format, and date picker UX
Claudegen_info_fetcher.py: - Name column: 'Station Name' -> 'Unit Name' (row 4 column name) - MW column: 'Reg Cap' -> 'Unit Capacity (MW AC)' - Add MWh: 'Agg Nameplate Storage Capacity (MWh)' -> capacity_mwh field index.html: - Date input starts disabled; enabled only after a BESS unit is selected - Label 'Battery (BESS)' -> 'BESS Unit' app.js: - Dropdown options now show: 'DUID (Name) (MW / MWh)' - onBessChange() enables date picker and Load button when BESS is chosen - Changing state resets date picker to disabled again https://claude.ai/code/session_01Vm5dTYdRf99LZ9MfB86vjxd31601b -
Initial implementation: NEM Battery SCADA Data Explorer
ClaudeFull-stack web app for exploring 4-second BESS SCADA data from AEMO NEMWEB: - FastAPI backend with async AEMO ZIP fetching, Polars-based CSV filtering, Parquet/CSV download endpoints, and SQLite analytics logging - Vanilla JS frontend with Plotly.js interactive time-series chart, state→BESS→date selection flow, data quality breakdown, and paginated table - Dockerfile configured for HuggingFace Spaces (port 7860) - Static BESS list (NSW/VIC/QLD/SA) with real DUIDs from AEMO NEM data - MW_QUALITY_FLAG explained to users (Good/Suspect/Bad) - IP-based analytics logging without user accounts https://claude.ai/code/session_01Vm5dTYdRf99LZ9MfB86vjxfd0bd3b