Spaces:
Running
Running
Upload 6 files
Browse files- LICENSE +21 -0
- README.md +174 -5
- app.mjs +46 -0
- core.mjs +206 -0
- index.html +66 -17
- styles.css +5 -0
LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
MIT License
|
| 2 |
+
|
| 3 |
+
Copyright (c) 2026 Rodrigo Pantoja
|
| 4 |
+
|
| 5 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
| 6 |
+
of this software and associated documentation files (the "Software"), to deal
|
| 7 |
+
in the Software without restriction, including without limitation the rights
|
| 8 |
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
| 9 |
+
copies of the Software, and to permit persons to whom the Software is
|
| 10 |
+
furnished to do so, subject to the following conditions:
|
| 11 |
+
|
| 12 |
+
The above copyright notice and this permission notice shall be included in all
|
| 13 |
+
copies or substantial portions of the Software.
|
| 14 |
+
|
| 15 |
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
| 16 |
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
| 17 |
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
| 18 |
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 19 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 20 |
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 21 |
+
SOFTWARE.
|
README.md
CHANGED
|
@@ -1,12 +1,181 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: static
|
|
|
|
| 7 |
pinned: false
|
| 8 |
license: mit
|
| 9 |
short_description: Local-first classifier evaluation and fairness analysis
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 10 |
---
|
| 11 |
|
| 12 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: PrismAudit | Responsible ML Evaluation Studio
|
| 3 |
+
emoji: π
|
| 4 |
+
colorFrom: green
|
| 5 |
+
colorTo: blue
|
| 6 |
sdk: static
|
| 7 |
+
app_file: index.html
|
| 8 |
pinned: false
|
| 9 |
license: mit
|
| 10 |
short_description: Local-first classifier evaluation and fairness analysis
|
| 11 |
+
tags:
|
| 12 |
+
- machine-learning
|
| 13 |
+
- model-evaluation
|
| 14 |
+
- classification
|
| 15 |
+
- responsible-ai
|
| 16 |
+
- explainability
|
| 17 |
+
- fairness
|
| 18 |
+
- data-analysis
|
| 19 |
+
- javascript
|
| 20 |
---
|
| 21 |
|
| 22 |
+
# PrismAudit | Responsible ML Evaluation Studio
|
| 23 |
+
|
| 24 |
+
**Project 24 of the Hugdrigo portfolio** Β· An open-source, local-first studio for evaluating binary classification scores, understanding decision thresholds, checking probability calibration and measuring differences across user-defined cohorts.
|
| 25 |
+
|
| 26 |
+
[**Open the Space**](https://huggingface.co/spaces/Hugdrigo/prismaudit-model-lab)
|
| 27 |
+
|
| 28 |
+
   
|
| 29 |
+
|
| 30 |
+
> **Interpretation limits:** PrismAudit **does not train a machine-learning model**, predict new scores, prove fairness or detect unlawful discrimination. It evaluates *existing* binary scores against known labels. Built-in sample data are **fictional and synthetic**, not measurements of any real company, people or deployed model. Reports are descriptive and do not constitute validation for deployment or legal compliance.
|
| 31 |
+
|
| 32 |
+
## Overview
|
| 33 |
+
|
| 34 |
+
Many classification dashboards stop at accuracy, hiding important trade-offs and subgroup differences. PrismAudit turns an existing set of probability scores into an interpretable, reproducible evaluation report that runs entirely in a browser.
|
| 35 |
+
|
| 36 |
+
| Module | Functions |
|
| 37 |
+
| --- | --- |
|
| 38 |
+
| **Overview** | Accuracy, F1, ROC AUC, Brier score; confusion matrix, ROC plot, 10-bin calibration diagnostic and explainable audit signals. |
|
| 39 |
+
| **Threshold explorer** | Interactive decision cutoff (0.00β1.00); precision, recall, specificity and balanced accuracy; 21-point sensitivity sweep; highest *in-sample* F1 operating point; seeded bootstrap interval. |
|
| 40 |
+
| **Fairness lens** | Cohort decision rates, prevalence, recall, false positive rates and absolute maxβmin differences, with sample-size and missing-class warnings. |
|
| 41 |
+
| **Data & methods** | Local CSV import and schema validation, dataset profiling, synthetic example download and transparent methodology. |
|
| 42 |
+
| **Export** | Reproducible JSON audit with all aggregate metrics; CSV operating-point and cohort tables. |
|
| 43 |
+
|
| 44 |
+
### No paid infrastructure
|
| 45 |
+
|
| 46 |
+
- Runs as an ordinary **Hugging Face Static Space** (`sdk: static`).
|
| 47 |
+
- Uses **vanilla HTML, CSS and ECMAScript modules**. No framework, API key, API calls, build pipeline, npm packages or cloud processing.
|
| 48 |
+
- CSV processing, metric computation, chart rendering and export happen in browser memory.
|
| 49 |
+
- No analytics, CDN, trackers, external telemetry or network requests are initiated by the application itself. Hosting-provider logging may still occur when accessing the Space.
|
| 50 |
+
|
| 51 |
+
## Start here
|
| 52 |
+
|
| 53 |
+
1. Open the **App** tab in Hugging Face.
|
| 54 |
+
2. Explore a fictional sample: **Support prioritization**, **Quality review** or **Training completion**.
|
| 55 |
+
3. Move the threshold slider and observe the changing confusion matrix, F1, precision, recall and group metrics. AUC, calibration and Brier score remain threshold-independent.
|
| 56 |
+
4. Use **Threshold explorer** to view all operating points and a seeded bootstrap interval.
|
| 57 |
+
5. Use **Fairness lens** for disaggregated cohort rates. Treat differences as prompts for investigation, not findings of unfairness.
|
| 58 |
+
6. Under **Data & methods**, import your own CSV, or download the example CSV to see the expected format.
|
| 59 |
+
7. Export a JSON audit and selected CSV tables with dedicated buttons.
|
| 60 |
+
|
| 61 |
+
## CSV contract
|
| 62 |
+
|
| 63 |
+
Required columns (case-insensitive header names):
|
| 64 |
+
|
| 65 |
+
| Column | Example | Definition |
|
| 66 |
+
| --- | --- | --- |
|
| 67 |
+
| `score` | `0.86` | Numeric model-assigned score in the closed interval `[0, 1]`. Interpreted as an estimated positive-class probability for calibration analysis. |
|
| 68 |
+
| `label` | `1` | Known ground-truth binary outcome: `0` or `1`. |
|
| 69 |
+
| `group` | `North` | Categorical cohort assignment, nonempty, maximum 72 characters. |
|
| 70 |
+
| `id` | `CASE-001` | Optional row identifier; not required for any metric. |
|
| 71 |
+
|
| 72 |
+
Example:
|
| 73 |
+
|
| 74 |
+
```csv
|
| 75 |
+
id,score,label,group
|
| 76 |
+
CASE-001,0.86,1,North
|
| 77 |
+
CASE-002,0.22,0,South
|
| 78 |
+
CASE-003,0.62,0,North
|
| 79 |
+
CASE-004,0.14,1,Central
|
| 80 |
+
```
|
| 81 |
+
|
| 82 |
+
Requirements and behavior:
|
| 83 |
+
|
| 84 |
+
- File extension `.csv`, maximum **2 MiB**; at most **10,000 data rows** and **35 distinct cohorts**.
|
| 85 |
+
- Supports UTF-8 CSV files with `,` delimiters, CRLF line endings, quoted fields, doubled quotes and embedded line breaks in quoted fields.
|
| 86 |
+
- Rejects missing or duplicate required headers, missing groups, non-binary labels, invalid scores and malformed row lengths with actionable messages.
|
| 87 |
+
- No scores are produced, adjusted or retrained. The slider changes only the positive/negative *decision* cutoff.
|
| 88 |
+
- CSV rows are not included in JSON audit exports. Aggregate metrics and the imported **file name** may be present in the report, so review outputs before sharing.
|
| 89 |
+
- Samples with very few rows or only one label class can yield undefined metrics. The UI uses `β` and JSON uses `null`, **not zero**, in these cases.
|
| 90 |
+
|
| 91 |
+
### Dataset privacy
|
| 92 |
+
|
| 93 |
+
Imported data are parsed in the browser. PrismAudit's own code does not upload the file to any server. It does not persist data in cookies or browser storage. The application is not a substitute for an organization's data governance controls: prefer sanitized or synthetic datasets when evaluating potentially personal, regulated or confidential data.
|
| 94 |
+
|
| 95 |
+
## Methods and equations
|
| 96 |
+
|
| 97 |
+
Let **TP**, **FP**, **TN**, **FN** be the confusion counts at a chosen threshold `t`, with predicted positive iff `score >= t`.
|
| 98 |
+
|
| 99 |
+
| Measure | Definition |
|
| 100 |
+
| --- | --- |
|
| 101 |
+
| Accuracy | `(TP + TN) / N` |
|
| 102 |
+
| Precision | `TP / (TP + FP)` |
|
| 103 |
+
| Recall / true positive rate | `TP / (TP + FN)` |
|
| 104 |
+
| Specificity / true negative rate | `TN / (TN + FP)` |
|
| 105 |
+
| False positive rate | `FP / (FP + TN)` |
|
| 106 |
+
| F1 | `2 * precision * recall / (precision + recall)` |
|
| 107 |
+
| Balanced accuracy | `(recall + specificity) / 2` |
|
| 108 |
+
| Positive prediction rate | `(TP + FP) / N` |
|
| 109 |
+
| Brier score | `mean((score - label)^2)` |
|
| 110 |
+
| ECE | `sum_b((n_b / N) * abs(mean_score_b - observed_positive_rate_b))` for 10 equal-width score bins |
|
| 111 |
+
| ROC AUC | Tie-aware MannβWhitney rank statistic, equivalent to ROC AUC for these empirical binary-score samples |
|
| 112 |
+
|
| 113 |
+
Division with a zero denominator yields an undefined rate. An F1 of zero is returned for defined zero-valued precision or recall; otherwise undefined inputs remain undefined. AUC is undefined if only one observed outcome class exists.
|
| 114 |
+
|
| 115 |
+
### Fairness lens definitions
|
| 116 |
+
|
| 117 |
+
For each user-defined group:
|
| 118 |
+
|
| 119 |
+
- **Demographic / positive parity gap:** `max_g P(predicted=1|group=g) - min_g P(predicted=1|group=g)`.
|
| 120 |
+
- **Equal opportunity proxy / recall gap:** `max_g TPR_g - min_g TPR_g` among groups with defined TPR.
|
| 121 |
+
- **False positive rate gap:** `max_g FPR_g - min_g FPR_g` among groups with defined FPR.
|
| 122 |
+
|
| 123 |
+
These are **absolute range differences**, not statistical significance tests, causal effects or legal standards. The UI warns when any cohort contains fewer than 30 observations or has only one outcome class. Such warnings are practical heuristics, not a universal threshold for adequate inference. Comparisons may be unreliable even with larger samples.
|
| 124 |
+
|
| 125 |
+
### Threshold analysis
|
| 126 |
+
|
| 127 |
+
The dashboard evaluates thresholds `0.00, 0.05, ... 1.00` for its comparison table. It identifies the threshold with the largest **observed in-sample F1** among these 21 points (ties broken in favor of the smaller threshold). Because this is determined on the same evaluation data, it is exploratory and **not a validated, cost-aware, safety-aware or fairness-aware deployment recommendation**. The slider itself permits increments of 0.01.
|
| 128 |
+
|
| 129 |
+
### Bootstrap
|
| 130 |
+
|
| 131 |
+
The *Calculate 95% interval* control performs 240 deterministic row-level bootstrap resamples with replacement, using PRNG seed `24`. It computes the empirical F1 distribution at the current threshold and displays percentile bounds at 2.5% and 97.5%, provided enough replicates produce a defined value. Intervals are conditional on independent and identically distributed row sampling; they may be misleading for time series, clusters, distribution shift, leakage or biased observational samples.
|
| 132 |
+
|
| 133 |
+
## Synthetic examples
|
| 134 |
+
|
| 135 |
+
Three reproducible 720-record examples are programmatically generated with a seeded PRNG. Samples use the generic cohort labels **North**, **Central**, **South**. The data include scores, synthetic outcomes and minor cohort-linked shifts to illustrate auditing. They do not represent any measured model, protected characteristic, real demographic data or customer group.
|
| 136 |
+
|
| 137 |
+
## Project structure
|
| 138 |
+
|
| 139 |
+
```text
|
| 140 |
+
README.md Hugging Face Space metadata and full documentation
|
| 141 |
+
index.html Accessible dashboard layout and all four modules
|
| 142 |
+
styles.css Responsive dashboard design without external assets
|
| 143 |
+
core.mjs Auditable evaluation mathematics, CSV validation and simulation
|
| 144 |
+
app.mjs Browser interaction, charts, privacy-safe exports
|
| 145 |
+
LICENSE MIT license
|
| 146 |
+
```
|
| 147 |
+
|
| 148 |
+
Source bundle additionally includes `tests/core.test.mjs`, `tests/browser_check.py` and browser screenshots. The deployable ZIP contains **only the six root files** listed above.
|
| 149 |
+
|
| 150 |
+
## Run locally and test
|
| 151 |
+
|
| 152 |
+
You can open a local static web server (ES modules require a normal HTTP origin):
|
| 153 |
+
|
| 154 |
+
```bash
|
| 155 |
+
python -m http.server 8000
|
| 156 |
+
# Open http://localhost:8000
|
| 157 |
+
```
|
| 158 |
+
|
| 159 |
+
Run JavaScript unit tests with Node.js 20+:
|
| 160 |
+
|
| 161 |
+
```bash
|
| 162 |
+
node --test tests/core.test.mjs
|
| 163 |
+
```
|
| 164 |
+
|
| 165 |
+
The optional Playwright smoke test uses Python Playwright and Chromium. It embeds the same CSS and JS into an isolated browser document to avoid requiring Internet access.
|
| 166 |
+
|
| 167 |
+
## Reproducibility and provenance
|
| 168 |
+
|
| 169 |
+
- Evaluation logic: `core.mjs`.
|
| 170 |
+
- Demonstration sample generation: `syntheticRows()` with fixed seed `2026`.
|
| 171 |
+
- Bootstrap confidence interval: `bootstrapInterval()` with fixed seed `24`.
|
| 172 |
+
- CSV parser, diagnostics, charts and downloadable reports: no third-party computation service.
|
| 173 |
+
- Licensed under MIT. See `LICENSE`.
|
| 174 |
+
|
| 175 |
+
## Limitations
|
| 176 |
+
|
| 177 |
+
PrismAudit is a professional **evaluation and educational prototype**, not an independently validated risk management system. No model cards, dataset cards, signed model artifacts, CI/CD deployment or out-of-sample holdout guarantee are provided by this Space. AUC, Brier score and calibration metrics assume that scores and labels refer to the same positive event; selecting a different label polarity changes their meaning. A visually small gap does not prove fairness, and a visually large gap does not establish its cause. Independent testing, domain-specific impact analysis, privacy review and expert validation remain necessary before high-stakes use.
|
| 178 |
+
|
| 179 |
+
---
|
| 180 |
+
|
| 181 |
+
**Project 24 Β· [Hugdrigo](https://huggingface.co/Hugdrigo) Β· MIT License**
|
app.mjs
ADDED
|
@@ -0,0 +1,46 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import { VERSION, parseCSV, validateRows, evaluate, fairness, rocCurve, thresholdSweep, bootstrapInterval, syntheticRows, toCSV, round } from './core.mjs';
|
| 2 |
+
|
| 3 |
+
const $ = id => document.getElementById(id);
|
| 4 |
+
const state = { preset:'support', rows:syntheticRows('support'), threshold:.5, source:'synthetic', fileName:null, sweep:[], current:null, group:null, interval:null, page:'overview' };
|
| 5 |
+
const pct = v => v === null || !Number.isFinite(v) ? 'β' : `${(v*100).toFixed(1)}%`;
|
| 6 |
+
const score = v => v === null || !Number.isFinite(v) ? 'β' : v.toFixed(3);
|
| 7 |
+
const number = n => n.toLocaleString('en-US');
|
| 8 |
+
const h = t => String(t).replace(/[&<>"']/g,c=>({'&':'&','<':'<','>':'>','"':'"',"'":'''}[c]));
|
| 9 |
+
const color = {green:'#b4f887',purple:'#b2a1ff',blue:'#70d5f7',amber:'#ffc277',dim:'#566d7b',grid:'#2b3c47'};
|
| 10 |
+
let toastTimer;
|
| 11 |
+
function toast(message,isError=false){ const el=$('toast');el.textContent=message;el.className=`toast show${isError?' error':''}`;clearTimeout(toastTimer);toastTimer=setTimeout(()=>el.classList.remove('show'),3900); }
|
| 12 |
+
function alertMessage(message){const el=$('globalAlert');el.textContent=message;el.hidden=false;window.scrollTo({top:0,behavior:'smooth'});}
|
| 13 |
+
function clearAlert(){$('globalAlert').hidden=true;}
|
| 14 |
+
function svgBase(content,label=''){return `<svg viewBox="0 0 680 275" preserveAspectRatio="xMidYMid meet" xmlns="http://www.w3.org/2000/svg" aria-label="${h(label)}"><defs><linearGradient id="plotFill" x1="0" x2="0" y1="0" y2="1"><stop offset="0%" stop-color="#b4f887" stop-opacity=".18"/><stop offset="100%" stop-color="#b4f887" stop-opacity="0"/></linearGradient></defs>${content}</svg>`;}
|
| 15 |
+
function svgAxes(xlabel,ylabel){let out='';for(let i=0;i<=5;i++){const x=48+(620/5)*i,y=230-(196/5)*i;out+=`<line x1="48" y1="${y}" x2="668" y2="${y}" stroke="${color.grid}" stroke-width="1" stroke-dasharray="2 5"/><text x="39" y="${y+4}" fill="#6f8795" font-size="11" text-anchor="end">${(i/5).toFixed(1)}</text><text x="${x}" y="250" fill="#6f8795" font-size="11" text-anchor="middle">${(i/5).toFixed(1)}</text>`;}return out+`<text x="358" y="270" font-size="11" fill="#96a9b4" text-anchor="middle">${h(xlabel)}</text><text x="13" y="139" font-size="11" fill="#96a9b4" text-anchor="middle" transform="rotate(-90 13 139)">${h(ylabel)}</text>`;}
|
| 16 |
+
const X = x=>48+620*x, Y=y=>230-196*y;
|
| 17 |
+
function pathFor(points, xkey, ykey){let active=false;return points.map(p=>{const x=p[xkey],y=p[ykey];if(y===null || !Number.isFinite(y)){active=false;return '';}const prefix=active?'L':'M';active=true;return `${prefix}${X(x).toFixed(2)},${Y(y).toFixed(2)}`;}).filter(Boolean).join(' ');}
|
| 18 |
+
function renderRoc(){const points=rocCurve(state.rows);let content=svgAxes('False positive rate','True positive rate');content+=`<path d="M${X(0)} ${Y(0)} L${X(1)} ${Y(1)}" stroke="${color.dim}" stroke-width="2" stroke-dasharray="5 7"/>`;if(points.length){content+=`<path d="${pathFor(points,'fpr','tpr')}" fill="none" stroke="${color.green}" stroke-width="3" stroke-linecap="round" stroke-linejoin="round"/>`;const tail=points[points.length-1];content+=`<circle cx="${X(tail.fpr)}" cy="${Y(tail.tpr)}" r="4" fill="${color.green}"/>`;}else content+=`<text x="370" y="125" fill="#ffc277" font-size="16" text-anchor="middle">ROC undefined: only one ground-truth class</text>`;$('rocChart').innerHTML=svgBase(content,'ROC curve');}
|
| 19 |
+
function renderCalibration(){const bins=state.current.bins;let content=svgAxes('Mean predicted score','Observed positive frequency');content+=`<path d="M${X(0)} ${Y(0)} L${X(1)} ${Y(1)}" stroke="${color.dim}" stroke-width="2" stroke-dasharray="5 7"/>`;const active=bins.filter(b=>b.n);content+=active.map(b=>`<line x1="${X(b.confidence)}" y1="${Y(0)}" x2="${X(b.confidence)}" y2="${Y(b.outcome)}" stroke="${color.purple}" stroke-opacity=".28" stroke-width="7"/><circle cx="${X(b.confidence)}" cy="${Y(b.outcome)}" r="5" fill="${color.purple}"><title>Bin ${b.bin}: n=${b.n}, mean score=${score(b.confidence)}, observed=${pct(b.outcome)}</title></circle>`).join('');if(active.length>1)content+=`<path d="${pathFor(active,'confidence','outcome')}" fill="none" stroke="${color.purple}" stroke-width="2"/>`;$('calChart').innerHTML=svgBase(content,'Calibration chart');}
|
| 20 |
+
function renderThreshold(){const sw=state.sweep;let content=svgAxes('Decision threshold','Metric score');for(const [key,tone] of [['f1','green'],['precision','purple'],['recall','blue']])content+=`<path d="${pathFor(sw,'threshold',key)}" fill="none" stroke="${color[tone]}" stroke-width="2.6" stroke-linecap="round"/>`;content+=`<line x1="${X(state.threshold)}" y1="34" x2="${X(state.threshold)}" y2="230" stroke="${color.amber}" stroke-width="2" stroke-dasharray="4 4"/>`;const y = state.current.f1===null?.5:state.current.f1;content+=`<circle cx="${X(state.threshold)}" cy="${Y(y)}" r="5" fill="${color.amber}" stroke="#111d27" stroke-width="2"/>`;$('thresholdChart').innerHTML=svgBase(content,'F1, precision and recall versus threshold');}
|
| 21 |
+
function renderInsights(){const m=state.current,f=state.group;const notes=[];if(m.n<100)notes.push(['Small evaluation set',`Only ${number(m.n)} rows are available. Estimates may be unstable.`,true]);if(m.auc===null)notes.push(['Ranking unavailable','ROC AUC requires both positive and negative ground-truth observations.',true]);else if(m.auc<.6)notes.push(['Limited discrimination',`ROC AUC is ${score(m.auc)}. Check signal quality and whether the scores rank outcomes usefully.`,true]);else notes.push(['Discrimination snapshot',`ROC AUC of ${score(m.auc)} describes ranking on this dataset, not performance in production.`,false]);if(m.ece>=.10)notes.push(['Calibration needs review',`Estimated calibration error is ${pct(m.ece)} across ten equal-width bins.`,true]);else notes.push(['Calibration diagnostic',`Estimated calibration error is ${pct(m.ece)}. Validate calibration on an independent sample.`,false]);if(f.groups.length<2)notes.push(['One observed cohort','At least two groups are required for between-cohort comparisons.',true]);else if(!f.reliable)notes.push(['Cohort uncertainty','Some cohorts have fewer than 30 records or lack a required observed outcome class. Gaps need cautious interpretation.',true]);else notes.push(['Cohort differences',`Observed positive decision rate gap is ${pct(f.demographicParityGap)} across ${f.groups.length} cohorts; this is not a fairness verdict.`,f.demographicParityGap>.15]);if(m.prevalence<.1||m.prevalence>.9)notes.push(['Class imbalance',`Only ${pct(m.prevalence)} of the observed records are positive. Accuracy alone may be misleading.`,true]);$('insightList').innerHTML=notes.slice(0,4).map(([title,body,warn])=>`<div class="insight${warn?' warn':''}"><span class="insight-icon">${warn?'!':'β'}</span><div><b>${h(title)}</b><p>${h(body)}</p></div></div>`).join('');}
|
| 22 |
+
function renderGroupBars(){const groups=state.group.groups;const max=Math.max(.01,...groups.map(g=>g.positiveRate));$('groupBars').innerHTML=groups.map(g=>`<div class="group-row"><span class="group-label">${h(g.group)}${g.n<30?' β ':''}</span><div class="group-bar"><div style="width:${(g.positiveRate/max*100).toFixed(1)}%"></div></div><span class="group-val">${pct(g.positiveRate)}</span></div>`).join('');}
|
| 23 |
+
function renderTable(){const threshold=state.threshold;const rows=[...state.sweep].sort((a,b)=>Math.abs(a.threshold-threshold)-Math.abs(b.threshold-threshold)).slice(0,9).sort((a,b)=>a.threshold-b.threshold);$('sweepBody').innerHTML=rows.map(r=>`<tr class="${Math.abs(r.threshold-threshold)<.025?'current':''}"><td>${r.threshold.toFixed(2)}</td><td>${pct(r.f1)}</td><td>${pct(r.precision)}</td><td>${pct(r.recall)}</td><td>${pct(r.accuracy)}</td><td>${pct(r.positiveRate)}</td></tr>`).join('');$('groupsBody').innerHTML=state.group.groups.map(g=>`<tr><td>${h(g.group)}${g.n<30?' β ':''}</td><td>${number(g.n)}</td><td>${pct(g.prevalence)}</td><td>${pct(g.positiveRate)}</td><td>${pct(g.recall)}</td><td>${pct(g.fpr)}</td><td>${pct(g.accuracy)}</td></tr>`).join('');}
|
| 24 |
+
function renderProfile(){const m=state.current;const groups=state.group.groups;$('dataProfile').innerHTML=[['Records',number(m.n)],['Cohorts',number(groups.length)],['Positive labels',pct(m.prevalence)],['Score range',`${Math.min(...state.rows.map(x=>x.score)).toFixed(2)}β${Math.max(...state.rows.map(x=>x.score)).toFixed(2)}`],['Positive decisions',number(m.tp+m.fp)],['Data origin',state.source==='synthetic'?'Synthetic':'CSV import']].map(([k,v])=>`<div class="data-item"><span>${h(k)}</span><strong>${h(v)}</strong></div>`).join('');}
|
| 25 |
+
function updatePage(){document.querySelectorAll('[data-page]').forEach(btn=>{const active=btn.dataset.page===state.page;btn.classList.toggle('active',active);active?btn.setAttribute('aria-current','page'):btn.removeAttribute('aria-current');});document.querySelectorAll('.page').forEach(p=>p.hidden=p.id!==`page-${state.page}`);}
|
| 26 |
+
function renderAll(){const m=state.current=evaluate(state.rows,state.threshold), f=state.group=fairness(state.rows,state.threshold);$('thresholdValue').textContent=state.threshold.toFixed(2);$('thresholdRange').value=Math.round(state.threshold*100);$('metricAccuracy').textContent=pct(m.accuracy);$('metricF1').textContent=score(m.f1);$('metricAuc').textContent=score(m.auc);$('metricBrier').textContent=score(m.brier);$('metricPrecision').textContent=pct(m.precision);$('metricRecall').textContent=pct(m.recall);$('metricSpecificity').textContent=pct(m.specificity);$('metricBalanced').textContent=pct(m.balancedAccuracy);$('tpCount').textContent=number(m.tp);$('fpCount').textContent=number(m.fp);$('tnCount').textContent=number(m.tn);$('fnCount').textContent=number(m.fn);$('confusionFoot').textContent=`Predicted positive at score β₯ ${state.threshold.toFixed(2)}. Total = ${number(m.n)} observations.`;$('sampleSize').textContent=`${number(m.n)} OBSERVATIONS`;$('aucTag').textContent=score(m.auc);$('eceTag').textContent=score(m.ece);$('gapParity').textContent=pct(f.demographicParityGap);$('gapRecall').textContent=pct(f.equalOpportunityGap);$('gapFpr').textContent=pct(f.falsePositiveRateGap);$('minGroup').textContent=number(f.minimumGroupN);$('fairnessWarning').className=`fairness-warning${f.reliable?' good':''}`;$('fairnessWarning').textContent=f.groups.length<2?'Only one group is present: between-cohort parity comparisons are undefined.':f.reliable?'All cohorts have at least 30 rows and both outcome classes. Statistical uncertainty and context still matter.':'Caution: one or more groups have fewer than 30 records, or one group lacks an observed outcome class. Some conditional rate gaps may be undefined or unstable.';renderRoc();renderCalibration();renderThreshold();renderInsights();renderGroupBars();renderTable();renderProfile();const eligible=state.sweep.filter(s=>s.f1!==null).sort((a,b)=>b.f1-a.f1 || a.threshold-b.threshold);const best=eligible[0];$('bestThreshold').textContent=best?best.threshold.toFixed(2):'N/A';$('bestNote').textContent=best?`Highest displayed F1: ${pct(best.f1)} at cutoff ${best.threshold.toFixed(2)}. Calculated on the same data; not validated for deployment.`:'No defined F1 score is available for this sample.';$('applyBest').disabled=!best;state.interval=null;$('bootstrapResult').textContent='Awaiting calculation';}
|
| 27 |
+
function loadRows(rows,source,fileName){state.rows=validateRows(rows);state.source=source;state.fileName=fileName||null;state.threshold=.5;state.sweep=thresholdSweep(rows);if(source==='uploaded'){$('datasetSelect').value='uploaded';$('datasetMeta').textContent=`CSV Β· ${number(rows.length)} rows`;$('modeNotice').innerHTML='<strong>Imported dataset.</strong> Evaluation happens entirely in your browser. Verify labels, consent and data provenance before interpreting results.';}else{$('datasetSelect').value=state.preset;$('datasetMeta').textContent=`Synthetic Β· ${number(rows.length)} records`;$('modeNotice').innerHTML='<strong>Demonstration mode.</strong> All records are fictional and algorithmically generated. No ML training happens in this application.';}renderAll();clearAlert();}
|
| 28 |
+
function showPage(page){if(!['overview','threshold','fairness','data'].includes(page))return;state.page=page;updatePage();window.scrollTo({top:0,behavior:'smooth'});}
|
| 29 |
+
function saveFile(filename,content,type){const blob=new Blob([content],{type});const url=URL.createObjectURL(blob);const a=document.createElement('a');a.href=url;a.download=filename;document.body.appendChild(a);a.click();a.remove();setTimeout(()=>URL.revokeObjectURL(url),5000);toast(`Saved ${filename}`);}
|
| 30 |
+
function safeReport(){const m=state.current,f=state.group;const summary={app:'PrismAudit',version:VERSION,project:24,createdAt:new Date().toISOString(),dataSource:state.source,fileName:state.fileName,recordCount:state.rows.length,threshold:state.threshold,metrics:{n:m.n,tp:m.tp,fp:m.fp,tn:m.tn,fn:m.fn,prevalence:m.prevalence,accuracy:m.accuracy,precision:m.precision,recall:m.recall,specificity:m.specificity,f1:m.f1,balancedAccuracy:m.balancedAccuracy,fpr:m.fpr,fnr:m.fnr,positiveRate:m.positiveRate,auc:m.auc,brier:m.brier,ece:m.ece},calibration:m.bins,fairness:{demographicParityGap:f.demographicParityGap,equalOpportunityGap:f.equalOpportunityGap,falsePositiveRateGap:f.falsePositiveRateGap,reliable:f.reliable,groups:f.groups.map(({group,n,prevalence,positiveRate,recall,fpr,accuracy})=>({group,n,prevalence,positiveRate,recall,fpr,accuracy}))},thresholdSweep:state.sweep,bootstrap:state.interval,disclaimer:'Descriptive audit only. Not training, causal assessment, compliance certification or deployment recommendation. Missing rates are represented as null. Synthetic examples are fictional.'};return summary;}
|
| 31 |
+
async function handleFile(file){if(!file)return;if(file.size>2*1024*1024){toast('File exceeds the 2 MB limit.',true);return;}if(!file.name.toLowerCase().endsWith('.csv')){toast('Please use a .csv file.',true);return;}try{const rows=parseCSV(await file.text());let custom=$('datasetSelect').querySelector('option[value="uploaded"]');if(!custom){custom=document.createElement('option');custom.value='uploaded';$('datasetSelect').appendChild(custom);}custom.textContent=`Imported: ${file.name.slice(0,28)}`;loadRows(rows,'uploaded',file.name);toast(`Imported ${number(rows.length)} records. Nothing was uploaded to a server.`);showPage('overview');}catch(error){toast(error.message,true);alertMessage(`Import failed: ${error.message}`);}finally{$('fileInput').value='';}}
|
| 32 |
+
|
| 33 |
+
document.querySelectorAll('[data-page]').forEach(btn=>btn.addEventListener('click',()=>showPage(btn.dataset.page)));
|
| 34 |
+
$('thresholdRange').addEventListener('input',event=>{state.threshold=Number(event.target.value)/100;renderAll();});
|
| 35 |
+
$('resetThreshold').addEventListener('click',()=>{state.threshold=.5;renderAll();});
|
| 36 |
+
$('datasetSelect').addEventListener('change',event=>{if(event.target.value==='uploaded')return;state.preset=event.target.value;loadRows(syntheticRows(state.preset),'synthetic');toast(`Switched to ${event.target.options[event.target.selectedIndex].text}.`);});
|
| 37 |
+
$('applyBest').addEventListener('click',()=>{const best=state.sweep.filter(s=>s.f1!==null).sort((a,b)=>b.f1-a.f1 || a.threshold-b.threshold)[0];if(!best)return;state.threshold=best.threshold;renderAll();toast(`Evaluating cutoff ${best.threshold.toFixed(2)}. This is not a deployment recommendation.`);});
|
| 38 |
+
$('runBootstrap').addEventListener('click',()=>{const btn=$('runBootstrap');btn.disabled=true;$('bootstrapResult').textContent='Calculating 240 seeded resamplesβ¦';setTimeout(()=>{try{const res=state.interval=bootstrapInterval(state.rows,state.threshold);$('bootstrapResult').textContent=res.lower===null?'Interval unavailable: too many undefined replicate metrics.':`95% percentile interval for F1: [${score(res.lower)}, ${score(res.upper)}] Β· ${res.valid}/${res.repeats} valid repeats Β· seed 24`;}catch(error){toast(error.message,true);$('bootstrapResult').textContent=error.message;}finally{btn.disabled=false;}},40);});
|
| 39 |
+
$('exportReport').addEventListener('click',()=>saveFile('prismaudit_report.json',JSON.stringify(safeReport(),null,2),'application/json'));
|
| 40 |
+
$('exportSweep').addEventListener('click',()=>saveFile('prismaudit_thresholds.csv',toCSV(state.sweep,['threshold','f1','accuracy','balancedAccuracy','precision','recall','positiveRate','parityGap']),'text/csv;charset=utf-8'));
|
| 41 |
+
$('exportGroups').addEventListener('click',()=>saveFile('prismaudit_groups.csv',toCSV(state.group.groups,['group','n','prevalence','positiveRate','recall','fpr','accuracy']),'text/csv;charset=utf-8'));
|
| 42 |
+
$('downloadSample').addEventListener('click',()=>saveFile('prismaudit_example.csv',toCSV(syntheticRows('support',120),['id','score','label','group']),'text/csv;charset=utf-8'));
|
| 43 |
+
$('restoreDemo').addEventListener('click',()=>{state.preset='support';loadRows(syntheticRows('support'),'synthetic');toast('Restored fictional example data.');});
|
| 44 |
+
$('importTop').addEventListener('click',()=>$('fileInput').click());$('browseData').addEventListener('click',event=>{event.stopPropagation();$('fileInput').click();});$('fileInput').addEventListener('change',event=>handleFile(event.target.files[0]));
|
| 45 |
+
const drop=$('dropZone');drop.addEventListener('click',()=>$('fileInput').click());drop.addEventListener('keydown',e=>{if(e.key==='Enter'||e.key===' '){e.preventDefault();$('fileInput').click();}});for(const ev of ['dragenter','dragover'])drop.addEventListener(ev,e=>{e.preventDefault();drop.classList.add('dragging');});for(const ev of ['dragleave','drop'])drop.addEventListener(ev,e=>{e.preventDefault();drop.classList.remove('dragging');});drop.addEventListener('drop',e=>handleFile(e.dataTransfer?.files?.[0]));
|
| 46 |
+
loadRows(state.rows,'synthetic');updatePage();
|
core.mjs
ADDED
|
@@ -0,0 +1,206 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/** PrismAudit analytical core. Pure functions; no network, no dependencies. MIT. */
|
| 2 |
+
export const VERSION = '1.0.0';
|
| 3 |
+
export const MAX_ROWS = 10000;
|
| 4 |
+
export const REQUIRED_COLUMNS = ['score', 'label', 'group'];
|
| 5 |
+
export const DEFAULT_THRESHOLD = 0.50;
|
| 6 |
+
|
| 7 |
+
export function parseCSV(csv) {
|
| 8 |
+
if (typeof csv !== 'string') throw new Error('Expected a CSV text file.');
|
| 9 |
+
csv = csv.replace(/^\uFEFF/, '');
|
| 10 |
+
const table = []; let row = [], val = '', quoted = false;
|
| 11 |
+
for (let i = 0; i < csv.length; i++) {
|
| 12 |
+
const c = csv[i];
|
| 13 |
+
if (quoted) {
|
| 14 |
+
if (c === '"' && csv[i+1] === '"') { val += '"'; i++; }
|
| 15 |
+
else if (c === '"') quoted = false;
|
| 16 |
+
else val += c;
|
| 17 |
+
} else if (c === '"') {
|
| 18 |
+
if (val.trim() !== '') throw new Error('Unexpected quote in CSV field.');
|
| 19 |
+
quoted = true;
|
| 20 |
+
} else if (c === ',') { row.push(val); val = ''; }
|
| 21 |
+
else if (c === '\n' || c === '\r') {
|
| 22 |
+
if (c === '\r' && csv[i+1] === '\n') i++;
|
| 23 |
+
row.push(val); val = '';
|
| 24 |
+
if (row.some(v => v.trim().length)) table.push(row);
|
| 25 |
+
row = [];
|
| 26 |
+
if (table.length > MAX_ROWS + 1) throw new Error(`Maximum ${MAX_ROWS.toLocaleString()} data rows allowed.`);
|
| 27 |
+
} else { val += c; }
|
| 28 |
+
}
|
| 29 |
+
if (quoted) throw new Error('Unclosed quoted field in CSV.');
|
| 30 |
+
row.push(val); if (row.some(v => v.trim().length)) table.push(row);
|
| 31 |
+
if (table.length < 2) throw new Error('CSV must contain a header and at least one data row.');
|
| 32 |
+
const headers = table.shift().map(s => s.trim().toLowerCase());
|
| 33 |
+
if (new Set(headers).size !== headers.length) throw new Error('CSV has duplicate column names.');
|
| 34 |
+
for (const h of REQUIRED_COLUMNS) if (!headers.includes(h)) throw new Error(`Missing required column: ${h}.`);
|
| 35 |
+
const indices = Object.fromEntries(headers.map((h, i) => [h, i]));
|
| 36 |
+
if (table.length > MAX_ROWS) throw new Error(`Maximum ${MAX_ROWS.toLocaleString()} rows allowed.`);
|
| 37 |
+
const rows = table.map((fields, i) => {
|
| 38 |
+
if (fields.length !== headers.length) throw new Error(`Row ${i + 2}: expected ${headers.length} columns, got ${fields.length}.`);
|
| 39 |
+
const rawScore = fields[indices.score].trim(), rawLabel = fields[indices.label].trim();
|
| 40 |
+
if (!/^(?:0(?:\.\d+)?|1(?:\.0+)?)$/.test(rawScore)) throw new Error(`Row ${i + 2}: score must be between 0 and 1.`);
|
| 41 |
+
if (!/^[01]$/.test(rawLabel)) throw new Error(`Row ${i + 2}: label must be exactly 0 or 1.`);
|
| 42 |
+
const group = fields[indices.group].trim();
|
| 43 |
+
if (!group || group.length > 72) throw new Error(`Row ${i + 2}: group is empty or exceeds 72 characters.`);
|
| 44 |
+
const id = indices.id === undefined ? String(i + 1) : fields[indices.id].trim().slice(0,120);
|
| 45 |
+
return { id, score: Number(rawScore), label: Number(rawLabel), group };
|
| 46 |
+
});
|
| 47 |
+
return validateRows(rows);
|
| 48 |
+
}
|
| 49 |
+
|
| 50 |
+
export function validateRows(rows) {
|
| 51 |
+
if (!Array.isArray(rows) || rows.length < 1 || rows.length > MAX_ROWS) throw new Error(`Use 1 to ${MAX_ROWS.toLocaleString()} rows.`);
|
| 52 |
+
const groupCount = new Set();
|
| 53 |
+
for (const [i, r] of rows.entries()) {
|
| 54 |
+
if (!Number.isFinite(r.score) || r.score < 0 || r.score > 1) throw new Error(`Row ${i + 1}: invalid score.`);
|
| 55 |
+
if (r.label !== 0 && r.label !== 1) throw new Error(`Row ${i + 1}: label must be 0 or 1.`);
|
| 56 |
+
if (typeof r.group !== 'string' || !r.group.trim() || r.group.length > 72) throw new Error(`Row ${i + 1}: invalid group.`);
|
| 57 |
+
groupCount.add(r.group);
|
| 58 |
+
if (groupCount.size > 35) throw new Error('A maximum of 35 unique groups is supported.');
|
| 59 |
+
}
|
| 60 |
+
return rows;
|
| 61 |
+
}
|
| 62 |
+
|
| 63 |
+
export const safeDivide = (a, b) => b === 0 ? null : a / b;
|
| 64 |
+
export const round = (v, places = 4) => v === null || !Number.isFinite(v) ? null : Number(v.toFixed(places));
|
| 65 |
+
|
| 66 |
+
export function confusion(rows, threshold) {
|
| 67 |
+
if (!Number.isFinite(threshold) || threshold < 0 || threshold > 1) throw new Error('Threshold must be between 0 and 1.');
|
| 68 |
+
let tp = 0, fp = 0, tn = 0, fn = 0;
|
| 69 |
+
for (const r of rows) {
|
| 70 |
+
const positive = r.score >= threshold;
|
| 71 |
+
if (positive && r.label === 1) tp++;
|
| 72 |
+
else if (positive && r.label === 0) fp++;
|
| 73 |
+
else if (!positive && r.label === 0) tn++;
|
| 74 |
+
else fn++;
|
| 75 |
+
}
|
| 76 |
+
return { tp, fp, tn, fn };
|
| 77 |
+
}
|
| 78 |
+
|
| 79 |
+
/** Tie-aware Mann-Whitney (rank statistic) ROC AUC. Undefined if only one observed class. */
|
| 80 |
+
export function auc(rows) {
|
| 81 |
+
let positives = 0, negatives = 0;
|
| 82 |
+
for (const r of rows) r.label ? positives++ : negatives++;
|
| 83 |
+
if (!positives || !negatives) return null;
|
| 84 |
+
const sorted = [...rows].sort((a, b) => a.score - b.score);
|
| 85 |
+
let rank = 1, sumPositiveRanks = 0;
|
| 86 |
+
for (let i = 0; i < sorted.length;) {
|
| 87 |
+
let j = i + 1;
|
| 88 |
+
while (j < sorted.length && sorted[j].score === sorted[i].score) j++;
|
| 89 |
+
const meanRank = (rank + (rank + j - i - 1)) / 2;
|
| 90 |
+
for (let k = i; k < j; k++) if (sorted[k].label) sumPositiveRanks += meanRank;
|
| 91 |
+
rank += j - i; i = j;
|
| 92 |
+
}
|
| 93 |
+
return (sumPositiveRanks - positives * (positives + 1) / 2) / (positives * negatives);
|
| 94 |
+
}
|
| 95 |
+
|
| 96 |
+
export function calibration(rows, bins = 10) {
|
| 97 |
+
const slots = Array.from({length: bins}, (_, i) => ({index:i, n:0, totalScore:0, totalLabel:0}));
|
| 98 |
+
let brierSum = 0;
|
| 99 |
+
for (const r of rows) {
|
| 100 |
+
const b = Math.min(bins - 1, Math.floor(r.score * bins));
|
| 101 |
+
slots[b].n++; slots[b].totalScore += r.score; slots[b].totalLabel += r.label;
|
| 102 |
+
brierSum += (r.score - r.label) ** 2;
|
| 103 |
+
}
|
| 104 |
+
let ece = 0;
|
| 105 |
+
const data = slots.map(({index,n,totalScore,totalLabel}) => {
|
| 106 |
+
const confidence = safeDivide(totalScore,n), outcome = safeDivide(totalLabel,n);
|
| 107 |
+
if (n) ece += n / rows.length * Math.abs(confidence - outcome);
|
| 108 |
+
return {bin:index + 1, n, confidence, outcome};
|
| 109 |
+
});
|
| 110 |
+
return {bins:data, ece, brier:brierSum / rows.length};
|
| 111 |
+
}
|
| 112 |
+
|
| 113 |
+
export function evaluate(rows, threshold = DEFAULT_THRESHOLD) {
|
| 114 |
+
validateRows(rows);
|
| 115 |
+
const {tp, fp, tn, fn} = confusion(rows, threshold);
|
| 116 |
+
const n = rows.length, p = tp + fn, negative = tn + fp;
|
| 117 |
+
const precision = safeDivide(tp, tp + fp);
|
| 118 |
+
const recall = safeDivide(tp, p);
|
| 119 |
+
const specificity = safeDivide(tn, negative);
|
| 120 |
+
const f1 = precision === null || recall === null || precision + recall === 0 ? (precision === 0 || recall === 0 ? 0 : null) : 2 * precision * recall / (precision + recall);
|
| 121 |
+
const balancedAccuracy = recall === null || specificity === null ? null : (recall + specificity) / 2;
|
| 122 |
+
return {
|
| 123 |
+
n, threshold, tp, fp, tn, fn, prevalence:p/n,
|
| 124 |
+
accuracy:(tp + tn)/n, precision, recall, specificity, f1,
|
| 125 |
+
balancedAccuracy, fpr:safeDivide(fp,negative), fnr:safeDivide(fn,p),
|
| 126 |
+
positiveRate:(tp + fp)/n, auc:auc(rows), ...calibration(rows)
|
| 127 |
+
};
|
| 128 |
+
}
|
| 129 |
+
|
| 130 |
+
export function byGroup(rows, threshold) {
|
| 131 |
+
const map = new Map();
|
| 132 |
+
for (const r of rows) { if (!map.has(r.group)) map.set(r.group, []); map.get(r.group).push(r); }
|
| 133 |
+
return [...map.entries()].sort((a,b)=>a[0].localeCompare(b[0])).map(([group,subset])=>({group, ...evaluate(subset, threshold)}));
|
| 134 |
+
}
|
| 135 |
+
export function rangeOf(values) { const a=values.filter(x=>x!==null && Number.isFinite(x)); return a.length < 2 ? null : Math.max(...a)-Math.min(...a); }
|
| 136 |
+
export function fairness(rows, threshold) {
|
| 137 |
+
const groups=byGroup(rows,threshold);
|
| 138 |
+
return {groups, demographicParityGap:rangeOf(groups.map(x=>x.positiveRate)),
|
| 139 |
+
equalOpportunityGap:rangeOf(groups.map(x=>x.recall)),
|
| 140 |
+
falsePositiveRateGap:rangeOf(groups.map(x=>x.fpr)),
|
| 141 |
+
minimumGroupN:Math.min(...groups.map(x=>x.n)),
|
| 142 |
+
reliable:groups.length >= 2 && groups.every(x=>x.n>=30) && groups.every(x=>x.recall!==null && x.fpr!==null)};
|
| 143 |
+
}
|
| 144 |
+
|
| 145 |
+
export function rocCurve(rows) {
|
| 146 |
+
const p = rows.reduce((s,r)=>s+r.label,0), n=rows.length-p;
|
| 147 |
+
if (!p || !n) return [];
|
| 148 |
+
const sorted=[...rows].sort((a,b)=>b.score-a.score);
|
| 149 |
+
const points=[{fpr:0,tpr:0,threshold:1.000001}];
|
| 150 |
+
let tp=0, fp=0;
|
| 151 |
+
for(let i=0;i<sorted.length;) {
|
| 152 |
+
const score=sorted[i].score;
|
| 153 |
+
while(i<sorted.length && sorted[i].score===score) { if(sorted[i].label)tp++;else fp++;i++; }
|
| 154 |
+
points.push({fpr:fp/n,tpr:tp/p,threshold:score});
|
| 155 |
+
}
|
| 156 |
+
return points;
|
| 157 |
+
}
|
| 158 |
+
|
| 159 |
+
export function thresholdSweep(rows, step = 0.05) {
|
| 160 |
+
const out=[];
|
| 161 |
+
for(let i=0;i<=Math.round(1/step);i++) {
|
| 162 |
+
const threshold=Number((i*step).toFixed(4));
|
| 163 |
+
const m=evaluate(rows,threshold), f=fairness(rows,threshold);
|
| 164 |
+
out.push({threshold, f1:m.f1, accuracy:m.accuracy, balancedAccuracy:m.balancedAccuracy, precision:m.precision, recall:m.recall, positiveRate:m.positiveRate, parityGap:f.demographicParityGap});
|
| 165 |
+
}
|
| 166 |
+
return out;
|
| 167 |
+
}
|
| 168 |
+
|
| 169 |
+
/** Deterministic noncryptographic PRNG, never for security or personal data. */
|
| 170 |
+
export function rng(seed) {let t=seed>>>0;return ()=>{t+=0x6D2B79F5;let x=t; x=Math.imul(x^x>>>15,x|1);x^=x+Math.imul(x^x>>>7,x|61);return ((x^x>>>14)>>>0)/4294967296;};}
|
| 171 |
+
export function bootstrapInterval(rows, threshold, metric = 'f1', repeats = 240, seed = 24) {
|
| 172 |
+
if(!['f1','accuracy','balancedAccuracy','auc','brier'].includes(metric))throw new Error('Unsupported metric.');
|
| 173 |
+
if (!Number.isInteger(repeats) || repeats < 50 || repeats > 500) throw new Error('Use 50 to 500 bootstrap repeats.');
|
| 174 |
+
validateRows(rows); const random=rng(seed), values=[];
|
| 175 |
+
for (let b=0;b<repeats;b++) {
|
| 176 |
+
const sample=Array.from({length:rows.length},()=>rows[Math.floor(random()*rows.length)]);
|
| 177 |
+
const metricValue=evaluate(sample,threshold)[metric];
|
| 178 |
+
if(metricValue!==null)values.push(metricValue);
|
| 179 |
+
}
|
| 180 |
+
if(values.length<Math.ceil(repeats*0.8))return {lower:null,upper:null,repeats,valid:values.length};
|
| 181 |
+
values.sort((a,b)=>a-b);
|
| 182 |
+
const quantile=q=>{const at=(values.length-1)*q, lo=Math.floor(at), hi=Math.ceil(at);return values[lo]+(values[hi]-values[lo])*(at-lo);};
|
| 183 |
+
return {lower:quantile(.025),upper:quantile(.975),repeats,valid:values.length};
|
| 184 |
+
}
|
| 185 |
+
|
| 186 |
+
export function csvEscape(v) {const s=String(v??'');return /[,"\r\n]/.test(s)?`"${s.replace(/"/g,'""')}"`:s;}
|
| 187 |
+
export function toCSV(rows, columns) {return [columns.join(','), ...rows.map(r=>columns.map(c=>csvEscape(r[c])).join(','))].join('\r\n')+'\r\n';}
|
| 188 |
+
|
| 189 |
+
export function syntheticRows(preset='support',count=720,seed=2026) {
|
| 190 |
+
if(!['support','quality','education'].includes(preset)) throw new Error('Unknown demo dataset.');
|
| 191 |
+
const random=rng(seed), rows=[], cohorts=['North','Central','South'];
|
| 192 |
+
const sigmoid=x=>1/(1+Math.exp(-x));
|
| 193 |
+
for(let i=0;i<count;i++) {
|
| 194 |
+
const group=cohorts[i%3];
|
| 195 |
+
const index=i%3;
|
| 196 |
+
const latent=random()*2-1;
|
| 197 |
+
const groupDelta=preset==='support'?[.22,-.12,-.22][index]:preset==='quality'?[.05,.12,-.1][index]:[-.1,.2,-.16][index];
|
| 198 |
+
const base=preset==='support' ? -.2 : preset==='quality' ? -.48 : .1;
|
| 199 |
+
const trueP=sigmoid(base+latent*3+groupDelta);
|
| 200 |
+
const label=random()<trueP?1:0;
|
| 201 |
+
const noise=(random()-.5)*.35;
|
| 202 |
+
const score=Math.max(.005,Math.min(.995,sigmoid(base+latent*3+groupDelta*1.65+noise)));
|
| 203 |
+
rows.push({id:`SYN-${String(i+1).padStart(4,'0')}`,score:Number(score.toFixed(4)),label,group});
|
| 204 |
+
}
|
| 205 |
+
return rows;
|
| 206 |
+
}
|
index.html
CHANGED
|
@@ -1,19 +1,68 @@
|
|
| 1 |
<!doctype html>
|
| 2 |
-
<html>
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
</html>
|
|
|
|
| 1 |
<!doctype html>
|
| 2 |
+
<html lang="en">
|
| 3 |
+
<head>
|
| 4 |
+
<meta charset="utf-8">
|
| 5 |
+
<meta name="viewport" content="width=device-width,initial-scale=1">
|
| 6 |
+
<meta name="color-scheme" content="dark">
|
| 7 |
+
<meta name="theme-color" content="#0a1018">
|
| 8 |
+
<meta name="description" content="PrismAudit: local-first model evaluation, calibration and cohort parity exploration. No backend, no external services.">
|
| 9 |
+
<title>PrismAudit | Responsible ML Evaluation Studio</title>
|
| 10 |
+
<link rel="stylesheet" href="./styles.css">
|
| 11 |
+
<script type="module" src="./app.mjs"></script>
|
| 12 |
+
</head>
|
| 13 |
+
<body>
|
| 14 |
+
<div class="app-shell">
|
| 15 |
+
<aside class="sidebar" aria-label="Main navigation">
|
| 16 |
+
<div class="brand"><div class="brand-symbol" aria-hidden="true"><span></span><span></span><span></span></div><div><strong>prism<span>audit</span></strong><small>MODEL INTELLIGENCE</small></div></div>
|
| 17 |
+
<div class="side-label">WORKSPACE</div>
|
| 18 |
+
<nav class="nav" aria-label="Views">
|
| 19 |
+
<button class="nav-btn active" data-page="overview" aria-current="page"><span class="nav-icon">β</span> Overview <span class="nav-index">01</span></button>
|
| 20 |
+
<button class="nav-btn" data-page="threshold"><span class="nav-icon">β</span> Threshold explorer <span class="nav-index">02</span></button>
|
| 21 |
+
<button class="nav-btn" data-page="fairness"><span class="nav-icon">β₯</span> Fairness lens <span class="nav-index">03</span></button>
|
| 22 |
+
<button class="nav-btn" data-page="data"><span class="nav-icon">β€</span> Data & methods <span class="nav-index">04</span></button>
|
| 23 |
+
</nav>
|
| 24 |
+
<div class="side-label datasets-label">DATASETS</div>
|
| 25 |
+
<div class="side-panel"><label for="datasetSelect">Active dataset</label><select id="datasetSelect"><option value="support">Support prioritization</option><option value="quality">Quality review</option><option value="education">Training completion</option></select><div class="dataset-meta"><span class="dot"></span><span id="datasetMeta">Synthetic Β· 720 records</span></div></div>
|
| 26 |
+
<div class="side-bottom"><div class="status-dot"></div><span>LOCAL-FIRST ANALYTICS<br><small>Data stays in your browser</small></span></div>
|
| 27 |
+
</aside>
|
| 28 |
+
<div class="workspace">
|
| 29 |
+
<header class="topbar"><div class="breadcrumb">Portfolio / Project 24 <span>/</span> <b>Model Evaluation</b></div><div class="top-actions"><span class="release-pill">β v1.0 stable</span><button class="btn ghost" id="importTop" type="button">β Import CSV</button><button class="btn primary" id="exportReport" type="button">β Export audit</button></div></header>
|
| 30 |
+
<main>
|
| 31 |
+
<div class="hero"><div><div class="eyebrow"><span class="eyebrow-line"></span> EVALUATION WORKSPACE <span class="eyebrow-count">/ 024</span></div><h1>See beyond <em>accuracy.</em></h1><p>Understand classification performance, decision thresholds, calibration and cohort-level disparities, without sending your data anywhere.</p></div><div class="hero-art" aria-hidden="true"><svg viewBox="0 0 190 130" fill="none"><circle cx="92" cy="68" r="56" stroke="#345260" stroke-width="1"/><circle cx="92" cy="68" r="39" stroke="#345260" stroke-width="1"/><path d="M24 99L61 49L92 88L126 33L174 51" stroke="#b9ff84" stroke-width="2.6"/><path d="M24 115L61 90L92 102L126 78L174 84" stroke="#8b8fff" stroke-width="1.7"/><circle cx="126" cy="33" r="5" fill="#b9ff84"/><circle cx="61" cy="90" r="4" fill="#8b8fff"/><path d="M12 68H180M92 5V129" stroke="#243842" stroke-dasharray="3 5"/></svg></div></div>
|
| 32 |
+
<div class="demo-note" role="note"><span class="note-icon">i</span><span id="modeNotice"><strong>Demonstration mode.</strong> All records are fictional and algorithmically generated. No ML training happens in this application.</span></div>
|
| 33 |
+
<div class="control-strip"><div class="control-title"><div class="control-icon">β</div><div><b>Decision threshold</b><small>A record is predicted positive when score β₯ threshold</small></div></div><div class="range-group"><span class="range-min">0.00</span><input aria-label="Decision threshold" id="thresholdRange" type="range" min="0" max="100" step="1" value="50"><span class="range-max">1.00</span><output id="thresholdValue" for="thresholdRange">0.50</output></div><button id="resetThreshold" class="btn thin" type="button" title="Restore decision threshold to 0.50">Reset</button></div>
|
| 34 |
+
<div id="globalAlert" class="alert" role="status" aria-live="polite" hidden></div>
|
| 35 |
+
<section class="page" id="page-overview" aria-label="Overview">
|
| 36 |
+
<div class="section-header"><div><div class="micro">01 / PERFORMANCE</div><h2>Performance snapshot</h2></div><span class="section-detail" id="sampleSize">720 OBSERVATIONS</span></div>
|
| 37 |
+
<div class="metric-grid"><article class="metric-card accent-green"><div class="metric-caption">Accuracy <span class="metric-mark">β</span></div><div class="metric-value" id="metricAccuracy">β</div><div class="metric-desc">Overall correct predictions</div></article><article class="metric-card accent-purple"><div class="metric-caption">F1 score <span class="metric-mark">β</span></div><div class="metric-value" id="metricF1">β</div><div class="metric-desc">Precision / recall balance</div></article><article class="metric-card accent-blue"><div class="metric-caption">ROC AUC <span class="metric-mark">β¬</span></div><div class="metric-value" id="metricAuc">β</div><div class="metric-desc">Threshold-free discrimination</div></article><article class="metric-card accent-amber"><div class="metric-caption">Brier score <span class="metric-mark">β·</span></div><div class="metric-value" id="metricBrier">β</div><div class="metric-desc">Mean squared score error Β· lower is better</div></article></div>
|
| 38 |
+
<div class="chart-grid"><article class="panel"><div class="panel-head"><div><div class="micro">CLASSIFICATION QUALITY</div><h3>Confusion matrix</h3></div><span class="mini-tag">COUNTS</span></div><div class="confusion-layout"><div class="axis-label">PREDICTED CLASS β</div><div class="confusion-matrix"><div class="matrix-headers"><span></span><span>Actual positive</span><span>Actual negative</span></div><div class="matrix-row"><span>Positive</span><div class="matrix-cell positive"><strong id="tpCount">β</strong><small>True positive</small></div><div class="matrix-cell false-positive"><strong id="fpCount">β</strong><small>False positive</small></div></div><div class="matrix-row"><span>Negative</span><div class="matrix-cell false-negative"><strong id="fnCount">β</strong><small>False negative</small></div><div class="matrix-cell negative"><strong id="tnCount">β</strong><small>True negative</small></div></div></div></div><div class="panel-foot" id="confusionFoot">Threshold adjusted classification counts</div></article><article class="panel"><div class="panel-head"><div><div class="micro">DISCRIMINATION</div><h3>ROC curve</h3></div><span class="mini-tag">AUC <span id="aucTag">β</span></span></div><div id="rocChart" class="svg-chart" role="img" aria-label="Receiver operating characteristic curve"></div><div class="chart-legend"><span><i class="legend-dot green"></i> Model ROC</span><span><i class="legend-dot muted"></i> Random baseline</span></div></article></div>
|
| 39 |
+
<div class="chart-grid lower"><article class="panel"><div class="panel-head"><div><div class="micro">PROBABILITY QUALITY</div><h3>Reliability diagram</h3></div><span class="mini-tag">ECE <span id="eceTag">β</span></span></div><div id="calChart" class="svg-chart" role="img" aria-label="Predicted confidence versus observed frequency by decile"></div><div class="chart-legend"><span><i class="legend-dot purple"></i> Observed outcome</span><span><i class="legend-dot muted"></i> Perfect calibration</span></div></article><article class="panel insight-panel"><div class="panel-head"><div><div class="micro">AUTOMATED INTERPRETATION</div><h3>Audit signals</h3></div><span class="mini-tag">DESCRIPTIVE</span></div><div id="insightList" class="insight-list" aria-live="polite"></div><div class="panel-foot">Heuristics flag items to investigate; they are not certification, legal advice or causal findings.</div></article></div>
|
| 40 |
+
</section>
|
| 41 |
+
<section class="page" id="page-threshold" aria-label="Threshold explorer" hidden>
|
| 42 |
+
<div class="section-header"><div><div class="micro">02 / DECISION POLICY</div><h2>Threshold explorer</h2><p class="section-sub">Compare alternative operating points with no model retraining.</p></div><span class="section-detail">21 THRESHOLD POINTS</span></div>
|
| 43 |
+
<div class="metric-grid compact"><article class="metric-card accent-green"><div class="metric-caption">Precision</div><div class="metric-value" id="metricPrecision">β</div><div class="metric-desc">Correct among predicted positives</div></article><article class="metric-card accent-purple"><div class="metric-caption">Recall</div><div class="metric-value" id="metricRecall">β</div><div class="metric-desc">Captured actual positives</div></article><article class="metric-card accent-blue"><div class="metric-caption">Specificity</div><div class="metric-value" id="metricSpecificity">β</div><div class="metric-desc">Correct among actual negatives</div></article><article class="metric-card accent-amber"><div class="metric-caption">Balanced accuracy</div><div class="metric-value" id="metricBalanced">β</div><div class="metric-desc">Mean of recall and specificity</div></article></div>
|
| 44 |
+
<article class="panel full-panel"><div class="panel-head"><div><div class="micro">TRADE-OFFS</div><h3>Threshold performance landscape</h3></div><span class="mini-tag">DRAG THE SLIDER ABOVE</span></div><div class="large-chart" id="thresholdChart" role="img" aria-label="Precision, recall and F1 across thresholds"></div><div class="chart-legend"><span><i class="legend-dot green"></i> F1</span><span><i class="legend-dot purple"></i> Precision</span><span><i class="legend-dot blue"></i> Recall</span><span><i class="legend-dot amber"></i> Current threshold</span></div></article>
|
| 45 |
+
<div class="split-panels"><article class="panel"><div class="panel-head"><div><div class="micro">DESCRIPTIVE COMPARISON</div><h3>Strongest observed F1</h3></div></div><div class="suggestion-number" id="bestThreshold">β</div><p class="dim copy" id="bestNote">Highest F1 among the 21 displayed operating points. Not a deployment recommendation.</p><button class="btn outline" id="applyBest" type="button">Try this threshold β</button></article><article class="panel"><div class="panel-head"><div><div class="micro">UNCERTAINTY / RESAMPLING</div><h3>Bootstrap interval</h3></div></div><p class="dim copy">Compute a seeded 95% percentile interval for the current F1 using 240 bootstrap resamples. Interpret cautiously for small, clustered or shifted samples.</p><button class="btn outline" id="runBootstrap" type="button">Calculate 95% interval</button><div id="bootstrapResult" class="bootstrap-result" aria-live="polite">Awaiting calculation</div></article></div>
|
| 46 |
+
<article class="panel table-panel"><div class="panel-head"><div><div class="micro">REPRODUCIBILITY</div><h3>Operating point comparison</h3></div><button class="btn thin" id="exportSweep" type="button">β Export CSV</button></div><div class="table-wrap"><table><thead><tr><th>Threshold</th><th>F1</th><th>Precision</th><th>Recall</th><th>Accuracy</th><th>Positive rate</th></tr></thead><tbody id="sweepBody"></tbody></table></div></article>
|
| 47 |
+
</section>
|
| 48 |
+
<section class="page" id="page-fairness" aria-label="Fairness lens" hidden>
|
| 49 |
+
<div class="section-header"><div><div class="micro">03 / GROUP MONITORING</div><h2>Fairness lens</h2><p class="section-sub">Identify performance differences between labeled cohorts.</p></div><span class="section-detail">DESCRIPTIVE Β· NOT CAUSAL</span></div>
|
| 50 |
+
<div class="metric-grid compact"><article class="metric-card accent-green"><div class="metric-caption">Positive rate gap</div><div class="metric-value" id="gapParity">β</div><div class="metric-desc">Max β min predicted positive rates</div></article><article class="metric-card accent-purple"><div class="metric-caption">Recall gap</div><div class="metric-value" id="gapRecall">β</div><div class="metric-desc">Equal opportunity proxy</div></article><article class="metric-card accent-blue"><div class="metric-caption">False positive rate gap</div><div class="metric-value" id="gapFpr">β</div><div class="metric-desc">Largest between-group disparity</div></article><article class="metric-card accent-amber"><div class="metric-caption">Smallest cohort</div><div class="metric-value" id="minGroup">β</div><div class="metric-desc">Check sample size before interpretation</div></article></div>
|
| 51 |
+
<div class="fairness-warning" id="fairnessWarning" role="status"></div>
|
| 52 |
+
<article class="panel full-panel"><div class="panel-head"><div><div class="micro">GROUP PERFORMANCE</div><h3>Positive decision rates by cohort</h3></div><span class="mini-tag">COMPARE, NOT CONCLUDE</span></div><div id="groupBars" class="group-bars" role="img" aria-label="Decision rates for each cohort"></div></article>
|
| 53 |
+
<article class="panel table-panel"><div class="panel-head"><div><div class="micro">COHORT DISAGGREGATION</div><h3>Performance by group</h3></div><button class="btn thin" id="exportGroups" type="button">β Export CSV</button></div><div class="table-wrap"><table><thead><tr><th>Group</th><th>N</th><th>Base rate</th><th>Positive rate</th><th>Recall</th><th>False pos. rate</th><th>Accuracy</th></tr></thead><tbody id="groupsBody"></tbody></table></div><div class="panel-foot">Rates are undefined (β) when the required outcome class does not exist within a group. Cohorts of fewer than 30 rows are flagged as low sample size.</div></article>
|
| 54 |
+
<article class="method-card"><strong>Interpretation boundary</strong><p>Group gaps are descriptive. They do not identify discrimination, its cause, legal compliance or what decision should be made. Group definitions can encode sensitive characteristics or proxies. Examine data provenance, deployment context, error costs and uncertainty before drawing conclusions.</p></article>
|
| 55 |
+
</section>
|
| 56 |
+
<section class="page" id="page-data" aria-label="Data and methods" hidden>
|
| 57 |
+
<div class="section-header"><div><div class="micro">04 / DATA GOVERNANCE</div><h2>Data & methodology</h2><p class="section-sub">Transparent inputs, explicit assumptions and local processing.</p></div><span class="section-detail">NO BACKEND REQUIRED</span></div>
|
| 58 |
+
<div class="split-panels"><article class="panel upload-panel"><div class="panel-head"><div><div class="micro">YOUR EVALUATION DATA</div><h3>Import a CSV</h3></div><span class="mini-tag">LOCAL ONLY</span></div><div id="dropZone" class="drop-zone" tabindex="0" role="button" aria-label="Select or drop a CSV file"><div class="upload-icon">β</div><b>Drop your file here</b><span>or click to select a CSV, up to 2 MB / 10,000 rows</span><button class="btn outline" id="browseData" type="button">Choose CSV</button></div><input id="fileInput" type="file" accept=".csv,text/csv" hidden><div class="panel-foot">Required: <code>score</code> β [0,1], <code>label</code> β {0,1}, <code>group</code>. Optional: <code>id</code>. Avoid personal or confidential data when not necessary.</div><button class="btn thin" id="downloadSample" type="button">β Download example CSV</button></article><article class="panel"><div class="panel-head"><div><div class="micro">DATASET DIAGNOSTICS</div><h3>Current data profile</h3></div></div><div id="dataProfile" class="data-profile"></div><div class="panel-foot">Metrics are recomputed on each threshold change. The uploaded file remains in browser memory until the page is reloaded or replaced.</div><button class="btn thin" id="restoreDemo" type="button">βΆ Restore demo data</button></article></div>
|
| 59 |
+
<article class="panel methods-panel"><div class="panel-head"><div><div class="micro">HOW THIS WORKS</div><h3>Evaluation methods, precisely defined</h3></div><span class="mini-tag">OPEN SOURCE</span></div><div class="method-grid"><div><b>Threshold decision</b><p>A score greater than or equal to the selected cutoff is classified as positive. Existing scores are never modified.</p></div><div><b>ROC AUC</b><p>Tie-aware MannβWhitney rank statistic. It is undefined when all ground-truth labels have the same class.</p></div><div><b>Brier score / ECE</b><p>Brier is the mean squared difference between score and label. ECE uses 10 equal-width bins weighted by each bin's sample share.</p></div><div><b>Fairness gaps</b><p>Max minus min of group-level positive prediction rate, true positive rate and false positive rate. Undefined conditional rates are excluded and identified.</p></div><div><b>Bootstrap interval</b><p>Deterministic seeded row-level sampling with replacement, 240 repeats, percentile 2.5% to 97.5%. Not valid for every sampling design.</p></div><div><b>Limitations</b><p>Evaluation of existing binary scores only. No model is trained, no causal inference is claimed, and synthetic data are not evidence of real-world performance.</p></div></div></article>
|
| 60 |
+
<div class="closing-note"><span>β§</span> Built for transparent, reproducible ML evaluation. No analytics, trackers, external requests or server uploads.</div>
|
| 61 |
+
</section>
|
| 62 |
+
<footer class="footer"><span>PRISMAUDIT Β© 2026 Β· MIT LICENSE</span><span>HUGDRIGO / OPEN SOURCE PORTFOLIO Β· PROJECT 024</span></footer>
|
| 63 |
+
</main>
|
| 64 |
+
</div>
|
| 65 |
+
</div>
|
| 66 |
+
<div class="toast" id="toast" role="status" aria-live="polite"></div>
|
| 67 |
+
</body>
|
| 68 |
</html>
|
styles.css
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
:root{font-family:Inter,ui-sans-serif,system-ui,-apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif;color:#edf4f4;background:#0a1018;font-synthesis:none;--ink:#edf4f4;--muted:#92a3b0;--line:#24323d;--panel:#111c26;--panel2:#17232e;--green:#b4f887;--purple:#b2a1ff;--blue:#70d5f7;--amber:#ffc277}*{box-sizing:border-box}html{scroll-behavior:smooth}body{margin:0;min-width:320px}button,input,select{font:inherit}button{cursor:pointer}button:focus-visible,input:focus-visible,select:focus-visible,.drop-zone:focus-visible{outline:2px solid var(--green);outline-offset:3px}a{color:var(--green)}.app-shell{display:grid;grid-template-columns:254px minmax(0,1fr);min-height:100vh}.sidebar{background:#0b141d;border-right:1px solid #1f303b;padding:35px 20px 26px;display:flex;flex-direction:column;min-height:100vh;position:sticky;top:0;height:100vh}.brand{display:flex;align-items:center;gap:14px;margin:0 8px 71px}.brand strong{font-size:23px;letter-spacing:-1.15px;color:#f5ffff}.brand strong span{color:var(--green)}.brand small{display:block;font-size:9px;font-weight:700;color:#698193;letter-spacing:2px;margin-top:4px}.brand-symbol{width:33px;height:35px;position:relative;display:grid;place-items:center;transform:rotate(-24deg)}.brand-symbol span{position:absolute;border:2px solid var(--green);width:22px;height:27px;border-radius:2px;transform:skew(-8deg)}.brand-symbol span:nth-child(2){transform:translate(5px,-4px) skew(-8deg);border-color:#b2a1ff}.brand-symbol span:nth-child(3){transform:translate(-4px,4px) skew(-8deg);border-color:#60c9b8}.side-label{color:#62798b;font-size:10px;font-weight:800;letter-spacing:1.8px;margin:0 12px 17px}.nav{display:flex;flex-direction:column;gap:7px}.nav-btn{width:100%;background:transparent;color:#8399a8;text-align:left;border:1px solid transparent;border-radius:10px;padding:13px 11px;font-size:12px;font-weight:600;display:flex;align-items:center;gap:13px}.nav-btn:hover{color:#fff;background:#152430}.nav-btn.active{color:var(--green);background:#1b312e;border-color:#2b4b43}.nav-icon{font-size:20px;font-weight:400;width:21px;text-align:center}.nav-index{margin-left:auto;color:#597178;font-size:10px}.datasets-label{margin-top:53px}.side-panel{background:#121f2b;border:1px solid #263844;border-radius:11px;padding:14px}.side-panel label{display:block;color:#8297a3;font-size:11px;margin-bottom:9px}.side-panel select{width:100%;background:#0f1b26;border:1px solid #2b3d49;border-radius:6px;color:#eef4f2;padding:11px 6px;font-size:11px;min-width:0}.dataset-meta{display:flex;align-items:center;gap:7px;margin-top:12px;font-size:10px;color:#8196a5}.dot,.status-dot{width:7px;height:7px;background:var(--green);border-radius:50%;box-shadow:0 0 10px #b4f88755}.side-bottom{border-top:1px solid #1d303b;margin-top:auto;padding:25px 8px 5px;display:flex;align-items:flex-start;gap:12px;color:#d6e4e5;font-size:10px;line-height:1.8;letter-spacing:.9px;font-weight:700}.side-bottom small{color:#6b8594;letter-spacing:0;font-weight:500;font-size:11px}.status-dot{margin-top:5px}.workspace{min-width:0}.topbar{height:79px;display:flex;justify-content:space-between;align-items:center;border-bottom:1px solid #22313b;padding:0 48px;background:#0c151e}.breadcrumb{color:#75909c;font-size:12px}.breadcrumb span{padding:0 9px;color:#415b68}.breadcrumb b{color:#edf3f5;font-weight:600}.top-actions{display:flex;align-items:center;gap:10px}.release-pill{color:var(--green);background:#1b322e;border:1px solid #304b44;font-size:10px;padding:8px 12px;border-radius:40px;margin-right:9px}.btn{border:1px solid #354958;border-radius:8px;padding:11px 16px;color:#dae8e8;background:#162631;font-weight:700;font-size:11px;transition:background .15s,transform .15s;white-space:nowrap}.btn:hover{background:#27414a;transform:translateY(-1px)}.btn.primary{color:#112015;background:var(--green);border-color:var(--green)}.btn.primary:hover{background:#c7ffa6}.btn.ghost{background:transparent}.btn.thin{padding:9px 13px;background:#15232d;color:#b3c7cc;font-size:10px}.btn.outline{background:transparent;color:var(--green);border-color:#426751}main{max-width:1500px;margin:auto;padding:0 48px}.hero{padding:45px 0 27px;display:flex;justify-content:space-between;align-items:center;gap:20px}.eyebrow,.micro{font-size:10px;font-weight:800;color:#8a9caa;letter-spacing:2px}.eyebrow{display:flex;align-items:center;gap:9px;color:var(--green)}.eyebrow-line{width:20px;height:1px;background:var(--green)}.eyebrow-count{color:#73868d}h1,h2,h3,p{margin-top:0}h1{font-size:clamp(35px,4vw,62px);letter-spacing:-2.8px;font-weight:750;line-height:1.12;margin:17px 0 13px}h1 em{font-weight:700;font-style:normal;color:var(--green)}.hero p{max-width:630px;color:#8fa2ae;line-height:1.8;font-size:13px;margin:0}.hero-art{width:198px;min-width:180px;margin-right:15px;opacity:.95}.hero-art svg{display:block;width:100%}.demo-note{border:1px solid #33463d;background:#1c2b28;border-radius:9px;padding:11px 15px;color:#afc5bd;font-size:11px;display:flex;align-items:center;gap:11px;margin:12px 0 19px;line-height:1.5}.demo-note strong{color:var(--green)}.note-icon{border:1px solid #617c64;color:var(--green);width:20px;height:20px;text-align:center;line-height:17px;border-radius:50%;font-weight:700;flex-shrink:0}.control-strip{display:flex;align-items:center;gap:19px;background:#14212b;border:1px solid #31414c;border-radius:12px;padding:18px 20px;margin-bottom:36px}.control-title{display:flex;align-items:center;gap:12px;min-width:265px}.control-icon{width:37px;height:37px;border-radius:10px;background:#243a34;color:var(--green);display:grid;place-items:center;font-size:20px}.control-title b{font-size:12px;display:block}.control-title small{display:block;color:#80939e;font-size:10px;margin-top:4px}.range-group{display:flex;align-items:center;gap:12px;flex:1}.range-group input{width:100%;accent-color:var(--green);min-width:90px;cursor:pointer}.range-min,.range-max{color:#748c9b;font-size:10px}output{color:var(--green);font-size:17px;font-weight:800;min-width:42px;text-align:right}.alert{background:#3e2f20;border:1px solid #a46e41;border-radius:10px;color:#ffe0b3;padding:13px 17px;margin:0 0 20px;font-size:12px}.alert[hidden],.page[hidden]{display:none!important}.section-header{display:flex;align-items:flex-end;justify-content:space-between;gap:15px;margin:0 0 20px}.section-header h2{font-size:23px;letter-spacing:-.65px;margin:8px 0 0}.section-detail{font-size:10px;color:#748d9a;letter-spacing:1.2px;font-weight:700;white-space:nowrap}.section-sub{color:#869ca8;font-size:12px;margin:9px 0 0;line-height:1.6}.metric-grid{display:grid;grid-template-columns:repeat(4,minmax(0,1fr));gap:15px;margin-bottom:19px}.metric-card{background:#121f29;border:1px solid #293c46;border-radius:11px;padding:20px 19px 17px;min-height:151px;position:relative;overflow:hidden}.metric-card:before{content:"";position:absolute;top:0;left:0;right:0;height:2px;background:var(--accent)}.accent-green{--accent:var(--green)}.accent-purple{--accent:var(--purple)}.accent-blue{--accent:var(--blue)}.accent-amber{--accent:var(--amber)}.metric-caption{font-size:11px;color:#9caeb9;display:flex;justify-content:space-between}.metric-mark{font-size:17px;color:var(--accent)}.metric-value{font-size:36px;letter-spacing:-1.2px;font-weight:750;margin:19px 0 7px;color:var(--accent);line-height:1}.metric-desc{font-size:10px;color:#718795;line-height:1.6}.chart-grid{display:grid;grid-template-columns:1fr 1fr;gap:18px;margin-bottom:18px}.panel{background:#111d27;border:1px solid #293b47;border-radius:11px;padding:21px 24px;min-width:0}.panel-head{display:flex;align-items:flex-start;justify-content:space-between;gap:12px}.panel-head h3{font-size:16px;font-weight:700;letter-spacing:-.2px;margin:9px 0 15px}.mini-tag{border:1px solid #344752;background:#172631;border-radius:5px;color:#91a7b0;font-size:9px;font-weight:800;letter-spacing:.7px;padding:6px 8px;white-space:nowrap}.micro{color:#718a97;font-size:9px;letter-spacing:1.8px}.axis-label{font-size:9px;color:#78919e;letter-spacing:1px;margin-bottom:11px}.confusion-matrix{display:grid;gap:7px;max-width:510px}.matrix-headers,.matrix-row{display:grid;grid-template-columns:94px 1fr 1fr;gap:7px;align-items:center}.matrix-headers{color:#8197a4;text-align:center;font-size:10px}.matrix-row>span{color:#8da5b2;font-size:10px}.matrix-cell{border-radius:7px;padding:13px 12px;min-height:72px;display:flex;flex-direction:column;justify-content:center;align-items:center}.matrix-cell strong{font-size:27px;letter-spacing:-1px}.matrix-cell small{font-size:9px;margin-top:4px}.matrix-cell.positive{background:#234738;color:#b9ff93}.matrix-cell.negative{background:#233947;color:#91def8}.matrix-cell.false-positive{background:#46382f;color:#ffd8ab}.matrix-cell.false-negative{background:#3e2f44;color:#e1bbf4}.panel-foot{color:#718795;font-size:10px;line-height:1.7;margin-top:20px;padding-top:14px;border-top:1px solid #283844}.svg-chart{height:245px;width:100%;margin-top:5px}.svg-chart svg,.large-chart svg{width:100%;height:100%;display:block}.chart-legend{display:flex;gap:20px;align-items:center;color:#8ca4af;font-size:10px;padding-top:11px}.legend-dot{display:inline-block;width:7px;height:7px;border-radius:50%;margin-right:7px}.legend-dot.green{background:var(--green)}.legend-dot.purple{background:var(--purple)}.legend-dot.blue{background:var(--blue)}.legend-dot.amber{background:var(--amber)}.legend-dot.muted{background:#5e7382}.insight-list{display:flex;flex-direction:column;gap:12px;padding-top:6px}.insight{display:flex;gap:12px;align-items:flex-start;background:#192934;border:1px solid #2b404b;border-radius:8px;padding:13px}.insight-icon{width:26px;height:26px;display:grid;place-items:center;border-radius:6px;background:#254137;color:var(--green);flex-shrink:0}.insight.warn .insight-icon{background:#453828;color:var(--amber)}.insight p{margin:4px 0 0;color:#8299a6;line-height:1.6;font-size:11px}.insight b{font-size:11px}.large-chart{height:320px}.full-panel{margin-bottom:18px}.compact .metric-card{min-height:141px}.split-panels{display:grid;grid-template-columns:1fr 1fr;gap:18px;margin-bottom:18px}.suggestion-number{font-size:41px;font-weight:800;color:var(--green);padding-top:4px}.dim{color:#8ca1ae}.copy{font-size:12px;line-height:1.75}.bootstrap-result{color:var(--purple);background:#22273a;border:1px solid #393d57;margin-top:15px;padding:14px;border-radius:7px;font-size:12px;font-weight:700}.table-panel{margin-bottom:18px}.table-wrap{overflow:auto}.table-wrap table{border-collapse:collapse;width:100%;white-space:nowrap;text-align:left;font-size:11px}.table-wrap th{font-size:9px;letter-spacing:1px;color:#7c96a4;padding:13px 10px;border-bottom:1px solid #2e404b}.table-wrap td{padding:11px 10px;color:#b9cdd4;border-bottom:1px solid #22333d}.table-wrap tr:last-child td{border-bottom:0}.table-wrap tr.current{background:#254033}.table-wrap tr.current td:first-child{color:var(--green);font-weight:800}.fairness-warning{border-radius:9px;padding:13px 16px;background:#2c322a;border:1px solid #5a5e3b;color:#e8d29a;line-height:1.6;font-size:11px;margin-bottom:19px}.fairness-warning.good{background:#1c332c;border-color:#426b51;color:#b6eac0}.group-bars{display:flex;flex-direction:column;gap:23px;padding:16px 2px 26px}.group-row{display:grid;grid-template-columns:110px 1fr 52px;gap:16px;align-items:center}.group-row .group-label{color:#b5c6cb;font-size:11px;overflow-wrap:anywhere}.group-bar{height:13px;border-radius:5px;background:#263945;overflow:hidden}.group-bar>div{height:100%;background:linear-gradient(90deg,#7fd3c6,#b4f887);border-radius:5px}.group-row .group-val{font-size:11px;font-weight:700;color:var(--green);text-align:right}.method-card{border-left:3px solid var(--purple);padding:18px 21px;background:#1a2432;border-radius:0 10px 10px 0;margin-bottom:24px}.method-card strong{font-size:12px;color:#cfcbff}.method-card p{font-size:12px;color:#98a9b6;line-height:1.8;margin:7px 0 0}.drop-zone{border:1px dashed #56716c;border-radius:11px;background:#172b2c;min-height:240px;display:flex;flex-direction:column;align-items:center;justify-content:center;gap:12px;padding:25px 18px;text-align:center;color:#e3f4ed}.drop-zone:hover,.drop-zone.dragging{background:#233c36;border-color:var(--green)}.drop-zone b{font-size:14px}.drop-zone span{font-size:11px;color:#95afa8}.upload-icon{border:1px solid #668a74;color:var(--green);border-radius:50%;width:45px;height:45px;display:grid;place-items:center;font-size:28px}.upload-panel>.btn{margin-top:14px}.data-profile{display:grid;grid-template-columns:1fr 1fr;gap:13px;padding:10px 0}.data-item{background:#172833;border:1px solid #2e444c;border-radius:8px;padding:18px}.data-item strong{color:#bdf3d3;display:block;font-size:22px;margin-top:8px}.data-item span{font-size:10px;color:#849caa}.methods-panel{margin-bottom:18px}.method-grid{display:grid;grid-template-columns:repeat(3,minmax(0,1fr));gap:18px;padding-top:10px}.method-grid>div{border-top:1px solid #344450;padding-top:16px}.method-grid b{font-size:12px}.method-grid p{font-size:11px;color:#8ba1ad;line-height:1.75;margin:8px 0 0}.closing-note{padding:21px;color:#8eb1ad;font-size:12px;text-align:center;background:#182729;border-radius:9px;border:1px solid #304e43}.closing-note span{color:var(--green);padding-right:9px;font-size:18px}.footer{border-top:1px solid #243540;padding:30px 0;margin-top:36px;display:flex;justify-content:space-between;gap:10px;color:#66808e;font-size:9px;letter-spacing:1.2px}.toast{position:fixed;right:24px;bottom:24px;max-width:440px;background:#2b4b40;color:#ecfff0;border:1px solid #5b9f7c;border-radius:9px;padding:15px 19px;font-size:12px;box-shadow:0 13px 32px #0008;opacity:0;pointer-events:none;transform:translateY(15px);transition:opacity .2s,transform .2s;z-index:10}.toast.show{opacity:1;transform:translateY(0)}.toast.error{background:#49362b;border-color:#bf785c;color:#fff1de}
|
| 2 |
+
@media(max-width:1200px){.app-shell{grid-template-columns:220px minmax(0,1fr)}main{padding:0 28px}.topbar{padding:0 28px}.sidebar{padding-left:13px;padding-right:13px}.control-title{min-width:235px}.metric-card{padding:17px 13px}.metric-value{font-size:30px}.panel{padding:19px 16px}.matrix-headers,.matrix-row{grid-template-columns:68px 1fr 1fr}.matrix-cell{padding:10px 5px}.hero-art{width:150px;min-width:130px}}
|
| 3 |
+
@media(max-width:900px){.app-shell{display:block}.sidebar{height:auto;min-height:0;position:static;display:block;padding:16px 22px 12px;border-right:0;border-bottom:1px solid #30404c}.brand{margin:0 0 17px}.side-label,.side-panel,.side-bottom{display:none}.nav{display:flex;flex-direction:row;overflow-x:auto;padding-bottom:4px}.nav-btn{width:auto;white-space:nowrap;padding:10px 12px}.nav-index{display:none}.topbar{height:68px}main{padding:0 23px}.metric-grid{grid-template-columns:repeat(2,minmax(0,1fr))}.control-strip{flex-wrap:wrap}.control-title{flex:1}.range-group{flex:2}.hero-art{display:none}}
|
| 4 |
+
@media(max-width:620px){.sidebar{padding:16px 14px 9px}.brand strong{font-size:19px}.nav{gap:5px}.nav-btn{font-size:11px;padding:9px}.topbar{padding:0 16px;height:auto;min-height:68px;flex-wrap:wrap;gap:7px}.breadcrumb{font-size:10px}.release-pill{display:none}.top-actions{gap:5px}.top-actions .btn{padding:9px 8px;font-size:9px}main{padding:0 16px}.hero{padding:32px 0 20px}.hero h1{font-size:38px;letter-spacing:-1.6px}.hero p{font-size:12px}.control-strip{padding:13px;gap:13px}.control-title{min-width:100%}.range-group{min-width:calc(100% - 76px);gap:5px}.range-min,.range-max{display:none}.control-strip>.btn{margin-left:auto}.section-header{align-items:flex-start}.section-detail{font-size:9px;padding-top:3px}.section-header h2{font-size:21px}.metric-grid{gap:10px}.metric-card{padding:14px 12px;min-height:132px}.metric-value{font-size:29px}.metric-caption{font-size:10px}.metric-desc{font-size:9px}.chart-grid,.split-panels{grid-template-columns:1fr}.svg-chart{height:240px}.group-row{grid-template-columns:85px 1fr 40px;gap:7px}.method-grid{grid-template-columns:1fr}.footer{font-size:8px;line-height:1.7}.matrix-headers,.matrix-row{grid-template-columns:82px 1fr 1fr}.panel{padding:17px}.release-pill{display:none}}
|
| 5 |
+
@media(prefers-reduced-motion:reduce){*{scroll-behavior:auto!important;transition:none!important}}
|