Hugdrigo commited on
Commit
7c9e8d9
·
verified ·
1 Parent(s): 4905d8d

Upload 28 files

Browse files
CHANGELOG.md ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Changelog
2
+
3
+ ## v2.0.0 · 2026-10-08
4
+
5
+ - New research workspace built on existing Academic Rhetorical Intent Mini v1.1 synthetic data.
6
+ - Four deterministic JavaScript lexical classifiers trained in the browser on fixed 842-row training partition.
7
+ - Added 130-row controlled IID and 54-row held-out novel-pattern comparisons.
8
+ - Added deterministic metrics, class confusion, language and domain slices, error inspection, seed-controlled bootstrap sampling, and experiment summary registry.
9
+ - Added provenance audit, limitations, real-world transfer cautions, and CSV / JSON / Markdown exports.
10
+ - Added independent Node test suite, Node CLI benchmark, and optional browser QA.
11
+ - Added polished desktop/mobile UI and no paid-server dependencies.
12
+
13
+ No existing ResearchLens, CareerMatch AI, ModelBench or Academic Rhetoric Lab repository is overwritten by this separate Space.
DATA_CARD.md ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Dataset Card · Academic Rhetorical Intent Mini v1.1
2
+
3
+ ## Overview
4
+
5
+ A dataset of **1,026 synthetic-curated academic-style single sentences**, previously developed for a multilingual research-classification experiment. This project embeds a read-only JSON conversion of the original source CSV.
6
+
7
+ ## Dataset fields
8
+
9
+ `id` stable example key; `text` synthetic sentence; `language` (`en`, `es`, `pt`); `label` one of `background`, `objective`, `method`, `result`, `limitation`, `implication`; `domain` topical source domain; `template_group` synthetic generation family. The source CSV additionally contains `split`, `source_type`.
10
+
11
+ ## Fixed partitions
12
+
13
+ | Partition | Size | Use |
14
+ |---|---:|---|
15
+ | train | 842 | Fit the baseline algorithms |
16
+ | test | 130 | Controlled, likely optimistic IID evaluation |
17
+ | challenge | 54 | Fixed novel-pattern synthetic evaluation |
18
+
19
+ No repartitioning is performed at startup. Original CSV entries are included so all labels, source fields and partitions can be verified.
20
+
21
+ ## Scope / limitations
22
+
23
+ The examples are artificial. They were not sampled randomly from peer-reviewed publications and are not a representative corpus for language-model fairness, plagiarism, or scholarly decision-making. The dataset contains templates and may have related paraphrase families across partitions even when exact overlap is zero. No student data or private manuscripts were used.
24
+
25
+ ## License
26
+
27
+ CC BY 4.0, as stated by the original dataset card (`ORIGINAL_DATASET_CARD.md`), separate from the application MIT license.
DATA_LICENSE.md ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ # Data license and attribution
2
+
3
+ The bundled `academic_rhetoric_intent.csv`, `train.json`, `iid_test.json`, and `challenge.json` contain the same synthetic-curated examples from **Academic Rhetorical Intent Mini v1.1** by Rodrigo Pantoja, licensed under **Creative Commons Attribution 4.0 International (CC BY 4.0)**.
4
+
5
+ License text: https://creativecommons.org/licenses/by/4.0/legalcode
6
+
7
+ **Attribution:** Academic Rhetorical Intent Mini v1.1, Rodrigo Pantoja (2026), synthetic-curated multilingual academic-sentence dataset, CC BY 4.0. Dataset JSONs in this repository were converted from the original CSV without changing texts, labels, or split assignments. Changes: CSV-to-JSON structural conversion and a standalone manifest.
8
+
9
+ The application source files are under the separate MIT license. Neither license implies verified model quality or unrestricted rights to non-project third-party material.
DEPLOYMENT_ES.md ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Guía sencilla para publicar ModelOps Research Lab en Hugging Face
2
+
3
+ **No hace falta programar ni pagar.** Este es un Space `Static` gratuito.
4
+
5
+ 1. Descarga `ModelOps_Research_Lab_ARCHIVOS_LISTOS_PARA_SUBIR.zip` desde ChatGPT.
6
+ 2. En tu Mac abre **Finder → Descargas**, y haz doble clic para descomprimirlo.
7
+ 3. Abre https://huggingface.co e inicia sesión en la cuenta **Hugdrigo**.
8
+ 4. Haz clic en **+ New → Space → Manual setup**.
9
+ 5. Completa el formulario:
10
+ - Owner: `Hugdrigo`
11
+ - Space name: `modelops-research-lab`
12
+ - Short description: `Reproducible multilingual NLP model evaluation and audits.`
13
+ - License: `MIT` seleccionada del menú. Si la creación rechaza el campo, déjalo vacío; el README del paquete incluye `license: mit`.
14
+ - SDK: `Static`
15
+ - Template: `Blank`
16
+ - Visibility: `Public`
17
+ 6. Haz clic en **Create Space**.
18
+ 7. Abre directamente la lista de archivos https://huggingface.co/spaces/Hugdrigo/modelops-research-lab/tree/main (evita el editor).
19
+ 8. Elige **+ Contribute → Upload files**. En Finder abre la carpeta descomprimida, presiona **⌘ Command + A** y arrastra TODOS los archivos al recuadro de carga. Sube archivos individuales, no el ZIP cerrado.
20
+ 9. Comprueba que no haya mensajes rojos. Pulsa **Commit changes to main**.
21
+ 10. Abre https://huggingface.co/spaces/Hugdrigo/modelops-research-lab y entra en **App**. Espera que aparezca `Ready. Four models trained locally`.
22
+ 11. Revisa **Overview** y pulsa **Experiments → Run four baselines**.
23
+ 12. En **Error lab**, busca fallos y prueba una frase. En **Export & governance** descarga `Metrics JSON` y `Prediction CSV`.
24
+
25
+ ### Resultado esperado
26
+
27
+ En el conjunto `Novel-pattern challenge · 54`, el mejor de los cuatro métodos deberá mostrar aproximadamente **77.8% Accuracy**. En el conjunto `Controlled IID · 130` mostrará una puntuación más alta. Este es un dataset **sintético**, no un certificado de validez científica.
28
+
29
+ ### Si aparece un error
30
+
31
+ - `Missing SDK`: revisa que `README.md` esté en la raíz, y que la cabecera incluya `sdk: static` y `app_file: index.html`.
32
+ - Mensaje de metadata: no edites el README. Comparte la captura para corregir el archivo.
33
+ - `Could not initialize`: verifica que `train.json`, `iid_test.json`, `challenge.json`, `engine.js`, `app.js`, `styles.css` e `index.html` estén en la raíz junto al README.
34
+ - Si la página aparece vacía, espera unos segundos y recarga una sola vez.
35
+
36
+ **No borres ni vuelvas a crear tus Spaces anteriores.**
LICENSE ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Rodrigo Pantoja
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
22
+
23
+ Note: Bundled dataset materials are separately CC BY 4.0 as stated in DATA_LICENSE.md.
LIMITATIONS.md ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Limitations, safeguards and next research steps
2
+
3
+ - **Synthetic-only benchmarks:** accuracy and macro F1 are not estimates of real-world scholarly-document performance.
4
+ - **Domain shift:** the controlled IID set gives substantially higher performance than the fixed novel-pattern challenge. The gap is visible in the app. A prior *different* Python classifier only reached 35.7% on a 28-sentence PubMed development smoke sample, demonstrating a transfer concern but not a definitive external evaluation for this app.
5
+ - **Dataset size:** the challenge has only 54 rows; small subgroup samples make comparisons fragile.
6
+ - **Leakage audit:** exact text and ID overlap checks cannot rule out shared templates or semantically similar training text.
7
+ - **Tokenizer constraints:** accent normalization and word/character n-grams do not understand discourse context or domain meaning.
8
+ - **Uncertainty:** bootstrap resampling from a fixed synthetic set does not capture selection bias or future deployment shift.
9
+ - **Scoring:** Naive Bayes outputs are not calibrated; cosine scores are not probabilities.
10
+ - **Fairness:** language/domain slice breakdowns are descriptive diagnostics, not equity certification or bias-free evidence.
11
+ - **Risk:** no individual-level consequential use (grading, admissions, hiring, publication acceptance, misconduct allegations).
12
+ - **External data:** a future phase could seek permitted, independent, human-labeled corpora; preserve source licensing and consent as needed. New external scores must come from real executed experiments, never estimated or fabricated.
METHODS.md ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Methods and evaluation protocol
2
+
3
+ ## Lexical models
4
+
5
+ All models are deterministic and train in JavaScript on the 842 `train.json` rows.
6
+
7
+ 1. **Multinomial Naive Bayes:** token count frequencies for Unicode-normalized word features (accent stripping for compatibility), per-class Laplace smoothing alpha=1, additive log-likelihoods and empirical smoothed class priors.
8
+ 2. **Word TF-IDF centroid:** per-document TF=(1+log(count)) and IDF=(log((N+1)/(df+1))+1). L2-normalize each training document vector; sum by class; normalize the final class centroid; classify by cosine similarity.
9
+ 3. **Character trigram TF-IDF centroid:** same TF-IDF scheme on normalized, space-padded character 3-grams. Language-neutral, lexical (not semantic).
10
+ 4. **Weighted hybrid:** 0.55 × word-centroid cosine + 0.45 × character-centroid cosine. Fixed mixture weights; not learned from the held-out challenge.
11
+
12
+ ## Evaluation metrics
13
+
14
+ Accuracy = correct / total. Class F1=2PR/(P+R), P=TP/(TP+FP) and R=TP/(TP+FN), with division-by-zero returning 0. Macro F1 averages classes with positive true-label support. The overall test and challenge sets contain all six labels. Subgroup results depend on subgroup supports and can be unstable for small n.
15
+
16
+ `engine.js` computes all predictions at runtime, then aggregates confusion matrices, language and domain slices, and example-level error rows. Historical Python-model results appear only as clearly separate context.
17
+
18
+ ## Uncertainty calculation
19
+
20
+ The `Experiments` tab uses a seeded Mulberry32 pseudo-random generator to draw 200 or 500 bootstrap resamples of held-out prediction rows, with replacement. The 2.5 and 97.5 percentiles form an *approximate sample-resampling interval* for Accuracy. This interval does not include uncertainty from training procedures, domain sampling, annotators, temporal changes or feature choices. Inference confidence scores in Error lab are uncalibrated similarity or posterior-like outputs, and must not be interpreted as reliable correctness probabilities.
21
+
22
+ ## Release gates
23
+
24
+ - Recompute both fixed splits; never train on evaluation rows.
25
+ - Audit IDs and exact text overlaps; inspect sample diversity in future work.
26
+ - Disclose large IID-to-novel-pattern generalization gaps.
27
+ - Require a separate external human-labeled research benchmark before making claims of real-world utility.
MODEL_CARD.md ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Model Card · ModelOps Research Lab baselines
2
+
3
+ ## Intended purpose
4
+
5
+ Research demonstration of reproducible, browser-trained multilingual academic-sentence classification. Labels: background, objective, method, result, limitation, implication.
6
+
7
+ ## Model architectures
8
+
9
+ Not neural networks. Four transparent lexical baselines: multinomial Naive Bayes, word TF-IDF class centroid, character trigram TF-IDF class centroid and a fixed 55/45 centroid hybrid. No pretrained weights and no API inference.
10
+
11
+ ## Training / evaluation
12
+
13
+ Training on the 842-row original fixed synthetic-curated training set (EN / ES / PT). Held-out assessments use 130 controlled IID examples and 54 fixed novel-pattern synthetic examples. Benchmark code is published in `engine.js` and reproduced by `benchmark_cli.js`. The release metrics are in `evaluation_results.json`.
14
+
15
+ ## Scores
16
+
17
+ | Model | IID Accuracy | Challenge Accuracy | Challenge Macro F1 |
18
+ |---|---:|---:|---:|
19
+ | NB | 100.0% | 74.1% | 73.6% |
20
+ | Word centroid | 98.5% | 72.2% | 70.6% |
21
+ | Char centroid | 95.4% | 70.4% | 68.4% |
22
+ | Hybrid | 97.7% | 77.8% | 76.2% |
23
+
24
+ No external human-labeled validation has been performed for **these** browser models. A separate archived Python classifier was once smoke-tested on PubMed; those figures are contextual only and not directly comparable.
25
+
26
+ ## Known limitations and risks
27
+
28
+ Controlled synthetic data, likely recurring templates, limited domain scope, demographic/fairness relevance not assessed, no calibrated probabilities, unreliable transfer to real scholarly prose. **Prohibited for grading, academic-integrity judgments, student assessment, admissions, employment selection or publication decisions.**
29
+
30
+ ## License
31
+
32
+ MIT code; CC BY 4.0 training/evaluation dataset (see DATA_LICENSE.md).
ORIGINAL_DATASET_CARD.md ADDED
@@ -0,0 +1,80 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-4.0
3
+ language:
4
+ - en
5
+ - es
6
+ - pt
7
+ task_categories:
8
+ - text-classification
9
+ pretty_name: Academic Rhetorical Intent Mini
10
+ size_categories:
11
+ - 1K<n<10K
12
+ ---
13
+
14
+ # Academic Rhetorical Intent Mini
15
+
16
+ A multilingual **synthetic-curated** dataset for classifying rhetorical function in academic-style sentences.
17
+
18
+ ## Labels
19
+ `background`, `objective`, `method`, `result`, `limitation`, `implication`
20
+
21
+ ## Composition
22
+ - Training examples: **842**
23
+ - Controlled IID test: **130**
24
+ - Fixed novel-pattern challenge: **54**
25
+ - Total: **1026**
26
+ - Languages: English, Spanish, Portuguese
27
+
28
+ The training set was deliberately expanded with additional paraphrase families after an early prototype showed poor generalization. **The challenge set remained unchanged and was never added to training.**
29
+
30
+ ## Provenance
31
+ All examples were newly authored for this project. No journal articles, student submissions or private documents were copied.
32
+
33
+ ## Intended uses
34
+ Educational NLP, multilingual classification baselines, academic-writing support prototypes and reproducible experimentation.
35
+
36
+ ## Limitations
37
+ The corpus is synthetic and controlled. Real scholarly discourse is more ambiguous, domain-specific and context-dependent.
38
+
39
+ ## High-stakes restriction
40
+ Do not use for grading, misconduct detection, admissions, publication acceptance or other consequential decisions about people.
41
+
42
+ ## License
43
+ CC BY 4.0.
44
+
45
+ ## Dataset description
46
+
47
+ Academic Rhetorical Intent Mini is a multilingual, synthetic-curated sentence-classification dataset designed to support reproducible experiments on academic rhetorical function.
48
+
49
+ Each row represents one academic-style sentence labeled as `background`, `objective`, `method`, `result`, `limitation` or `implication`.
50
+
51
+ ## Methodology / processing
52
+
53
+ The corpus was created from manually designed rhetorical sentence families in English, Spanish and Portuguese, combined with controlled domain substitutions across education, business, economics, international relations, marketing and public policy.
54
+
55
+ After an early prototype exposed poor generalization, additional paraphrase families were added only to the training split. The fixed challenge set remained unchanged.
56
+
57
+ Processing steps:
58
+
59
+ 1. author rhetorical sentence families;
60
+ 2. generate domain/language combinations;
61
+ 3. assign stable labels and IDs;
62
+ 4. split controlled examples into train/test;
63
+ 5. maintain a separate fixed novel-pattern challenge set;
64
+ 6. export CSV and JSONL representations;
65
+ 7. run duplicate and schema checks.
66
+
67
+ ## Fields / schema
68
+
69
+ | Field | Type | Description |
70
+ |---|---|---|
71
+ | `id` | string | Stable example identifier |
72
+ | `language` | string | `en`, `es`, or `pt` |
73
+ | `label` | string | Rhetorical-function target |
74
+ | `domain` | string | Content domain used in sentence construction |
75
+ | `text` | string | Academic-style sentence |
76
+ | `source_type` | string | Synthetic-curated provenance flag |
77
+ | `template_group` | string | Generation family used for leakage control |
78
+ | `split` | string | `train`, `test`, or `challenge` |
79
+
80
+ The dataset contains no journal articles, student submissions or private documents.
QA_REPORT.md ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ModelOps Research Lab · Validation report
2
+
3
+ **Release:** v2.0.0
4
+ **Date:** 2026-10-08
5
+ **Environment:** Node 22, Python Playwright / Chromium headless (local test fixtures), no paid APIs.
6
+
7
+ ## Test results
8
+
9
+ | Validation | Result |
10
+ |---|---|
11
+ | Core JavaScript unit tests (`node --test test_engine.js`) | **34/34 PASS** |
12
+ | Independent executable benchmark (`node benchmark_cli.js`) | **PASS** |
13
+ | Browser workflow (navigation, controls, scoring, local history, exports) | **PASS** |
14
+ | Browser JavaScript errors | **0** |
15
+ | Desktop preview | Captured |
16
+ | Mobile 390 px layout | Captured, document scrollWidth = 390 px |
17
+ | Hugging Face Static metadata preflight (`python preflight.py`) | **PASS** |
18
+ | Exact data integrity and SHA-256 check | **PASS** |
19
+
20
+ ## Recomputed results
21
+
22
+ | Baseline | Controlled IID accuracy, n=130 | Fixed challenge accuracy, n=54 | Fixed challenge Macro F1 |
23
+ |---|---:|---:|---:|
24
+ | Multinomial Naive Bayes | 100.0% | 74.1% | 73.6% |
25
+ | Word TF-IDF centroid | 98.5% | 72.2% | 70.6% |
26
+ | Character TF-IDF centroid | 95.4% | 70.4% | 68.4% |
27
+ | Word/character hybrid | 97.7% | **77.8%** | 76.2% |
28
+
29
+ These figures are reproducible from the bundled **synthetic-curated** source corpus. They are not validated against independent real-world scholarly text.
30
+
31
+ ## Browser checks performed
32
+
33
+ 1. Startup, dataset fetch and model training completed.
34
+ 2. Leaderboard and confusion matrix rendered.
35
+ 3. Test/challenge selector updated dashboard.
36
+ 4. Seeded experiment runner generated four results, intervals and local history.
37
+ 5. Per-language and per-domain performance tables updated.
38
+ 6. Error inspection and keyword filtering worked.
39
+ 7. Sentence-level inference returned a label.
40
+ 8. Domain-shift comparison rendered.
41
+ 9. Data-integrity audit rendered.
42
+ 10. Four downloads were created: prediction CSV, metrics JSON, Markdown report and run history JSON.
43
+ 11. Desktop preview was captured.
44
+ 12. Mobile application loaded without horizontal document overflow.
45
+ 13. Mobile research-module navigation worked.
46
+ 14. No uncaught browser JavaScript errors.
47
+
48
+ ## Important test-scope limitation
49
+
50
+ This test used directly injected local CSS, HTML, and JavaScript and simulated **only the three static dataset JSON fetches**; no external network or paid AI service is involved in the project. Hugging Face hosting itself has not yet been tested in the user account. The final deployment requires the user to create and upload the Space and confirm that it loads publicly.
51
+
52
+ ## Dataset licensing
53
+
54
+ Training and evaluation texts derive from Academic Rhetorical Intent Mini v1.1, CC BY 4.0. Browser software is MIT. These separate terms are declared in `README.md`, `DATA_LICENSE.md`, and `LICENSE`.
README.md CHANGED
@@ -1,12 +1,91 @@
1
  ---
2
- title: Modelops Research Lab
3
- emoji: ⚡
4
- colorFrom: gray
5
- colorTo: purple
6
  sdk: static
 
7
  pinned: false
8
  license: mit
9
  short_description: Reproducible multilingual NLP model evaluation and audits.
 
 
 
 
 
 
 
 
10
  ---
11
 
12
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: ModelOps Research Lab
3
+ emoji: "🧪"
4
+ colorFrom: indigo
5
+ colorTo: blue
6
  sdk: static
7
+ app_file: index.html
8
  pinned: false
9
  license: mit
10
  short_description: Reproducible multilingual NLP model evaluation and audits.
11
+ tags:
12
+ - static
13
+ - model-evaluation
14
+ - nlp
15
+ - multilingual
16
+ - reproducibility
17
+ - experiment-tracking
18
+ - explainable-ai
19
  ---
20
 
21
+ # 🧪 ModelOps Research Lab v2.0
22
+
23
+ An offline-first, zero-cost browser application for **reproducible multilingual text-classification experiments**, created for the applied AI research portfolio of Rodrigo Pantoja (`@Hugdrigo`).
24
+
25
+ ## Purpose
26
+
27
+ Move beyond attractive demos to **measured model behavior**: training, fixed evaluation partitions, experimental comparisons, error inspection, language/domain slices, bootstrap intervals, source audits, and traceable result exports.
28
+
29
+ ## Quickstart
30
+
31
+ 1. Open **App**. The four transparent lexical baselines are fitted *in your browser* on the included 842 synthetic training examples.
32
+ 2. See **Overview** for the 54-item novel-pattern challenge, and switch to the controlled 130-item IID test.
33
+ 3. Open **Experiments** and click **Run four baselines** to create a locally saved run with seed-controlled bootstrap sampling.
34
+ 4. Visit **Slice analysis**, **Error lab**, **Shift diagnostics** and **Data audit**.
35
+ 5. Export predictions (CSV), metrics (JSON), reports (Markdown), and run history (JSON).
36
+
37
+ ## Exact research data and licensing
38
+
39
+ Source: **Academic Rhetorical Intent Mini v1.1** (originally developed as an earlier portfolio research dataset). All **1,026 examples are synthetic-curated**, written for educational experimental use; no real scholarly manuscripts or student submissions were copied. Distribution: 842 training, 130 controlled IID, 54 held-out novel-pattern challenge. Languages: English, Spanish, Portuguese. Six rhetorical labels: background, objective, method, result, limitation, implication.
40
+
41
+ **Dataset**: CC BY 4.0. See `DATA_LICENSE.md` and `ORIGINAL_DATASET_CARD.md`. **App and baseline engine**: MIT. See `LICENSE`. The README's `license: mit` denotes the Space's application code and does not override the included dataset attribution.
42
+
43
+ ## Tested baselines and evaluation
44
+
45
+ All four algorithms are implemented in `engine.js` and trained from `train.json` on every page load:
46
+
47
+ | Method | Controlled IID Accuracy (130) | Novel-pattern Accuracy (54) | Novel-pattern Macro F1 |
48
+ |---|---:|---:|---:|
49
+ | Multinomial Naive Bayes | 100.0% | 74.1% | 73.6% |
50
+ | Word TF-IDF centroid | 98.5% | 72.2% | 70.6% |
51
+ | Character trigram TF-IDF centroid | 95.4% | 70.4% | 68.4% |
52
+ | Weighted word/character hybrid | 97.7% | **77.8%** | 76.2% |
53
+
54
+ These are **new browser baselines**; they do not reproduce the previously trained Python model. The hybrid is a weighted combination of two baselines, not a separately trained neural network. Scores are derived from the fixed **synthetic** dataset, not a scientific claim of transferable academic performance. See `MODEL_CARD.md`, `METHODS.md` and `LIMITATIONS.md`.
55
+
56
+ A historical *different Python model* reported 79.6% challenge Accuracy and only 35.7% on a very small 28-sentence PubMed development smoke test. The latter is shown separately in the dashboard to remind users about domain shift, **not** to imply that the browser classifiers were evaluated on PubMed.
57
+
58
+ ## Reproduce without a browser
59
+
60
+ ```bash
61
+ node --test test_engine.js
62
+ node benchmark_cli.js
63
+ ```
64
+
65
+ Node 18+ required. No npm install needed. In-browser operation needs JavaScript and no paid API. These tests do not require internet.
66
+
67
+ ## Methodological notes
68
+
69
+ - No training or hyperparameter fitting uses the holdout splits.
70
+ - Method weights and feature definitions are specified in code, **not optimized on the challenge**.
71
+ - Macro F1 averages labels present in the evaluated group. For 130/54 full sets, all six labels are present.
72
+ - Bootstrap intervals are **resampling intervals across fixed held-out rows**. They are not estimates of uncertainty due to training sets, label noise or real-world deployment.
73
+ - The Data audit checks exact text / ID overlap. **Semantic or template-level leakage is not excluded.**
74
+ - Scores from synthetic examples must **never** be used for grading, employment, admissions or certification of educational quality.
75
+
76
+ ## Repository structure
77
+
78
+ - `index.html`, `styles.css`, `app.js`, `engine.js`: standalone Static Space app and model code.
79
+ - `train.json`, `iid_test.json`, `challenge.json`: original fixed splits used by the live app.
80
+ - `academic_rhetoric_intent.csv`: original combined synthetic dataset, with source metadata.
81
+ - `dataset_manifest.json`, `DATA_CARD.md`, `MODEL_CARD.md`: provenance and technical cards.
82
+ - `evaluation_results.json`: independently recomputed baseline metrics.
83
+ - `test_engine.js`, `benchmark_cli.js`, `test_browser.py`: reproducibility and browser QA.
84
+ - `DEPLOYMENT_ES.md`: easy Spanish uploading guide.
85
+ - Other MD files: limitations, methods, source license and release checks.
86
+
87
+ ## Privacy, cost and limitations
88
+
89
+ The live app sends **no user sentence** to any server. Model training and scoring operate locally in the browser. Short experiment-summary history uses browser local storage; exports are local downloads. The bundled texts are synthetic and public. Zero paid model inference or managed compute is required.
90
+
91
+ See `SECURITY.md` and `LIMITATIONS.md` for caveats. Please treat the application as a **research demonstration**, not as a grading or decision-making system.
REPRODUCE.md ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Reproducing the experiments
2
+
3
+ ## Browser
4
+
5
+ Upload the repository files to one root folder in a Hugging Face Static Space. Open App. Models fit automatically (locally) to `train.json`. Open Overview, or Experiments → Run four baselines. Select challenge (54) or IID (130); enter a bootstrap seed. Download metrics JSON and prediction CSV.
6
+
7
+ ## Node, no dependencies
8
+
9
+ ```sh
10
+ node --test test_engine.js
11
+ node benchmark_cli.js
12
+ ```
13
+
14
+ Both commands use only Node 18+ built-ins and the bundled dataset files. No Hugging Face token, Python packages or GPU needed.
15
+
16
+ ## Browser test, optional
17
+
18
+ ```sh
19
+ python -m pip install playwright
20
+ python -m playwright install chromium
21
+ python test_browser.py
22
+ ```
23
+
24
+ The test injects local project files directly into the Chromium page and mocks `fetch` *only for fixture-loading* to avoid external network dependency in restricted CI. This is not a test of deployed Hugging Face hosting.
25
+
26
+ ## Interpreting results
27
+
28
+ Use the fixed split names. Do not compare any archived Python-model evaluations as if they used the same code. Synthetic evaluations are not proof of generalization to real scholarly work.
SECURITY.md ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ # Security and privacy
2
+
3
+ - All classification, training, metrics and exports run in your browser as JavaScript.
4
+ - The application bundles synthetic research examples only, not private files.
5
+ - User-entered examples are processed locally and are not intentionally transmitted over a network.
6
+ - Experiment history contains only date, dataset split, seed, model summary and numerical aggregate metrics. It is stored in the browser's `localStorage` (up to 15 entries), and can be cleared from the app.
7
+ - No credentials, cookies, telemetry, tokens, external scripts or API endpoints are required by the app.
8
+ - Inspect scripts before customizing or rehosting. Do not publish real private research texts as static assets.
9
+ - CSV exports are intended as diagnostic artifacts; avoid importing untrusted formula-bearing strings into spreadsheets without checking them.
academic_rhetoric_intent.csv ADDED
The diff for this file is too large to render. See raw diff
 
app.js ADDED
@@ -0,0 +1,53 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /* ModelOps Research Lab interface | MIT code; dataset attribution: CC BY 4.0 */
2
+ (() => {
3
+ 'use strict';
4
+ const E=globalThis.ModelOpsEngine;
5
+ const METHODS=[['nb','Multinomial Naive Bayes'],['word','Word TF-IDF centroid'],['char','Character TF-IDF centroid'],['hybrid','Hybrid word + character']];
6
+ const FILES=['README.md','index.html','styles.css','engine.js','app.js','train.json','iid_test.json','challenge.json','academic_rhetoric_intent.csv','challenge_predictions.csv','dataset_manifest.json','evaluation_results.json','LICENSE','DATA_LICENSE.md','DATA_CARD.md','ORIGINAL_DATASET_CARD.md','MODEL_CARD.md','METHODS.md','LIMITATIONS.md','SECURITY.md','REPRODUCE.md','DEPLOYMENT_ES.md','CHANGELOG.md','QA_REPORT.md','test_engine.js','benchmark_cli.js','test_browser.py','preflight.py'];
7
+ const state={train:[],iid:[],challenge:[],models:null,preds:{},metrics:{},history:[],experiment:null,ready:false};
8
+ const $=id=>document.getElementById(id);
9
+ const fmt=n=>new Intl.NumberFormat('en-US',{maximumFractionDigits:1}).format(n);
10
+ const pct=n=>`${(n*100).toFixed(1)}%`;
11
+ const esc=s=>String(s??'').replace(/[&<>"']/g,c=>({'&':'&amp;','<':'&lt;','>':'&gt;','"':'&quot;',"'":'&#39;'}[c]));
12
+ const metricName={accuracy:'Accuracy',macroF1:'Macro F1'};
13
+ const getPred=(split,method)=>state.preds[split]?.[method]||[];
14
+ const getMetric=(split,method)=>state.metrics[split]?.[method];
15
+ const status=(message,error=false)=>{$('status').innerHTML=(error?'⚠ ':'<span class="pulse"></span> ')+esc(message);$('status').className=error?'bad':'';};
16
+ function loadHistory(){try{const raw=JSON.parse(localStorage.getItem('modelops-v2-history')||'[]');state.history=Array.isArray(raw)?raw.slice(0,15):[];}catch{state.history=[];}}
17
+ function saveHistory(){try{localStorage.setItem('modelops-v2-history',JSON.stringify(state.history.slice(0,15)));}catch{status('Local storage unavailable. Runs can still be downloaded.',true);}}
18
+ function populateModels(){for(const id of ['confModel','sliceModel','errorModel','sampleModel'])$(id).innerHTML=METHODS.map(([k,name])=>`<option value="${k}">${esc(name)}</option>`).join('');$('confModel').value='hybrid';$('sliceModel').value='hybrid';$('errorModel').value='hybrid';$('sampleModel').value='hybrid';}
19
+ function setTab(name){document.querySelectorAll('.tab').forEach(b=>{const active=b.dataset.tab===name;b.classList.toggle('active',active);if(active)b.setAttribute('aria-current','page');else b.removeAttribute('aria-current');});document.querySelectorAll('.view').forEach(v=>v.classList.toggle('active',v.dataset.view===name));if(name==='errors')renderErrors();if(name==='audit')renderAudit();if(name==='exports')renderExports();}
20
+ function table(columns,rows){return `<table class="table"><thead><tr>${columns.map(c=>`<th${c.right?' class="number"':''}>${esc(c.title)}</th>`).join('')}</tr></thead><tbody>${rows.map(r=>`<tr>${r.map((v,i)=>`<td${columns[i]?.right?' class="number"':''}>${v}</td>`).join('')}</tr>`).join('')}</tbody></table>`;}
21
+ function bar(name,value,alt=false){const n=Math.max(0,Math.min(1,value));return `<div class="bar-row"><div class="bar-name">${esc(name)}</div><div class="bar-track"><div class="bar-fill ${alt?'alt':''}" style="width:${(n*100).toFixed(2)}%"></div></div><div class="bar-value">${pct(value)}</div></div>`;}
22
+ function renderOverview(){if(!state.ready)return;const split=$('overviewSplit').value,metric=$('metricSelect').value;const entries=METHODS.map(([id,name])=>({id,name,stats:getMetric(split,id)})).sort((a,b)=>b.stats.accuracy-a.stats.accuracy);const best=entries[0], other=getMetric(split==='iid'?'challenge':'iid',best.id);const total=split==='iid'?130:54;
23
+ $('kpis').innerHTML=[['Evaluation rows',String(total),'Fixed holdout'],['Baselines','4','Trained in browser'],['Best Accuracy',pct(best.stats.accuracy),best.name],['Best Macro F1',pct(Math.max(...entries.map(e=>e.stats.macroF1))),'Six rhetorical labels']].map(([a,b,c])=>`<div class="kpi"><small>${esc(a)}</small><strong>${esc(b)}</strong><em>${esc(c)}</em></div>`).join('');
24
+ const ranked=[...entries].sort((a,b)=>b.stats[metric]-a.stats[metric]);$('leaderboard').innerHTML=ranked.map((e,i)=>bar(e.name,e.stats[metric],i%2===1)).join('');
25
+ const gap=getMetric('iid',best.id).accuracy-getMetric('challenge',best.id).accuracy;
26
+ $('generalization').innerHTML=`<div class="insight-number ${gap>=.15?'gap-warn':'good'}">${(gap*100).toFixed(1)} percentage points</div><p>IID-to-challenge Accuracy gap for <strong>${esc(best.name)}</strong>.</p>${bar('Controlled IID',getMetric('iid',best.id).accuracy)}${bar('Novel challenge',getMetric('challenge',best.id).accuracy,true)}<p class="fine">The IID set and novel challenge differ by construction. This difference cannot be interpreted as a population-level causal or fairness estimate.</p>`;
27
+ const model=$('confModel').value;const conf=getMetric(split,model).confusion;const max=Math.max(1,...E.LABELS.map(a=>Math.max(...E.LABELS.map(b=>conf[a][b]))));
28
+ $('confusion').innerHTML=`<table class="table matrix"><thead><tr><th>True ↓ / Pred →</th>${E.LABELS.map(x=>`<th>${esc(x.slice(0,5))}</th>`).join('')}</tr></thead><tbody>${E.LABELS.map(a=>`<tr><th>${esc(a)}</th>${E.LABELS.map(b=>`<td class="matrixcell" style="--strength:${(conf[a][b]/max).toFixed(3)}" title="True ${a}, predicted ${b}: ${conf[a][b]}">${conf[a][b]}</td>`).join('')}</tr>`).join('')}</tbody></table><p class="fine">Counts of the fixed ${esc(split)} evaluation split. Matrix rows are true classes, columns are model predictions.</p>`;
29
+ const parts=[['842 training examples',842,'train'],['130 IID test',130,'iid'],['54 novel-pattern',54,'challenge']];$('partitionBars').innerHTML=`<div class="split-bar">${parts.map(([label,n,k])=>`<div class="split-part split-${k}" style="width:${n/1026*100}%">${n>=100?n:''}</div>`).join('')}</div><div class="legend">${parts.map(([label,n,k])=>`<span><i style="background:${{train:'#68e0c6',iid:'#86bbff',challenge:'#b8a5ff'}[k]}"></i>${esc(label)}</span>`).join('')}</div><div class="test-detail"><strong>Data provenance</strong><p>Original synthetic-curated dataset: Academic Rhetorical Intent Mini v1.1. Fixed partitions are bundled in the Space. Dataset licensed CC BY 4.0.</p></div>`;
30
+ }
31
+ function renderExperiments(){const e=state.experiment;if(e){$('experimentOutput').innerHTML=`<div class="cardhead"><div><div class="overline">LATEST REPRODUCIBLE RUN</div><h3>${esc(e.cohort)} · seed ${e.seed} · ${e.reps} resamples</h3></div><span class="pill">Completed</span></div><div class="h-scroll">${table([{title:'Baseline'},{title:'Accuracy',right:true},{title:'Macro F1',right:true},{title:'Sample 95% interval',right:true}],[...e.methods].sort((a,b)=>b.accuracy-a.accuracy).map(m=>[esc(m.name),pct(m.accuracy),pct(m.macroF1),pct(m.ci.low)+' to '+pct(m.ci.high)]))}</div><p class="fine">This confidence interval is a simple nonparametric bootstrap across labeled evaluation rows and does not account for training-data variability, domain selection or label noise.</p>`;}
32
+ $('historyList').innerHTML=state.history.length?table([{title:'Saved run'},{title:'Dataset'},{title:'Best method'},{title:'Best accuracy',right:true}],state.history.map(r=>[esc(r.timestamp.replace('T',' ').slice(0,16)),esc(r.cohort),esc(r.bestName),pct(r.bestAccuracy)])):'<p class="muted">No experiments saved yet. Run the four baselines above to add one.</p>';
33
+ $('lastRun').textContent=state.history.length?'Last saved: '+state.history[0].timestamp.replace('T',' ').slice(0,16):'No saved experiment yet';
34
+ }
35
+ function runExperiment(){if(!state.ready)return;const split=$('runSplit').value,seed=Number($('runSeed').value),reps=Number($('runReps').value);if(!Number.isSafeInteger(seed)||seed<1||seed>999999999){status('Seed must be a whole number from 1 to 999999999.',true);return;}const btn=$('runButton');btn.disabled=true;btn.textContent='Running…';try{const methods=METHODS.map(([id,name])=>{const stats=getMetric(split,id),ci=E.bootstrapAccuracy(getPred(split,id),seed,reps);return {id,name,accuracy:stats.accuracy,macroF1:stats.macroF1,ci};});const best=[...methods].sort((a,b)=>b.accuracy-a.accuracy)[0];const run={timestamp:new Date().toISOString(),version:E.version(),dataVersion:'academic-rhetorical-intent-mini-v1.1',cohort:split,seed,reps,methods,bestName:best.name,bestAccuracy:best.accuracy};state.experiment=run;state.history.unshift({timestamp:run.timestamp,cohort:split,bestName:best.name,bestAccuracy:best.accuracy,seed,reps});state.history=state.history.slice(0,15);saveHistory();renderExperiments();status('Experiment complete. Four reproducible baselines evaluated.');}catch(e){status('Experiment failed: '+e.message,true);}finally{btn.disabled=false;btn.textContent='Run four baselines';}}
36
+ function renderSlices(){if(!state.ready)return;const method=$('sliceModel').value,split=$('sliceSplit').value,preds=getPred(split,method);const langs=E.summarizeSlices(preds,'language'),domains=E.summarizeSlices(preds,'domain'),base=getMetric(split,method);
37
+ const sliceTable=items=>table([{title:'Slice'},{title:'n',right:true},{title:'Accuracy',right:true},{title:'Macro F1',right:true}],items.map(g=>[esc(g.name),g.n,pct(g.accuracy),pct(g.macroF1)]));$('languageSlices').innerHTML=sliceTable(langs);$('domainSlices').innerHTML=sliceTable(domains);$('labelSlices').innerHTML=table([{title:'Rhetorical function'},{title:'Support',right:true},{title:'Precision',right:true},{title:'Recall',right:true},{title:'F1',right:true}],E.LABELS.map(l=>[esc(l),base.byLabel[l].support,pct(base.byLabel[l].precision),pct(base.byLabel[l].recall),pct(base.byLabel[l].f1)]));}
38
+ function renderErrors(){if(!state.ready)return;const method=$('errorModel').value,split=$('errorSplit').value,q=String($('errorSearch').value||'').toLowerCase();const preds=getPred(split,method);const fails=preds.filter(r=>r.pred!==r.truth);const filtered=fails.filter(r=>(r.text+' '+r.truth+' '+r.pred+' '+r.language).toLowerCase().includes(q));$('errorsList').innerHTML=`<p class="fine"><strong>${fails.length} errors in ${preds.length} examples.</strong> Showing ${Math.min(filtered.length,20)} of ${filtered.length} matching errors. Full predictions are available under Export & governance.</p>`+(filtered.length?filtered.slice(0,20).map(r=>`<details class="test-detail"><summary>${esc(r.id)} <span class="badge badge-error">${esc(r.truth)} → ${esc(r.pred)}</span> <span class="badge">${esc(r.language)}</span></summary><p>${esc(r.text)}</p><small>Domain: ${esc(r.domain)} · Original annotation: ${esc(r.truth)} · Predicted: ${esc(r.pred)}</small></details>`).join(''):'<p class="muted">No errors match this search.</p>');}
39
+ function classify(){if(!state.ready)return;const text=$('sampleText').value.trim();if(!text){$('sampleOutput').innerHTML='<p class="bad">Enter a sentence to classify.</p>';return;}const method=$('sampleModel').value,p=E.predict(state.models,text,method);const ranked=E.LABELS.map((label,i)=>({label,score:p.scores[i]})).sort((a,b)=>b.score-a.score);const raw=ranked[0].score;$('sampleOutput').innerHTML=`<div class="report"><div class="overline">PREDICTED RHETORICAL FUNCTION</div><h3>${esc(p.label)}</h3><p>Model: ${esc(METHODS.find(m=>m[0]===method)?.[1])}</p><p class="fine">Uncalibrated score margin: ${p.margin.toFixed(3)}. Higher margins are not guaranteed to mean higher correctness.</p>${ranked.map((m,i)=>bar(m.label,method==='nb'?m.score:Math.max(0,m.score),i%2===1)).join('')}</div>`;}
40
+ function renderShift(){if(!state.ready)return;const rows=METHODS.map(([id,name])=>{const a=getMetric('iid',id),b=getMetric('challenge',id);return [esc(name),pct(a.accuracy),pct(b.accuracy),`<strong class="${a.accuracy-b.accuracy>.2?'gap-warn':''}">${((a.accuracy-b.accuracy)*100).toFixed(1)} pp</strong>`,pct(b.macroF1)];});$('shiftGrid').innerHTML=table([{title:'Baseline'},{title:'Controlled IID',right:true},{title:'Novel-pattern',right:true},{title:'Accuracy gap',right:true},{title:'Challenge Macro F1',right:true}],rows)+'<p class="fine">The 54-row challenge set is fixed and was not used in training or hyperparameter optimization. The four baselines share the same training data and held-out examples.</p>';}
41
+ function countBy(rows,k){const o={};for(const r of rows)o[String(r[k]||'unknown')]=(o[String(r[k]||'unknown')]||0)+1;return o;}
42
+ function renderAudit(){if(!state.ready)return;const a1=E.audit(state.train,state.iid),a2=E.audit(state.train,state.challenge);const labels=countBy([...state.train,...state.iid,...state.challenge],'label');const langs=countBy([...state.train,...state.iid,...state.challenge],'language');const domain=countBy([...state.train,...state.iid,...state.challenge],'domain');$('auditSummary').innerHTML=[['Training',842,'fixed source split'],['IID',130,'controlled holdout'],['Challenge',54,'unseen patterns'],['Label classes',6,'balanced overall']].map(([name,n,foot])=>`<div class="kpi"><small>${name}</small><strong>${n}</strong><em>${foot}</em></div>`).join('');$('datasetComposition').innerHTML=`<h4>Languages</h4>${Object.entries(langs).map(([k,v])=>bar(k,v/1026)).join('')}<h4>Classes</h4>${Object.entries(labels).map(([k,v])=>bar(k,v/1026)).join('')}<h4>Domains</h4>${table([{title:'Domain'},{title:'Rows',right:true}],Object.entries(domain).sort((a,b)=>b[1]-a[1]).map(([k,v])=>[esc(k),v]))}</h4>`;
43
+ $('leakageChecks').innerHTML=[['IID exact-text overlap',a1.exactTextOverlap],['Challenge exact-text overlap',a2.exactTextOverlap],['IID duplicate IDs',a1.duplicateIds],['Challenge duplicate IDs',a2.duplicateIds],['Invalid IID labels/text',a1.invalidRows],['Invalid challenge labels/text',a2.invalidRows]].map(([name,val])=>`<div class="test-detail">${val===0?'<span class="badge badge-success">PASS</span>':'<span class="badge badge-error">REVIEW</span>'} <strong>${esc(name)}</strong> <span style="float:right">${val}</span></div>`).join('')+'<p class="fine">This is an exact-text / ID audit only. It does not detect paraphrase-template overlap, semantic leakage or mislabeling.</p>';}
44
+ function renderExports(){$('fileInventory').innerHTML=FILES.map(n=>`<span>▤ ${esc(n)}</span>`).join('');}
45
+ function download(content,name,mime){const blob=new Blob([content],{type:mime});const url=URL.createObjectURL(blob);const link=document.createElement('a');link.href=url;link.download=name;link.style.display='none';document.body.appendChild(link);link.click();link.remove();setTimeout(()=>URL.revokeObjectURL(url),1200);status('Downloaded '+name+'.');}
46
+ function exportMetrics(){const payload={project:'ModelOps Research Lab',version:E.version(),dataset:'Academic Rhetorical Intent Mini v1.1',source_type:'synthetic_curated',train_rows:842,test_rows:130,challenge_rows:54,generated_utc:new Date().toISOString(),methods:Object.fromEntries(METHODS.map(([id,name])=>[id,{name,iid:{accuracy:getMetric('iid',id).accuracy,macroF1:getMetric('iid',id).macroF1},challenge:{accuracy:getMetric('challenge',id).accuracy,macroF1:getMetric('challenge',id).macroF1}}])),method_note:'Evaluation metrics on fixed original synthetic-curated partitions. Not external performance evidence.',latest_experiment:state.experiment};download(JSON.stringify(payload,null,2),'modelops_metrics.json','application/json');}
47
+ function exportPredictions(){const split=$('overviewSplit').value;const rows=[];for(const [id,name] of METHODS){for(const p of getPred(split,id))rows.push({method:name,split,id:p.id,language:p.language,domain:p.domain,true_label:p.truth,predicted_label:p.pred,correct:p.truth===p.pred,text:p.text});}download(E.toCSV(rows,['method','split','id','language','domain','true_label','predicted_label','correct','text']),'modelops_predictions_'+split+'.csv','text/csv;charset=utf-8');}
48
+ function exportReport(){const now=new Date().toISOString();let s=`# ModelOps Research Lab: Evaluation report\n\nGenerated UTC: ${now}\n\n## Research artifact\n- Dataset: Academic Rhetorical Intent Mini v1.1 (synthetic-curated), CC BY 4.0.\n- Code license: MIT.\n- Train: 842, controlled IID: 130, fixed challenge: 54.\n- Models are trained and evaluated locally in a browser.\n\n## Baseline comparisons\n\n| Model | IID accuracy | IID Macro F1 | Challenge accuracy | Challenge Macro F1 |\n|---|---:|---:|---:|---:|\n`;for(const [id,name] of METHODS)s+=`| ${name} | ${pct(getMetric('iid',id).accuracy)} | ${pct(getMetric('iid',id).macroF1)} | ${pct(getMetric('challenge',id).accuracy)} | ${pct(getMetric('challenge',id).macroF1)} |\n`;s+='\n## Methodology\nFour deterministic lexical baselines: word Multinomial Naive Bayes (Laplace smoothing), word TF-IDF class centroids, character trigram TF-IDF centroids, and a 55/45 weighted word/character hybrid. The split is fixed. Accuracy and macro F1 are computed against held-out labels. Any bootstrap interval describes synthetic sample variation only.\n\n## Limitations\nThis dataset consists of synthetic-curated academic-style sentences, not representative scholarly submissions. High IID metrics can reflect recurring templates. The fixed challenge is harder but not independent real-world validation. Training or hyperparameter selection was not performed on the challenge set. Do not use the application for grading, admission, employment or other consequential individual assessments.\n\n## External model evidence\nA distinct archived Python model from the Academic Rhetoric Classifier project reported 35.7% accuracy on a 28-sentence biomedical development smoke test. This is neither a production benchmark nor a direct comparison to the four browser baselines.\n';if(state.experiment)s+=`\n## Latest recorded experiment\nSplit: ${state.experiment.cohort}\nSeed: ${state.experiment.seed}\nBootstrap repetitions: ${state.experiment.reps}\n`;download(s,'modelops_research_report.md','text/markdown;charset=utf-8');}
49
+ async function loadData(){if(!E)throw Error('Evaluation engine not loaded');const files=['train.json','iid_test.json','challenge.json'];const response=await Promise.all(files.map(async file=>{const r=await fetch('./'+file,{cache:'no-store'});if(!r.ok)throw Error(file+' returned '+r.status);return r.json();}));[state.train,state.iid,state.challenge]=response;if(state.train.length!==842||state.iid.length!==130||state.challenge.length!==54)throw Error('Dataset partition size mismatch');for(const rows of response)for(const row of rows)if(!E.LABELS.includes(row.label)||!row.id||!row.text)throw Error('Invalid source row');state.models=E.train(state.train);for(const split of ['iid','challenge']){state.preds[split]={};state.metrics[split]={};for(const [id]of METHODS){state.preds[split][id]=E.evaluate(state.models,state[split],id);state.metrics[split][id]=E.evaluation(state.preds[split][id]);}}state.ready=true;}
50
+ function wire(){document.querySelectorAll('.tab').forEach(btn=>btn.addEventListener('click',()=>setTab(btn.dataset.tab)));for(const id of ['overviewSplit','metricSelect','confModel'])$(id).addEventListener('change',renderOverview);for(const id of ['sliceModel','sliceSplit'])$(id).addEventListener('change',renderSlices);for(const id of ['errorModel','errorSplit','errorSearch'])$(id).addEventListener(id==='errorSearch'?'input':'change',renderErrors);$('runButton').addEventListener('click',runExperiment);$('classifyButton').addEventListener('click',classify);$('downloadPred').addEventListener('click',exportPredictions);$('downloadMetrics').addEventListener('click',exportMetrics);$('downloadReport').addEventListener('click',exportReport);$('downloadRuns').addEventListener('click',()=>download(JSON.stringify({project:'ModelOps Research Lab',version:E.version(),runs:state.history},null,2),'modelops_run_history.json','application/json'));$('clearHistory').addEventListener('click',()=>{if(!confirm('Clear locally saved experiment summaries? This does not change project source files.'))return;state.history=[];saveHistory();renderExperiments();status('Local experiment history cleared.');});}
51
+ async function boot(){loadHistory();populateModels();wire();try{await loadData();renderOverview();renderSlices();renderErrors();renderShift();renderAudit();renderExperiments();renderExports();status('Ready. Four models trained locally on 842 examples; both holdouts evaluated.');}catch(e){status('Could not initialize: '+e.message+'. Check that all release files were uploaded together.',true);console.error(e);}}
52
+ if(document.readyState==='loading')document.addEventListener('DOMContentLoaded',boot);else boot();
53
+ })();
benchmark_cli.js ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ // Recompute the entire benchmark without a browser: node benchmark_cli.js
2
+ const fs=require('node:fs'),path=require('node:path'),E=require('./engine.js');
3
+ const read=n=>JSON.parse(fs.readFileSync(path.join(__dirname,n),'utf8'));
4
+ const training=read('train.json');const bundle=E.train(training);
5
+ const splits=[['Controlled IID','iid_test.json'],['Novel-pattern challenge','challenge.json']];
6
+ console.log('ModelOps Research Lab v'+E.version()+' | 842 synthetic-curated training rows');
7
+ for(const [split,file] of splits){console.log('\n'+split);const rows=read(file);const audit=E.audit(training,rows);console.log('Integrity:',JSON.stringify(audit));if(!audit.clean)throw Error('Split integrity failure');for(const method of ['nb','word','char','hybrid']){const p=E.evaluate(bundle,rows,method);const m=E.evaluation(p);console.log(method.padEnd(10), 'n='+m.n, 'Accuracy='+Math.round(m.accuracy*1000)/10+'%', 'MacroF1='+Math.round(m.macroF1*1000)/10+'%');}}
challenge.json ADDED
@@ -0,0 +1 @@
 
 
1
+ [{"id":"ari-0649","language":"en","label":"background","domain":"mixed","text":"Scholars continue to debate how digital tools reshape knowledge production.","template_group":"novel"},{"id":"ari-0650","language":"en","label":"background","domain":"mixed","text":"Regional economic integration has received renewed attention after recent supply disruptions.","template_group":"novel"},{"id":"ari-0651","language":"en","label":"background","domain":"mixed","text":"The rapid adoption of automated systems has intensified discussion about professional work.","template_group":"novel"},{"id":"ari-0652","language":"en","label":"objective","domain":"mixed","text":"Here we ask whether multilingual support changes the quality of academic revision.","template_group":"novel"},{"id":"ari-0653","language":"en","label":"objective","domain":"mixed","text":"Our purpose is to determine which institutional factors predict cross-border collaboration.","template_group":"novel"},{"id":"ari-0654","language":"en","label":"objective","domain":"mixed","text":"We set out to compare alternative explanations for differences in digital adoption.","template_group":"novel"},{"id":"ari-0655","language":"en","label":"method","domain":"mixed","text":"Participants completed two survey waves, after which responses were modeled with clustered standard errors.","template_group":"novel"},{"id":"ari-0656","language":"en","label":"method","domain":"mixed","text":"Interview transcripts were independently coded by two researchers before themes were reconciled.","template_group":"novel"},{"id":"ari-0657","language":"en","label":"method","domain":"mixed","text":"We assembled annual country indicators and estimated a difference-in-differences specification.","template_group":"novel"},{"id":"ari-0658","language":"en","label":"result","domain":"mixed","text":"Scores increased after the intervention, with the largest gains observed in the treatment group.","template_group":"novel"},{"id":"ari-0659","language":"en","label":"result","domain":"mixed","text":"Three recurring themes emerged from the interviews and were present across all study sites.","template_group":"novel"},{"id":"ari-0660","language":"en","label":"result","domain":"mixed","text":"The coefficient remained positive and statistically distinguishable from zero in every robustness check.","template_group":"novel"},{"id":"ari-0661","language":"en","label":"limitation","domain":"mixed","text":"Our conclusions are constrained by reliance on self-reported behavior.","template_group":"novel"},{"id":"ari-0662","language":"en","label":"limitation","domain":"mixed","text":"The short observation window prevents assessment of whether the effect persists.","template_group":"novel"},{"id":"ari-0663","language":"en","label":"limitation","domain":"mixed","text":"Because the cases were purposively selected, external validity remains uncertain.","template_group":"novel"},{"id":"ari-0664","language":"en","label":"implication","domain":"mixed","text":"Universities may therefore benefit from providing structured guidance on responsible tool use.","template_group":"novel"},{"id":"ari-0665","language":"en","label":"implication","domain":"mixed","text":"The pattern indicates that regional programs should account for institutional capacity differences.","template_group":"novel"},{"id":"ari-0666","language":"en","label":"implication","domain":"mixed","text":"Subsequent studies could test the mechanism with longitudinal data and broader samples.","template_group":"novel"},{"id":"ari-0667","language":"es","label":"background","domain":"mixed","text":"La literatura reciente debate cómo las herramientas digitales transforman la producción de conocimiento.","template_group":"novel"},{"id":"ari-0668","language":"es","label":"background","domain":"mixed","text":"La integración económica regional volvió a ganar atención tras las disrupciones en las cadenas de suministro.","template_group":"novel"},{"id":"ari-0669","language":"es","label":"background","domain":"mixed","text":"La rápida adopción de sistemas automatizados intensificó el debate sobre el trabajo profesional.","template_group":"novel"},{"id":"ari-0670","language":"es","label":"objective","domain":"mixed","text":"Aquí preguntamos si el apoyo multilingüe modifica la calidad de la revisión académica.","template_group":"novel"},{"id":"ari-0671","language":"es","label":"objective","domain":"mixed","text":"Nuestro propósito es determinar qué factores institucionales predicen la colaboración transfronteriza.","template_group":"novel"},{"id":"ari-0672","language":"es","label":"objective","domain":"mixed","text":"Nos proponemos comparar explicaciones alternativas de las diferencias en adopción digital.","template_group":"novel"},{"id":"ari-0673","language":"es","label":"method","domain":"mixed","text":"Los participantes completaron dos rondas de encuesta y las respuestas se modelaron con errores estándar agrupados.","template_group":"novel"},{"id":"ari-0674","language":"es","label":"method","domain":"mixed","text":"Dos investigadores codificaron de manera independiente las entrevistas antes de consensuar los temas.","template_group":"novel"},{"id":"ari-0675","language":"es","label":"method","domain":"mixed","text":"Construimos indicadores anuales por país y estimamos un modelo de diferencias en diferencias.","template_group":"novel"},{"id":"ari-0676","language":"es","label":"result","domain":"mixed","text":"Las puntuaciones aumentaron después de la intervención y las mayores mejoras aparecieron en el grupo tratado.","template_group":"novel"},{"id":"ari-0677","language":"es","label":"result","domain":"mixed","text":"De las entrevistas surgieron tres temas recurrentes presentes en todos los sitios estudiados.","template_group":"novel"},{"id":"ari-0678","language":"es","label":"result","domain":"mixed","text":"El coeficiente se mantuvo positivo y estadísticamente distinto de cero en todas las pruebas de robustez.","template_group":"novel"},{"id":"ari-0679","language":"es","label":"limitation","domain":"mixed","text":"Nuestras conclusiones están condicionadas por el uso de conductas autodeclaradas.","template_group":"novel"},{"id":"ari-0680","language":"es","label":"limitation","domain":"mixed","text":"El breve período de observación impide saber si el efecto persiste.","template_group":"novel"},{"id":"ari-0681","language":"es","label":"limitation","domain":"mixed","text":"Como los casos fueron seleccionados intencionalmente, la validez externa sigue siendo incierta.","template_group":"novel"},{"id":"ari-0682","language":"es","label":"implication","domain":"mixed","text":"Las universidades podrían beneficiarse de ofrecer orientaciones estructuradas sobre el uso responsable de estas herramientas.","template_group":"novel"},{"id":"ari-0683","language":"es","label":"implication","domain":"mixed","text":"El patrón indica que los programas regionales deberían considerar las diferencias de capacidad institucional.","template_group":"novel"},{"id":"ari-0684","language":"es","label":"implication","domain":"mixed","text":"Estudios posteriores podrían probar el mecanismo con datos longitudinales y muestras más amplias.","template_group":"novel"},{"id":"ari-0685","language":"pt","label":"background","domain":"mixed","text":"A literatura recente debate como ferramentas digitais transformam a produção de conhecimento.","template_group":"novel"},{"id":"ari-0686","language":"pt","label":"background","domain":"mixed","text":"A integração econômica regional voltou a receber atenção após interrupções nas cadeias de suprimento.","template_group":"novel"},{"id":"ari-0687","language":"pt","label":"background","domain":"mixed","text":"A rápida adoção de sistemas automatizados intensificou o debate sobre o trabalho profissional.","template_group":"novel"},{"id":"ari-0688","language":"pt","label":"objective","domain":"mixed","text":"Perguntamos aqui se o apoio multilíngue altera a qualidade da revisão acadêmica.","template_group":"novel"},{"id":"ari-0689","language":"pt","label":"objective","domain":"mixed","text":"Nosso propósito é determinar quais fatores institucionais predizem a colaboração transfronteiriça.","template_group":"novel"},{"id":"ari-0690","language":"pt","label":"objective","domain":"mixed","text":"Pretendemos comparar explicações alternativas para diferenças na adoção digital.","template_group":"novel"},{"id":"ari-0691","language":"pt","label":"method","domain":"mixed","text":"Os participantes responderam a duas ondas de pesquisa e as respostas foram modeladas com erros-padrão agrupados.","template_group":"novel"},{"id":"ari-0692","language":"pt","label":"method","domain":"mixed","text":"Dois pesquisadores codificaram independentemente as entrevistas antes de reconciliar os temas.","template_group":"novel"},{"id":"ari-0693","language":"pt","label":"method","domain":"mixed","text":"Reunimos indicadores anuais por país e estimamos uma especificação de diferenças em diferenças.","template_group":"novel"},{"id":"ari-0694","language":"pt","label":"result","domain":"mixed","text":"As pontuações aumentaram após a intervenção, com os maiores ganhos no grupo de tratamento.","template_group":"novel"},{"id":"ari-0695","language":"pt","label":"result","domain":"mixed","text":"Três temas recorrentes emergiram das entrevistas e apareceram em todos os locais estudados.","template_group":"novel"},{"id":"ari-0696","language":"pt","label":"result","domain":"mixed","text":"O coeficiente permaneceu positivo e estatisticamente diferente de zero em todos os testes de robustez.","template_group":"novel"},{"id":"ari-0697","language":"pt","label":"limitation","domain":"mixed","text":"Nossas conclusões são limitadas pela dependência de comportamento autorrelatado.","template_group":"novel"},{"id":"ari-0698","language":"pt","label":"limitation","domain":"mixed","text":"O curto período de observação impede avaliar se o efeito persiste.","template_group":"novel"},{"id":"ari-0699","language":"pt","label":"limitation","domain":"mixed","text":"Como os casos foram selecionados intencionalmente, a validade externa permanece incerta.","template_group":"novel"},{"id":"ari-0700","language":"pt","label":"implication","domain":"mixed","text":"As universidades podem se beneficiar de orientações estruturadas sobre o uso responsável dessas ferramentas.","template_group":"novel"},{"id":"ari-0701","language":"pt","label":"implication","domain":"mixed","text":"O padrão indica que programas regionais devem considerar diferenças de capacidade institucional.","template_group":"novel"},{"id":"ari-0702","language":"pt","label":"implication","domain":"mixed","text":"Estudos futuros podem testar o mecanismo com dados longitudinais e amostras mais amplas.","template_group":"novel"}]
challenge_predictions.csv ADDED
@@ -0,0 +1,217 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ method,id,language,domain,truth,pred,correct,text
2
+ nb,ari-0649,en,mixed,background,objective,false,Scholars continue to debate how digital tools reshape knowledge production.
3
+ nb,ari-0650,en,mixed,background,background,true,Regional economic integration has received renewed attention after recent supply disruptions.
4
+ nb,ari-0651,en,mixed,background,background,true,The rapid adoption of automated systems has intensified discussion about professional work.
5
+ nb,ari-0652,en,mixed,objective,objective,true,Here we ask whether multilingual support changes the quality of academic revision.
6
+ nb,ari-0653,en,mixed,objective,objective,true,Our purpose is to determine which institutional factors predict cross-border collaboration.
7
+ nb,ari-0654,en,mixed,objective,objective,true,We set out to compare alternative explanations for differences in digital adoption.
8
+ nb,ari-0655,en,mixed,method,method,true,"Participants completed two survey waves, after which responses were modeled with clustered standard errors."
9
+ nb,ari-0656,en,mixed,method,method,true,Interview transcripts were independently coded by two researchers before themes were reconciled.
10
+ nb,ari-0657,en,mixed,method,method,true,We assembled annual country indicators and estimated a difference-in-differences specification.
11
+ nb,ari-0658,en,mixed,result,result,true,"Scores increased after the intervention, with the largest gains observed in the treatment group."
12
+ nb,ari-0659,en,mixed,result,method,false,Three recurring themes emerged from the interviews and were present across all study sites.
13
+ nb,ari-0660,en,mixed,result,result,true,The coefficient remained positive and statistically distinguishable from zero in every robustness check.
14
+ nb,ari-0661,en,mixed,limitation,limitation,true,Our conclusions are constrained by reliance on self-reported behavior.
15
+ nb,ari-0662,en,mixed,limitation,limitation,true,The short observation window prevents assessment of whether the effect persists.
16
+ nb,ari-0663,en,mixed,limitation,limitation,true,"Because the cases were purposively selected, external validity remains uncertain."
17
+ nb,ari-0664,en,mixed,implication,method,false,Universities may therefore benefit from providing structured guidance on responsible tool use.
18
+ nb,ari-0665,en,mixed,implication,implication,true,The pattern indicates that regional programs should account for institutional capacity differences.
19
+ nb,ari-0666,en,mixed,implication,implication,true,Subsequent studies could test the mechanism with longitudinal data and broader samples.
20
+ nb,ari-0667,es,mixed,background,background,true,La literatura reciente debate cómo las herramientas digitales transforman la producción de conocimiento.
21
+ nb,ari-0668,es,mixed,background,implication,false,La integración económica regional volvió a ganar atención tras las disrupciones en las cadenas de suministro.
22
+ nb,ari-0669,es,mixed,background,limitation,false,La rápida adopción de sistemas automatizados intensificó el debate sobre el trabajo profesional.
23
+ nb,ari-0670,es,mixed,objective,limitation,false,Aquí preguntamos si el apoyo multilingüe modifica la calidad de la revisión académica.
24
+ nb,ari-0671,es,mixed,objective,objective,true,Nuestro propósito es determinar qué factores institucionales predicen la colaboración transfronteriza.
25
+ nb,ari-0672,es,mixed,objective,result,false,Nos proponemos comparar explicaciones alternativas de las diferencias en adopción digital.
26
+ nb,ari-0673,es,mixed,method,limitation,false,Los participantes completaron dos rondas de encuesta y las respuestas se modelaron con errores estándar agrupados.
27
+ nb,ari-0674,es,mixed,method,method,true,Dos investigadores codificaron de manera independiente las entrevistas antes de consensuar los temas.
28
+ nb,ari-0675,es,mixed,method,method,true,Construimos indicadores anuales por país y estimamos un modelo de diferencias en diferencias.
29
+ nb,ari-0676,es,mixed,result,limitation,false,Las puntuaciones aumentaron después de la intervención y las mayores mejoras aparecieron en el grupo tratado.
30
+ nb,ari-0677,es,mixed,result,result,true,De las entrevistas surgieron tres temas recurrentes presentes en todos los sitios estudiados.
31
+ nb,ari-0678,es,mixed,result,result,true,El coeficiente se mantuvo positivo y estadísticamente distinto de cero en todas las pruebas de robustez.
32
+ nb,ari-0679,es,mixed,limitation,limitation,true,Nuestras conclusiones están condicionadas por el uso de conductas autodeclaradas.
33
+ nb,ari-0680,es,mixed,limitation,limitation,true,El breve período de observación impide saber si el efecto persiste.
34
+ nb,ari-0681,es,mixed,limitation,limitation,true,"Como los casos fueron seleccionados intencionalmente, la validez externa sigue siendo incierta."
35
+ nb,ari-0682,es,mixed,implication,limitation,false,Las universidades podrían beneficiarse de ofrecer orientaciones estructuradas sobre el uso responsable de estas herramientas.
36
+ nb,ari-0683,es,mixed,implication,implication,true,El patrón indica que los programas regionales deberían considerar las diferencias de capacidad institucional.
37
+ nb,ari-0684,es,mixed,implication,implication,true,Estudios posteriores podrían probar el mecanismo con datos longitudinales y muestras más amplias.
38
+ nb,ari-0685,pt,mixed,background,background,true,A literatura recente debate como ferramentas digitais transformam a produção de conhecimento.
39
+ nb,ari-0686,pt,mixed,background,implication,false,A integração econômica regional voltou a receber atenção após interrupções nas cadeias de suprimento.
40
+ nb,ari-0687,pt,mixed,background,limitation,false,A rápida adoção de sistemas automatizados intensificou o debate sobre o trabalho profissional.
41
+ nb,ari-0688,pt,mixed,objective,objective,true,Perguntamos aqui se o apoio multilíngue altera a qualidade da revisão acadêmica.
42
+ nb,ari-0689,pt,mixed,objective,objective,true,Nosso propósito é determinar quais fatores institucionais predizem a colaboração transfronteiriça.
43
+ nb,ari-0690,pt,mixed,objective,objective,true,Pretendemos comparar explicações alternativas para diferenças na adoção digital.
44
+ nb,ari-0691,pt,mixed,method,method,true,Os participantes responderam a duas ondas de pesquisa e as respostas foram modeladas com erros-padrão agrupados.
45
+ nb,ari-0692,pt,mixed,method,method,true,Dois pesquisadores codificaram independentemente as entrevistas antes de reconciliar os temas.
46
+ nb,ari-0693,pt,mixed,method,method,true,Reunimos indicadores anuais por país e estimamos uma especificação de diferenças em diferenças.
47
+ nb,ari-0694,pt,mixed,result,implication,false,"As pontuações aumentaram após a intervenção, com os maiores ganhos no grupo de tratamento."
48
+ nb,ari-0695,pt,mixed,result,result,true,Três temas recorrentes emergiram das entrevistas e apareceram em todos os locais estudados.
49
+ nb,ari-0696,pt,mixed,result,result,true,O coeficiente permaneceu positivo e estatisticamente diferente de zero em todos os testes de robustez.
50
+ nb,ari-0697,pt,mixed,limitation,limitation,true,Nossas conclusões são limitadas pela dependência de comportamento autorrelatado.
51
+ nb,ari-0698,pt,mixed,limitation,limitation,true,O curto período de observação impede avaliar se o efeito persiste.
52
+ nb,ari-0699,pt,mixed,limitation,limitation,true,"Como os casos foram selecionados intencionalmente, a validade externa permanece incerta."
53
+ nb,ari-0700,pt,mixed,implication,limitation,false,As universidades podem se beneficiar de orientações estruturadas sobre o uso responsável dessas ferramentas.
54
+ nb,ari-0701,pt,mixed,implication,implication,true,O padrão indica que programas regionais devem considerar diferenças de capacidade institucional.
55
+ nb,ari-0702,pt,mixed,implication,implication,true,Estudos futuros podem testar o mecanismo com dados longitudinais e amostras mais amplas.
56
+ word,ari-0649,en,mixed,background,objective,false,Scholars continue to debate how digital tools reshape knowledge production.
57
+ word,ari-0650,en,mixed,background,background,true,Regional economic integration has received renewed attention after recent supply disruptions.
58
+ word,ari-0651,en,mixed,background,limitation,false,The rapid adoption of automated systems has intensified discussion about professional work.
59
+ word,ari-0652,en,mixed,objective,objective,true,Here we ask whether multilingual support changes the quality of academic revision.
60
+ word,ari-0653,en,mixed,objective,objective,true,Our purpose is to determine which institutional factors predict cross-border collaboration.
61
+ word,ari-0654,en,mixed,objective,objective,true,We set out to compare alternative explanations for differences in digital adoption.
62
+ word,ari-0655,en,mixed,method,method,true,"Participants completed two survey waves, after which responses were modeled with clustered standard errors."
63
+ word,ari-0656,en,mixed,method,method,true,Interview transcripts were independently coded by two researchers before themes were reconciled.
64
+ word,ari-0657,en,mixed,method,method,true,We assembled annual country indicators and estimated a difference-in-differences specification.
65
+ word,ari-0658,en,mixed,result,limitation,false,"Scores increased after the intervention, with the largest gains observed in the treatment group."
66
+ word,ari-0659,en,mixed,result,method,false,Three recurring themes emerged from the interviews and were present across all study sites.
67
+ word,ari-0660,en,mixed,result,result,true,The coefficient remained positive and statistically distinguishable from zero in every robustness check.
68
+ word,ari-0661,en,mixed,limitation,limitation,true,Our conclusions are constrained by reliance on self-reported behavior.
69
+ word,ari-0662,en,mixed,limitation,limitation,true,The short observation window prevents assessment of whether the effect persists.
70
+ word,ari-0663,en,mixed,limitation,limitation,true,"Because the cases were purposively selected, external validity remains uncertain."
71
+ word,ari-0664,en,mixed,implication,method,false,Universities may therefore benefit from providing structured guidance on responsible tool use.
72
+ word,ari-0665,en,mixed,implication,implication,true,The pattern indicates that regional programs should account for institutional capacity differences.
73
+ word,ari-0666,en,mixed,implication,implication,true,Subsequent studies could test the mechanism with longitudinal data and broader samples.
74
+ word,ari-0667,es,mixed,background,limitation,false,La literatura reciente debate cómo las herramientas digitales transforman la producción de conocimiento.
75
+ word,ari-0668,es,mixed,background,limitation,false,La integración económica regional volvió a ganar atención tras las disrupciones en las cadenas de suministro.
76
+ word,ari-0669,es,mixed,background,limitation,false,La rápida adopción de sistemas automatizados intensificó el debate sobre el trabajo profesional.
77
+ word,ari-0670,es,mixed,objective,limitation,false,Aquí preguntamos si el apoyo multilingüe modifica la calidad de la revisión académica.
78
+ word,ari-0671,es,mixed,objective,objective,true,Nuestro propósito es determinar qué factores institucionales predicen la colaboración transfronteriza.
79
+ word,ari-0672,es,mixed,objective,result,false,Nos proponemos comparar explicaciones alternativas de las diferencias en adopción digital.
80
+ word,ari-0673,es,mixed,method,method,true,Los participantes completaron dos rondas de encuesta y las respuestas se modelaron con errores estándar agrupados.
81
+ word,ari-0674,es,mixed,method,method,true,Dos investigadores codificaron de manera independiente las entrevistas antes de consensuar los temas.
82
+ word,ari-0675,es,mixed,method,method,true,Construimos indicadores anuales por país y estimamos un modelo de diferencias en diferencias.
83
+ word,ari-0676,es,mixed,result,limitation,false,Las puntuaciones aumentaron después de la intervención y las mayores mejoras aparecieron en el grupo tratado.
84
+ word,ari-0677,es,mixed,result,result,true,De las entrevistas surgieron tres temas recurrentes presentes en todos los sitios estudiados.
85
+ word,ari-0678,es,mixed,result,result,true,El coeficiente se mantuvo positivo y estadísticamente distinto de cero en todas las pruebas de robustez.
86
+ word,ari-0679,es,mixed,limitation,limitation,true,Nuestras conclusiones están condicionadas por el uso de conductas autodeclaradas.
87
+ word,ari-0680,es,mixed,limitation,limitation,true,El breve período de observación impide saber si el efecto persiste.
88
+ word,ari-0681,es,mixed,limitation,limitation,true,"Como los casos fueron seleccionados intencionalmente, la validez externa sigue siendo incierta."
89
+ word,ari-0682,es,mixed,implication,limitation,false,Las universidades podrían beneficiarse de ofrecer orientaciones estructuradas sobre el uso responsable de estas herramientas.
90
+ word,ari-0683,es,mixed,implication,implication,true,El patrón indica que los programas regionales deberían considerar las diferencias de capacidad institucional.
91
+ word,ari-0684,es,mixed,implication,implication,true,Estudios posteriores podrían probar el mecanismo con datos longitudinales y muestras más amplias.
92
+ word,ari-0685,pt,mixed,background,background,true,A literatura recente debate como ferramentas digitais transformam a produção de conhecimento.
93
+ word,ari-0686,pt,mixed,background,method,false,A integração econômica regional voltou a receber atenção após interrupções nas cadeias de suprimento.
94
+ word,ari-0687,pt,mixed,background,limitation,false,A rápida adoção de sistemas automatizados intensificou o debate sobre o trabalho profissional.
95
+ word,ari-0688,pt,mixed,objective,objective,true,Perguntamos aqui se o apoio multilíngue altera a qualidade da revisão acadêmica.
96
+ word,ari-0689,pt,mixed,objective,objective,true,Nosso propósito é determinar quais fatores institucionais predizem a colaboração transfronteiriça.
97
+ word,ari-0690,pt,mixed,objective,objective,true,Pretendemos comparar explicações alternativas para diferenças na adoção digital.
98
+ word,ari-0691,pt,mixed,method,method,true,Os participantes responderam a duas ondas de pesquisa e as respostas foram modeladas com erros-padrão agrupados.
99
+ word,ari-0692,pt,mixed,method,method,true,Dois pesquisadores codificaram independentemente as entrevistas antes de reconciliar os temas.
100
+ word,ari-0693,pt,mixed,method,method,true,Reunimos indicadores anuais por país e estimamos uma especificação de diferenças em diferenças.
101
+ word,ari-0694,pt,mixed,result,result,true,"As pontuações aumentaram após a intervenção, com os maiores ganhos no grupo de tratamento."
102
+ word,ari-0695,pt,mixed,result,result,true,Três temas recorrentes emergiram das entrevistas e apareceram em todos os locais estudados.
103
+ word,ari-0696,pt,mixed,result,result,true,O coeficiente permaneceu positivo e estatisticamente diferente de zero em todos os testes de robustez.
104
+ word,ari-0697,pt,mixed,limitation,limitation,true,Nossas conclusões são limitadas pela dependência de comportamento autorrelatado.
105
+ word,ari-0698,pt,mixed,limitation,limitation,true,O curto período de observação impede avaliar se o efeito persiste.
106
+ word,ari-0699,pt,mixed,limitation,limitation,true,"Como os casos foram selecionados intencionalmente, a validade externa permanece incerta."
107
+ word,ari-0700,pt,mixed,implication,limitation,false,As universidades podem se beneficiar de orientações estruturadas sobre o uso responsável dessas ferramentas.
108
+ word,ari-0701,pt,mixed,implication,implication,true,O padrão indica que programas regionais devem considerar diferenças de capacidade institucional.
109
+ word,ari-0702,pt,mixed,implication,implication,true,Estudos futuros podem testar o mecanismo com dados longitudinais e amostras mais amplas.
110
+ char,ari-0649,en,mixed,background,background,true,Scholars continue to debate how digital tools reshape knowledge production.
111
+ char,ari-0650,en,mixed,background,background,true,Regional economic integration has received renewed attention after recent supply disruptions.
112
+ char,ari-0651,en,mixed,background,limitation,false,The rapid adoption of automated systems has intensified discussion about professional work.
113
+ char,ari-0652,en,mixed,objective,limitation,false,Here we ask whether multilingual support changes the quality of academic revision.
114
+ char,ari-0653,en,mixed,objective,implication,false,Our purpose is to determine which institutional factors predict cross-border collaboration.
115
+ char,ari-0654,en,mixed,objective,objective,true,We set out to compare alternative explanations for differences in digital adoption.
116
+ char,ari-0655,en,mixed,method,method,true,"Participants completed two survey waves, after which responses were modeled with clustered standard errors."
117
+ char,ari-0656,en,mixed,method,method,true,Interview transcripts were independently coded by two researchers before themes were reconciled.
118
+ char,ari-0657,en,mixed,method,result,false,We assembled annual country indicators and estimated a difference-in-differences specification.
119
+ char,ari-0658,en,mixed,result,limitation,false,"Scores increased after the intervention, with the largest gains observed in the treatment group."
120
+ char,ari-0659,en,mixed,result,result,true,Three recurring themes emerged from the interviews and were present across all study sites.
121
+ char,ari-0660,en,mixed,result,result,true,The coefficient remained positive and statistically distinguishable from zero in every robustness check.
122
+ char,ari-0661,en,mixed,limitation,limitation,true,Our conclusions are constrained by reliance on self-reported behavior.
123
+ char,ari-0662,en,mixed,limitation,limitation,true,The short observation window prevents assessment of whether the effect persists.
124
+ char,ari-0663,en,mixed,limitation,limitation,true,"Because the cases were purposively selected, external validity remains uncertain."
125
+ char,ari-0664,en,mixed,implication,implication,true,Universities may therefore benefit from providing structured guidance on responsible tool use.
126
+ char,ari-0665,en,mixed,implication,implication,true,The pattern indicates that regional programs should account for institutional capacity differences.
127
+ char,ari-0666,en,mixed,implication,implication,true,Subsequent studies could test the mechanism with longitudinal data and broader samples.
128
+ char,ari-0667,es,mixed,background,background,true,La literatura reciente debate cómo las herramientas digitales transforman la producción de conocimiento.
129
+ char,ari-0668,es,mixed,background,implication,false,La integración económica regional volvió a ganar atención tras las disrupciones en las cadenas de suministro.
130
+ char,ari-0669,es,mixed,background,limitation,false,La rápida adopción de sistemas automatizados intensificó el debate sobre el trabajo profesional.
131
+ char,ari-0670,es,mixed,objective,method,false,Aquí preguntamos si el apoyo multilingüe modifica la calidad de la revisión académica.
132
+ char,ari-0671,es,mixed,objective,implication,false,Nuestro propósito es determinar qué factores institucionales predicen la colaboración transfronteriza.
133
+ char,ari-0672,es,mixed,objective,objective,true,Nos proponemos comparar explicaciones alternativas de las diferencias en adopción digital.
134
+ char,ari-0673,es,mixed,method,method,true,Los participantes completaron dos rondas de encuesta y las respuestas se modelaron con errores estándar agrupados.
135
+ char,ari-0674,es,mixed,method,method,true,Dos investigadores codificaron de manera independiente las entrevistas antes de consensuar los temas.
136
+ char,ari-0675,es,mixed,method,method,true,Construimos indicadores anuales por país y estimamos un modelo de diferencias en diferencias.
137
+ char,ari-0676,es,mixed,result,implication,false,Las puntuaciones aumentaron después de la intervención y las mayores mejoras aparecieron en el grupo tratado.
138
+ char,ari-0677,es,mixed,result,result,true,De las entrevistas surgieron tres temas recurrentes presentes en todos los sitios estudiados.
139
+ char,ari-0678,es,mixed,result,result,true,El coeficiente se mantuvo positivo y estadísticamente distinto de cero en todas las pruebas de robustez.
140
+ char,ari-0679,es,mixed,limitation,limitation,true,Nuestras conclusiones están condicionadas por el uso de conductas autodeclaradas.
141
+ char,ari-0680,es,mixed,limitation,limitation,true,El breve período de observación impide saber si el efecto persiste.
142
+ char,ari-0681,es,mixed,limitation,limitation,true,"Como los casos fueron seleccionados intencionalmente, la validez externa sigue siendo incierta."
143
+ char,ari-0682,es,mixed,implication,implication,true,Las universidades podrían beneficiarse de ofrecer orientaciones estructuradas sobre el uso responsable de estas herramientas.
144
+ char,ari-0683,es,mixed,implication,implication,true,El patrón indica que los programas regionales deberían considerar las diferencias de capacidad institucional.
145
+ char,ari-0684,es,mixed,implication,implication,true,Estudios posteriores podrían probar el mecanismo con datos longitudinales y muestras más amplias.
146
+ char,ari-0685,pt,mixed,background,background,true,A literatura recente debate como ferramentas digitais transformam a produção de conhecimento.
147
+ char,ari-0686,pt,mixed,background,implication,false,A integração econômica regional voltou a receber atenção após interrupções nas cadeias de suprimento.
148
+ char,ari-0687,pt,mixed,background,limitation,false,A rápida adoção de sistemas automatizados intensificou o debate sobre o trabalho profissional.
149
+ char,ari-0688,pt,mixed,objective,method,false,Perguntamos aqui se o apoio multilíngue altera a qualidade da revisão acadêmica.
150
+ char,ari-0689,pt,mixed,objective,implication,false,Nosso propósito é determinar quais fatores institucionais predizem a colaboração transfronteiriça.
151
+ char,ari-0690,pt,mixed,objective,objective,true,Pretendemos comparar explicações alternativas para diferenças na adoção digital.
152
+ char,ari-0691,pt,mixed,method,method,true,Os participantes responderam a duas ondas de pesquisa e as respostas foram modeladas com erros-padrão agrupados.
153
+ char,ari-0692,pt,mixed,method,method,true,Dois pesquisadores codificaram independentemente as entrevistas antes de reconciliar os temas.
154
+ char,ari-0693,pt,mixed,method,result,false,Reunimos indicadores anuais por país e estimamos uma especificação de diferenças em diferenças.
155
+ char,ari-0694,pt,mixed,result,implication,false,"As pontuações aumentaram após a intervenção, com os maiores ganhos no grupo de tratamento."
156
+ char,ari-0695,pt,mixed,result,result,true,Três temas recorrentes emergiram das entrevistas e apareceram em todos os locais estudados.
157
+ char,ari-0696,pt,mixed,result,result,true,O coeficiente permaneceu positivo e estatisticamente diferente de zero em todos os testes de robustez.
158
+ char,ari-0697,pt,mixed,limitation,limitation,true,Nossas conclusões são limitadas pela dependência de comportamento autorrelatado.
159
+ char,ari-0698,pt,mixed,limitation,limitation,true,O curto período de observação impede avaliar se o efeito persiste.
160
+ char,ari-0699,pt,mixed,limitation,limitation,true,"Como os casos foram selecionados intencionalmente, a validade externa permanece incerta."
161
+ char,ari-0700,pt,mixed,implication,implication,true,As universidades podem se beneficiar de orientações estruturadas sobre o uso responsável dessas ferramentas.
162
+ char,ari-0701,pt,mixed,implication,implication,true,O padrão indica que programas regionais devem considerar diferenças de capacidade institucional.
163
+ char,ari-0702,pt,mixed,implication,implication,true,Estudos futuros podem testar o mecanismo com dados longitudinais e amostras mais amplas.
164
+ hybrid,ari-0649,en,mixed,background,objective,false,Scholars continue to debate how digital tools reshape knowledge production.
165
+ hybrid,ari-0650,en,mixed,background,background,true,Regional economic integration has received renewed attention after recent supply disruptions.
166
+ hybrid,ari-0651,en,mixed,background,limitation,false,The rapid adoption of automated systems has intensified discussion about professional work.
167
+ hybrid,ari-0652,en,mixed,objective,objective,true,Here we ask whether multilingual support changes the quality of academic revision.
168
+ hybrid,ari-0653,en,mixed,objective,objective,true,Our purpose is to determine which institutional factors predict cross-border collaboration.
169
+ hybrid,ari-0654,en,mixed,objective,objective,true,We set out to compare alternative explanations for differences in digital adoption.
170
+ hybrid,ari-0655,en,mixed,method,method,true,"Participants completed two survey waves, after which responses were modeled with clustered standard errors."
171
+ hybrid,ari-0656,en,mixed,method,method,true,Interview transcripts were independently coded by two researchers before themes were reconciled.
172
+ hybrid,ari-0657,en,mixed,method,method,true,We assembled annual country indicators and estimated a difference-in-differences specification.
173
+ hybrid,ari-0658,en,mixed,result,limitation,false,"Scores increased after the intervention, with the largest gains observed in the treatment group."
174
+ hybrid,ari-0659,en,mixed,result,method,false,Three recurring themes emerged from the interviews and were present across all study sites.
175
+ hybrid,ari-0660,en,mixed,result,result,true,The coefficient remained positive and statistically distinguishable from zero in every robustness check.
176
+ hybrid,ari-0661,en,mixed,limitation,limitation,true,Our conclusions are constrained by reliance on self-reported behavior.
177
+ hybrid,ari-0662,en,mixed,limitation,limitation,true,The short observation window prevents assessment of whether the effect persists.
178
+ hybrid,ari-0663,en,mixed,limitation,limitation,true,"Because the cases were purposively selected, external validity remains uncertain."
179
+ hybrid,ari-0664,en,mixed,implication,implication,true,Universities may therefore benefit from providing structured guidance on responsible tool use.
180
+ hybrid,ari-0665,en,mixed,implication,implication,true,The pattern indicates that regional programs should account for institutional capacity differences.
181
+ hybrid,ari-0666,en,mixed,implication,implication,true,Subsequent studies could test the mechanism with longitudinal data and broader samples.
182
+ hybrid,ari-0667,es,mixed,background,background,true,La literatura reciente debate cómo las herramientas digitales transforman la producción de conocimiento.
183
+ hybrid,ari-0668,es,mixed,background,implication,false,La integración económica regional volvió a ganar atención tras las disrupciones en las cadenas de suministro.
184
+ hybrid,ari-0669,es,mixed,background,limitation,false,La rápida adopción de sistemas automatizados intensificó el debate sobre el trabajo profesional.
185
+ hybrid,ari-0670,es,mixed,objective,limitation,false,Aquí preguntamos si el apoyo multilingüe modifica la calidad de la revisión académica.
186
+ hybrid,ari-0671,es,mixed,objective,objective,true,Nuestro propósito es determinar qué factores institucionales predicen la colaboración transfronteriza.
187
+ hybrid,ari-0672,es,mixed,objective,objective,true,Nos proponemos comparar explicaciones alternativas de las diferencias en adopción digital.
188
+ hybrid,ari-0673,es,mixed,method,method,true,Los participantes completaron dos rondas de encuesta y las respuestas se modelaron con errores estándar agrupados.
189
+ hybrid,ari-0674,es,mixed,method,method,true,Dos investigadores codificaron de manera independiente las entrevistas antes de consensuar los temas.
190
+ hybrid,ari-0675,es,mixed,method,method,true,Construimos indicadores anuales por país y estimamos un modelo de diferencias en diferencias.
191
+ hybrid,ari-0676,es,mixed,result,limitation,false,Las puntuaciones aumentaron después de la intervención y las mayores mejoras aparecieron en el grupo tratado.
192
+ hybrid,ari-0677,es,mixed,result,result,true,De las entrevistas surgieron tres temas recurrentes presentes en todos los sitios estudiados.
193
+ hybrid,ari-0678,es,mixed,result,result,true,El coeficiente se mantuvo positivo y estadísticamente distinto de cero en todas las pruebas de robustez.
194
+ hybrid,ari-0679,es,mixed,limitation,limitation,true,Nuestras conclusiones están condicionadas por el uso de conductas autodeclaradas.
195
+ hybrid,ari-0680,es,mixed,limitation,limitation,true,El breve período de observación impide saber si el efecto persiste.
196
+ hybrid,ari-0681,es,mixed,limitation,limitation,true,"Como los casos fueron seleccionados intencionalmente, la validez externa sigue siendo incierta."
197
+ hybrid,ari-0682,es,mixed,implication,limitation,false,Las universidades podrían beneficiarse de ofrecer orientaciones estructuradas sobre el uso responsable de estas herramientas.
198
+ hybrid,ari-0683,es,mixed,implication,implication,true,El patrón indica que los programas regionales deberían considerar las diferencias de capacidad institucional.
199
+ hybrid,ari-0684,es,mixed,implication,implication,true,Estudios posteriores podrían probar el mecanismo con datos longitudinales y muestras más amplias.
200
+ hybrid,ari-0685,pt,mixed,background,background,true,A literatura recente debate como ferramentas digitais transformam a produção de conhecimento.
201
+ hybrid,ari-0686,pt,mixed,background,implication,false,A integração econômica regional voltou a receber atenção após interrupções nas cadeias de suprimento.
202
+ hybrid,ari-0687,pt,mixed,background,limitation,false,A rápida adoção de sistemas automatizados intensificou o debate sobre o trabalho profissional.
203
+ hybrid,ari-0688,pt,mixed,objective,objective,true,Perguntamos aqui se o apoio multilíngue altera a qualidade da revisão acadêmica.
204
+ hybrid,ari-0689,pt,mixed,objective,objective,true,Nosso propósito é determinar quais fatores institucionais predizem a colaboração transfronteiriça.
205
+ hybrid,ari-0690,pt,mixed,objective,objective,true,Pretendemos comparar explicações alternativas para diferenças na adoção digital.
206
+ hybrid,ari-0691,pt,mixed,method,method,true,Os participantes responderam a duas ondas de pesquisa e as respostas foram modeladas com erros-padrão agrupados.
207
+ hybrid,ari-0692,pt,mixed,method,method,true,Dois pesquisadores codificaram independentemente as entrevistas antes de reconciliar os temas.
208
+ hybrid,ari-0693,pt,mixed,method,method,true,Reunimos indicadores anuais por país e estimamos uma especificação de diferenças em diferenças.
209
+ hybrid,ari-0694,pt,mixed,result,implication,false,"As pontuações aumentaram após a intervenção, com os maiores ganhos no grupo de tratamento."
210
+ hybrid,ari-0695,pt,mixed,result,result,true,Três temas recorrentes emergiram das entrevistas e apareceram em todos os locais estudados.
211
+ hybrid,ari-0696,pt,mixed,result,result,true,O coeficiente permaneceu positivo e estatisticamente diferente de zero em todos os testes de robustez.
212
+ hybrid,ari-0697,pt,mixed,limitation,limitation,true,Nossas conclusões são limitadas pela dependência de comportamento autorrelatado.
213
+ hybrid,ari-0698,pt,mixed,limitation,limitation,true,O curto período de observação impede avaliar se o efeito persiste.
214
+ hybrid,ari-0699,pt,mixed,limitation,limitation,true,"Como os casos foram selecionados intencionalmente, a validade externa permanece incerta."
215
+ hybrid,ari-0700,pt,mixed,implication,implication,true,As universidades podem se beneficiar de orientações estruturadas sobre o uso responsável dessas ferramentas.
216
+ hybrid,ari-0701,pt,mixed,implication,implication,true,O padrão indica que programas regionais devem considerar diferenças de capacidade institucional.
217
+ hybrid,ari-0702,pt,mixed,implication,implication,true,Estudos futuros podem testar o mecanismo com dados longitudinais e amostras mais amplas.
dataset_manifest.json ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "id": "modelops-research-lab-v2",
3
+ "version": "2.0.0",
4
+ "source_dataset": "Academic Rhetorical Intent Mini v1.1",
5
+ "source_type": "synthetic_curated",
6
+ "dataset_license": "CC-BY-4.0",
7
+ "code_license": "MIT",
8
+ "counts": {
9
+ "total": 1026,
10
+ "train": 842,
11
+ "iid_test": 130,
12
+ "challenge": 54
13
+ },
14
+ "languages": {
15
+ "en": 342,
16
+ "pt": 342,
17
+ "es": 342
18
+ },
19
+ "labels": {
20
+ "result": 171,
21
+ "objective": 171,
22
+ "limitation": 171,
23
+ "method": 171,
24
+ "implication": 171,
25
+ "background": 171
26
+ },
27
+ "domains": {
28
+ "business": 54,
29
+ "education": 54,
30
+ "educação": 54,
31
+ "negócios": 54,
32
+ "educación": 54,
33
+ "international_relations": 54,
34
+ "public_policy": 54,
35
+ "economics": 54,
36
+ "relações_internacionais": 54,
37
+ "política_pública": 108,
38
+ "marketing": 162,
39
+ "economia": 54,
40
+ "economía": 54,
41
+ "relaciones_internacionales": 54,
42
+ "negocios": 54,
43
+ "mixed": 54
44
+ },
45
+ "splits": {
46
+ "train": "train.json",
47
+ "iid_test": "iid_test.json",
48
+ "challenge": "challenge.json"
49
+ },
50
+ "original_dataset_csv": "academic_rhetoric_intent.csv",
51
+ "files_sha256": {
52
+ "train.json": "0fe09404aba02f733f068d2f4f5d79c77608483d2b22e56f5878cac7b1006d53",
53
+ "iid_test.json": "da645782ce3bf8b903da29672b19928bb4fcedfa42c6d4074737d07f35028f71",
54
+ "challenge.json": "bb88b8956def23691042a2088cfb08b8a5c9e93a9118a68fbd47230e94e8e5eb",
55
+ "academic_rhetoric_intent.csv": "10e55e45096266137e21450d439638ef19a4ee2c1affacb84b38a2f6b4e65001"
56
+ },
57
+ "notice": "All records are synthetic-curated; do not report metrics as real-world scholarly validity."
58
+ }
engine.js ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /* ModelOps Research Lab | zero-dependency reproducible text-classification baselines. MIT. */
2
+ (function(root, factory){const obj=factory();if(typeof module==='object'&&module.exports)module.exports=obj;root.ModelOpsEngine=obj;})(typeof globalThis!=='undefined'?globalThis:this,function(){
3
+ 'use strict';
4
+ const LABELS=['background','objective','method','result','limitation','implication'];
5
+ function clean(s){return String(s||'').toLowerCase().normalize('NFKD').replace(/[\u0300-\u036f]/g,'');}
6
+ function tokens(s){return (clean(s).match(/[a-z0-9]+/g)||[]).filter(t=>t.length>1);}
7
+ function features(s,kind){let items;if(kind==='char'){const t=' '+clean(s).replace(/[^a-z0-9 ]/g,' ').replace(/\s+/g,' ').trim()+' ';items=[];for(let i=0;i<t.length-2;i++){const x=t.slice(i,i+3);if(x.trim().length>1)items.push(x);}}else items=tokens(s);
8
+ const out=new Map();for(const x of items)out.set(x,(out.get(x)||0)+1);return out;}
9
+ function cosine(a,b){let d=0;const sm=a.size<b.size?a:b,other=a.size<b.size?b:a;for(const [k,v] of sm)d+=v*(other.get(k)||0);return d;}
10
+ function normalize(v){let n=0;for(const x of v.values())n+=x*x;n=Math.sqrt(n)||1;const out=new Map();for(const [k,x]of v)out.set(k,x/n);return out;}
11
+ function trainCentroid(rows,kind){const df=new Map();const itemRows=[];for(const r of rows){const v=features(r.text,kind);itemRows.push(v);for(const k of v.keys())df.set(k,(df.get(k)||0)+1);}const idf=new Map();for(const [k,c] of df)idf.set(k,Math.log((rows.length+1)/(c+1))+1);const profiles=Object.fromEntries(LABELS.map(x=>[x,new Map()]));const n=Object.fromEntries(LABELS.map(x=>[x,0]));for(let i=0;i<rows.length;i++){const r=rows[i];n[r.label]++;const v=itemRows[i];const weighted=new Map();for(const [k,c] of v)weighted.set(k,(1+Math.log(c))*(idf.get(k)||1));const vn=normalize(weighted);for(const [k,x]of vn)profiles[r.label].set(k,(profiles[r.label].get(k)||0)+x);}for(const x of LABELS)profiles[x]=normalize(profiles[x]);return {type:kind==='char'?'Character TF-IDF':'Word TF-IDF',kind,idf,profiles,n};}
12
+ function scoreCentroid(model,s){const v=features(s,model.kind);const weighted=new Map();for(const [k,c] of v)if(model.idf.has(k))weighted.set(k,(1+Math.log(c))*model.idf.get(k));const norm=normalize(weighted);return LABELS.map(label=>cosine(norm,model.profiles[label]));}
13
+ function trainNB(rows,alpha=1){const word=new Map(),tot=new Map(),prior=new Map(),vocab=new Set();for(const label of LABELS){word.set(label,new Map());tot.set(label,0);prior.set(label,0);}for(const r of rows){prior.set(r.label,prior.get(r.label)+1);const map=word.get(r.label);for(const [token,n] of features(r.text,'word')){map.set(token,(map.get(token)||0)+n);tot.set(r.label,tot.get(r.label)+n);vocab.add(token);}}return {type:'Multinomial Naive Bayes',word,tot,prior,vocab,alpha,n:rows.length};}
14
+ function scoreNB(model,s){const f=features(s,'word');const scores=LABELS.map(label=>{let v=Math.log((model.prior.get(label)+1)/(model.n+LABELS.length));const map=model.word.get(label);const denom=model.tot.get(label)+model.alpha*(model.vocab.size+1);for(const [token,n] of f){if(!model.vocab.has(token))continue;v+=n*Math.log(((map.get(token)||0)+model.alpha)/denom);}return v;});const max=Math.max(...scores);const exp=scores.map(x=>Math.exp(x-max)),z=exp.reduce((a,b)=>a+b,0);return exp.map(x=>x/z);}
15
+ function argmax(s){let m=0;for(let j=1;j<s.length;j++)if(s[j]>s[m])m=j;return m;}
16
+ function train(rows){if(!Array.isArray(rows)||rows.length===0)throw Error('Training data is empty');for(const r of rows)if(!LABELS.includes(r.label)||!String(r.text||'').trim())throw Error('Training row has invalid label or empty text');return {nb:trainNB(rows),word:trainCentroid(rows,'word'),char:trainCentroid(rows,'char'),trainedRows:rows.length};}
17
+ function predict(bundle,text,method){const word=scoreCentroid(bundle.word,text),char=scoreCentroid(bundle.char,text);let scores;if(method==='nb')scores=scoreNB(bundle.nb,text);else if(method==='word')scores=word;else if(method==='char')scores=char;else if(method==='hybrid')scores=word.map((v,i)=>.55*v+.45*char[i]);else throw Error('Invalid model '+method);const index=argmax(scores);return {label:LABELS[index],scores,margin:[...scores].sort((a,b)=>b-a)[0]-[...scores].sort((a,b)=>b-a)[1]};}
18
+ function safeDivide(a,b){return b?a/b:0;}
19
+ function evaluation(predictions){const n=predictions.length;if(!n)throw Error('No rows to evaluate');let correct=0,macro=0,withSupport=0;const confusion=Object.fromEntries(LABELS.map(a=>[a,Object.fromEntries(LABELS.map(b=>[b,0]))]));for(const r of predictions){if(!LABELS.includes(r.truth)||!LABELS.includes(r.pred))throw Error('Invalid evaluation label');confusion[r.truth][r.pred]++;if(r.pred===r.truth)correct++;}const byLabel={};for(const l of LABELS){const tp=confusion[l][l],fn=LABELS.reduce((s,b)=>s+confusion[l][b],0)-tp,fp=LABELS.reduce((s,a)=>s+confusion[a][l],0)-tp,p=safeDivide(tp,tp+fp),recall=safeDivide(tp,tp+fn),f1=safeDivide(2*p*recall,p+recall);byLabel[l]={precision:p,recall,f1,support:tp+fn};if(tp+fn>0){macro+=f1;withSupport++;}}return {n,correct,accuracy:correct/n,macroF1:macro/withSupport,byLabel,confusion};}
20
+ function evaluate(bundle,rows,method){return rows.map(r=>({id:r.id,language:r.language,domain:r.domain,truth:r.label,text:r.text,pred:predict(bundle,r.text,method).label}));}
21
+ function summarizeSlices(predictions,key){const groups={};for(const p of predictions){const k=String(p[key]||'unknown');(groups[k]??=[]).push(p);}return Object.entries(groups).sort((a,b)=>a[0].localeCompare(b[0])).map(([name,records])=>({name,...evaluation(records)}));}
22
+ function seeded(seed){let t=(Number(seed)||1)>>>0;return function(){t+=0x6D2B79F5;let x=t;x=Math.imul(x^(x>>>15),x|1);x^=x+Math.imul(x^(x>>>7),x|61);return((x^(x>>>14))>>>0)/4294967296;};}
23
+ function bootstrapAccuracy(predictions,seed=1234,reps=300){if(!predictions.length)throw Error('No predictions');if(reps<10||reps>5000)throw Error('Invalid repetitions');const rnd=seeded(seed),out=[];for(let i=0;i<reps;i++){let good=0;for(let j=0;j<predictions.length;j++){const p=predictions[Math.floor(rnd()*predictions.length)];if(p.pred===p.truth)good++;}out.push(good/predictions.length);}out.sort((a,b)=>a-b);return {low:out[Math.floor(.025*(reps-1))],high:out[Math.ceil(.975*(reps-1))],reps};}
24
+ function audit(trainRows,heldoutRows){const seen=new Set(trainRows.map(x=>clean(x.text).replace(/\s+/g,' ').trim()));const idSeen=new Set(trainRows.map(x=>x.id));let overlap=0,dupIds=0,empty=0;for(const r of heldoutRows){if(seen.has(clean(r.text).replace(/\s+/g,' ').trim()))overlap++;if(idSeen.has(r.id))dupIds++;if(!r.text||!LABELS.includes(r.label))empty++;}return {train:trainRows.length,evaluation:heldoutRows.length,exactTextOverlap:overlap,duplicateIds:dupIds,invalidRows:empty,clean:overlap===0&&dupIds===0&&empty===0};}
25
+ function compareRuns(a,b){const x=new Map(a.map(p=>[p.id,p]));let changed=0,bWins=0,aWins=0;for(const row of b){const old=x.get(row.id);if(!old)continue;if(old.pred!==row.pred)changed++;if(old.pred!==old.truth&&row.pred===row.truth)bWins++;if(old.pred===old.truth&&row.pred!==row.truth)aWins++;}return {changed,bWins,aWins};}
26
+ function csvCell(x){const s=String(x??'');return /[",\r\n]/.test(s)?'"'+s.replace(/"/g,'""')+'"':s;}
27
+ function toCSV(rows,fields){return [fields.join(','),...rows.map(r=>fields.map(k=>csvCell(r[k])).join(','))].join('\r\n');}
28
+ function version(){return '2.0.0';}
29
+ return {LABELS,features,train,predict,evaluation,evaluate,summarizeSlices,bootstrapAccuracy,audit,compareRuns,toCSV,version,seeded};
30
+ });
evaluation_results.json ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "project": "ModelOps Research Lab",
3
+ "release": "2.0.0",
4
+ "dataset": "Academic Rhetorical Intent Mini v1.1",
5
+ "source_type": "synthetic_curated",
6
+ "train_count": 842,
7
+ "test_count": 130,
8
+ "challenge_count": 54,
9
+ "run_type": "recomputed Node baselines",
10
+ "methods": {
11
+ "nb": {
12
+ "iid": {
13
+ "n": 130,
14
+ "correct": 130,
15
+ "accuracy": 1,
16
+ "macroF1": 1
17
+ },
18
+ "challenge": {
19
+ "n": 54,
20
+ "correct": 40,
21
+ "accuracy": 0.7407407407407407,
22
+ "macroF1": 0.7362809928289805
23
+ }
24
+ },
25
+ "word": {
26
+ "iid": {
27
+ "n": 130,
28
+ "correct": 128,
29
+ "accuracy": 0.9846153846153847,
30
+ "macroF1": 0.9844877344877346
31
+ },
32
+ "challenge": {
33
+ "n": 54,
34
+ "correct": 39,
35
+ "accuracy": 0.7222222222222222,
36
+ "macroF1": 0.7061942959001782
37
+ }
38
+ },
39
+ "char": {
40
+ "iid": {
41
+ "n": 130,
42
+ "correct": 124,
43
+ "accuracy": 0.9538461538461539,
44
+ "macroF1": 0.9537925842273668
45
+ },
46
+ "challenge": {
47
+ "n": 54,
48
+ "correct": 38,
49
+ "accuracy": 0.7037037037037037,
50
+ "macroF1": 0.6836089069592907
51
+ }
52
+ },
53
+ "hybrid": {
54
+ "iid": {
55
+ "n": 130,
56
+ "correct": 127,
57
+ "accuracy": 0.9769230769230769,
58
+ "macroF1": 0.9767441860465116
59
+ },
60
+ "challenge": {
61
+ "n": 54,
62
+ "correct": 42,
63
+ "accuracy": 0.7777777777777778,
64
+ "macroF1": 0.7617571707045391
65
+ }
66
+ }
67
+ }
68
+ }
iid_test.json ADDED
@@ -0,0 +1 @@
 
 
1
+ [{"id":"ari-0374","language":"es","label":"limitation","domain":"negocios","text":"El análisis no captura cambios de largo plazo en transformación digital.","template_group":"es-limitation-2"},{"id":"ari-0444","language":"pt","label":"background","domain":"política_pública","text":"prestação de serviços públicos tornou-se uma questão relevante para pesquisadores que estudam governos locais.","template_group":"pt-background-1"},{"id":"ari-0220","language":"es","label":"background","domain":"relaciones_internacionales","text":"La literatura previa ha examinado cada vez más cooperación transfronteriza entre organizaciones regionales.","template_group":"es-background-0"},{"id":"ari-0566","language":"pt","label":"result","domain":"negócios","text":"O efeito estimado de transformação digital permaneceu robusto em especificações alternativas.","template_group":"pt-result-4"},{"id":"ari-0292","language":"es","label":"method","domain":"relaciones_internacionales","text":"Analizamos datos de encuesta de organizaciones regionales mediante regresión multivariada.","template_group":"es-method-0"},{"id":"ari-0609","language":"pt","label":"limitation","domain":"economia","text":"Seriam necessários dados adicionais para verificar se o padrão de dinâmica da inflação persiste ao longo do tempo.","template_group":"pt-limitation-5"},{"id":"ari-0549","language":"pt","label":"result","domain":"economia","text":"Encontramos que níveis mais altos de dinâmica da inflação estiveram consistentemente relacionados a melhores resultados entre economias latino-americanas.","template_group":"pt-result-1"},{"id":"ari-0073","language":"en","label":"method","domain":"education","text":"We analyzed survey data collected from university students using multivariate regression.","template_group":"en-method-0"},{"id":"ari-0499","language":"pt","label":"objective","domain":"educação","text":"Nossa análise se concentra em explicar a variação de escrita acadêmica entre estudantes universitários.","template_group":"pt-objective-5"},{"id":"ari-0536","language":"pt","label":"method","domain":"negócios","text":"A análise utilizou dados em painel e modelos de efeitos fixos para estimar padrões de transformação digital.","template_group":"pt-method-5"},{"id":"ari-0455","language":"pt","label":"background","domain":"marketing","text":"Uma literatura crescente destaca a relevância de engajamento com marcas para consumidores digitais.","template_group":"pt-background-3"},{"id":"ari-0305","language":"es","label":"method","domain":"marketing","text":"Los datos sobre vinculación con marcas fueron codificados y analizados mediante un flujo estadístico reproducible.","template_group":"es-method-2"},{"id":"ari-0208","language":"en","label":"implication","domain":"international_relations","text":"These results may inform institutional decisions concerning cross-border cooperation and regional organizations.","template_group":"en-implication-4"},{"id":"ari-0125","language":"en","label":"result","domain":"marketing","text":"The analysis identified substantial variation in brand engagement across the observed cases.","template_group":"en-result-2"},{"id":"ari-0316","language":"es","label":"method","domain":"relaciones_internacionales","text":"Se utilizó un diseño comparativo de estudios de caso para examinar cooperación transfronteriza en varios contextos.","template_group":"es-method-4"},{"id":"ari-0542","language":"pt","label":"result","domain":"negócios","text":"Os resultados mostram uma associação significativa entre transformação digital e os resultados de pequenas empresas.","template_group":"pt-result-0"},{"id":"ari-0393","language":"es","label":"limitation","domain":"economía","text":"Se necesitarían datos adicionales para comprobar si el patrón de dinámica inflacionaria persiste en el tiempo.","template_group":"es-limitation-5"},{"id":"ari-0192","language":"en","label":"implication","domain":"public_policy","text":"The results highlight the practical importance of incorporating public service delivery into strategies for local governments.","template_group":"en-implication-1"},{"id":"ari-0032","language":"en","label":"background","domain":"business","text":"Debates on digital transformation have expanded across several disciplines in recent years.","template_group":"en-background-5"},{"id":"ari-0395","language":"es","label":"limitation","domain":"marketing","text":"Se necesitarían datos adicionales para comprobar si el patrón de vinculación con marcas persiste en el tiempo.","template_group":"es-limitation-5"},{"id":"ari-0188","language":"en","label":"implication","domain":"business","text":"The results highlight the practical importance of incorporating digital transformation into strategies for small firms.","template_group":"en-implication-1"},{"id":"ari-0540","language":"pt","label":"method","domain":"política_pública","text":"A análise utilizou dados em painel e modelos de efeitos fixos para estimar padrões de prestação de serviços públicos.","template_group":"pt-method-5"},{"id":"ari-0036","language":"en","label":"background","domain":"public_policy","text":"Debates on public service delivery have expanded across several disciplines in recent years.","template_group":"en-background-5"},{"id":"ari-0120","language":"en","label":"result","domain":"public_policy","text":"We found that higher levels of public service delivery were consistently related to improved outcomes among local governments.","template_group":"en-result-1"},{"id":"ari-0024","language":"en","label":"background","domain":"public_policy","text":"A growing body of literature highlights the relevance of public service delivery for local governments.","template_group":"en-background-3"},{"id":"ari-0352","language":"es","label":"result","domain":"relaciones_internacionales","text":"El efecto estimado de cooperación transfronteriza se mantuvo robusto en especificaciones alternativas.","template_group":"es-result-4"},{"id":"ari-0397","language":"es","label":"implication","domain":"educación","text":"Estos hallazgos sugieren que las políticas deberían considerar escritura académica al diseñar intervenciones para estudiantes universitarios.","template_group":"es-implication-0"},{"id":"ari-0482","language":"pt","label":"objective","domain":"negócios","text":"Investigamos a relação entre transformação digital e os resultados observados em pequenas empresas.","template_group":"pt-objective-2"},{"id":"ari-0254","language":"es","label":"objective","domain":"negocios","text":"Este estudio tiene como objetivo examinar cómo transformación digital afecta a pequeñas empresas.","template_group":"es-objective-0"},{"id":"ari-0330","language":"es","label":"result","domain":"política_pública","text":"Los resultados muestran una asociación significativa entre prestación de servicios públicos y los resultados de gobiernos locales.","template_group":"es-result-0"},{"id":"ari-0644","language":"pt","label":"implication","domain":"negócios","text":"Os achados apontam para a necessidade de políticas mais específicas que abordem transformação digital.","template_group":"pt-implication-5"},{"id":"ari-0577","language":"pt","label":"limitation","domain":"educação","text":"O estudo é limitado pelo tamanho relativamente pequeno da amostra de estudantes universitários.","template_group":"pt-limitation-0"},{"id":"ari-0182","language":"en","label":"implication","domain":"business","text":"These findings suggest that policymakers should consider digital transformation when designing interventions for small firms.","template_group":"en-implication-0"},{"id":"ari-0219","language":"es","label":"background","domain":"economía","text":"La literatura previa ha examinado cada vez más dinámica inflacionaria entre economías latinoamericanas.","template_group":"es-background-0"},{"id":"ari-0169","language":"en","label":"limitation","domain":"education","text":"The geographic scope limits the generalizability of the findings about academic writing.","template_group":"en-limitation-4"},{"id":"ari-0261","language":"es","label":"objective","domain":"economía","text":"El objetivo de esta investigación es evaluar dinámica inflacionaria entre economías latinoamericanas.","template_group":"es-objective-1"},{"id":"ari-0153","language":"en","label":"limitation","domain":"economics","text":"Because the data are observational, causal claims about inflation dynamics should be interpreted cautiously.","template_group":"en-limitation-1"},{"id":"ari-0401","language":"es","label":"implication","domain":"marketing","text":"Estos hallazgos sugieren que las políticas deberían considerar vinculación con marcas al diseñar intervenciones para consumidores digitales.","template_group":"es-implication-0"},{"id":"ari-0545","language":"pt","label":"result","domain":"marketing","text":"Os resultados mostram uma associação significativa entre engajamento com marcas e os resultados de consumidores digitais.","template_group":"pt-result-0"},{"id":"ari-0317","language":"es","label":"method","domain":"marketing","text":"Se utilizó un diseño comparativo de estudios de caso para examinar vinculación con marcas en varios contextos.","template_group":"es-method-4"},{"id":"ari-0243","language":"es","label":"background","domain":"economía","text":"A pesar del interés sostenido, el contexto más amplio de dinámica inflacionaria sigue documentado de forma desigual.","template_group":"es-background-4"},{"id":"ari-0458","language":"pt","label":"background","domain":"negócios","text":"Apesar do interesse contínuo, o contexto mais amplo de transformação digital permanece documentado de forma desigual.","template_group":"pt-background-4"},{"id":"ari-0145","language":"en","label":"limitation","domain":"education","text":"The study is limited by the relatively small sample of university students.","template_group":"en-limitation-0"},{"id":"ari-0507","language":"pt","label":"method","domain":"economia","text":"Analisamos dados de pesquisa de economias latino-americanas por meio de regressão multivariada.","template_group":"pt-method-0"},{"id":"ari-0068","language":"en","label":"objective","domain":"business","text":"Our analysis focuses on explaining variation in digital transformation across small firms.","template_group":"en-objective-5"},{"id":"ari-0200","language":"en","label":"implication","domain":"business","text":"The evidence supports greater attention to digital transformation in programs targeting small firms.","template_group":"en-implication-3"},{"id":"ari-0056","language":"en","label":"objective","domain":"business","text":"This article seeks to identify the main drivers of digital transformation for small firms.","template_group":"en-objective-3"},{"id":"ari-0612","language":"pt","label":"limitation","domain":"política_pública","text":"Seriam necessários dados adicionais para verificar se o padrão de prestação de serviços públicos persiste ao longo do tempo.","template_group":"pt-limitation-5"},{"id":"ari-0335","language":"es","label":"result","domain":"marketing","text":"Encontramos que niveles más altos de vinculación con marcas se relacionaron de forma consistente con mejores resultados entre consumidores digitales.","template_group":"es-result-1"},{"id":"ari-0628","language":"pt","label":"implication","domain":"relações_internacionais","text":"Pesquisas futuras devem examinar se o padrão observado em cooperação transfronteiriça se estende a outros contextos.","template_group":"pt-implication-2"},{"id":"ari-0638","language":"pt","label":"implication","domain":"negócios","text":"Esses resultados podem orientar decisões institucionais relacionadas a transformação digital e pequenas empresas.","template_group":"pt-implication-4"},{"id":"ari-0302","language":"es","label":"method","domain":"negocios","text":"Los datos sobre transformación digital fueron codificados y analizados mediante un flujo estadístico reproducible.","template_group":"es-method-2"},{"id":"ari-0376","language":"es","label":"limitation","domain":"relaciones_internacionales","text":"El análisis no captura cambios de largo plazo en cooperación transfronteriza.","template_group":"es-limitation-2"},{"id":"ari-0468","language":"pt","label":"background","domain":"política_pública","text":"Os debates sobre prestação de serviços públicos se expandiram em várias disciplinas nos últimos anos.","template_group":"pt-background-5"},{"id":"ari-0381","language":"es","label":"limitation","domain":"economía","text":"El error de medición puede afectar la relación estimada que involucra dinámica inflacionaria.","template_group":"es-limitation-3"},{"id":"ari-0496","language":"pt","label":"objective","domain":"relações_internacionais","text":"O estudo avalia se mudanças em cooperação transfronteiriça estão associadas a resultados entre organizações regionais.","template_group":"pt-objective-4"},{"id":"ari-0159","language":"en","label":"limitation","domain":"economics","text":"The analysis does not capture long-term changes in inflation dynamics.","template_group":"en-limitation-2"},{"id":"ari-0285","language":"es","label":"objective","domain":"economía","text":"Nuestro análisis se centra en explicar la variación de dinámica inflacionaria entre economías latinoamericanas.","template_group":"es-objective-5"},{"id":"ari-0278","language":"es","label":"objective","domain":"negocios","text":"El estudio evalúa si los cambios en transformación digital se asocian con resultados entre pequeñas empresas.","template_group":"es-objective-4"},{"id":"ari-0553","language":"pt","label":"result","domain":"educação","text":"A análise identificou variação substancial de escrita acadêmica entre os casos observados.","template_group":"pt-result-2"},{"id":"ari-0645","language":"pt","label":"implication","domain":"economia","text":"Os achados apontam para a necessidade de políticas mais específicas que abordem dinâmica da inflação.","template_group":"pt-implication-5"},{"id":"ari-0232","language":"es","label":"background","domain":"relaciones_internacionales","text":"Los cambios recientes en cooperación transfronteriza han modificado el contexto en el que operan organizaciones regionales.","template_group":"es-background-2"},{"id":"ari-0107","language":"en","label":"method","domain":"marketing","text":"The analysis used panel data and fixed-effects models to estimate patterns in brand engagement.","template_group":"en-method-5"},{"id":"ari-0584","language":"pt","label":"limitation","domain":"negócios","text":"Como os dados são observacionais, afirmações causais sobre transformação digital devem ser interpretadas com cautela.","template_group":"pt-limitation-1"},{"id":"ari-0098","language":"en","label":"method","domain":"business","text":"A comparative case-study design was used to examine digital transformation across multiple settings.","template_group":"en-method-4"},{"id":"ari-0339","language":"es","label":"result","domain":"economía","text":"El análisis identificó una variación sustancial de dinámica inflacionaria entre los casos observados.","template_group":"es-result-2"},{"id":"ari-0484","language":"pt","label":"objective","domain":"relações_internacionais","text":"Investigamos a relação entre cooperação transfronteiriça e os resultados observados em organizações regionais.","template_group":"pt-objective-2"},{"id":"ari-0245","language":"es","label":"background","domain":"marketing","text":"A pesar del interés sostenido, el contexto más amplio de vinculación con marcas sigue documentado de forma desigual.","template_group":"es-background-4"},{"id":"ari-0071","language":"en","label":"objective","domain":"marketing","text":"Our analysis focuses on explaining variation in brand engagement across online consumers.","template_group":"en-objective-5"},{"id":"ari-0433","language":"pt","label":"background","domain":"educação","text":"A literatura anterior tem examinado cada vez mais escrita acadêmica entre estudantes universitários.","template_group":"pt-background-0"},{"id":"ari-0359","language":"es","label":"result","domain":"marketing","text":"Los resultados revelan que los cambios en vinculación con marcas se concentraron en un subconjunto de consumidores digitales.","template_group":"es-result-5"},{"id":"ari-0108","language":"en","label":"method","domain":"public_policy","text":"The analysis used panel data and fixed-effects models to estimate patterns in public service delivery.","template_group":"en-method-5"},{"id":"ari-0624","language":"pt","label":"implication","domain":"política_pública","text":"Os resultados destacam a importância prática de incorporar prestação de serviços públicos em estratégias voltadas para governos locais.","template_group":"pt-implication-1"},{"id":"ari-0494","language":"pt","label":"objective","domain":"negócios","text":"O estudo avalia se mudanças em transformação digital estão associadas a resultados entre pequenas empresas.","template_group":"pt-objective-4"},{"id":"ari-0396","language":"es","label":"limitation","domain":"política_pública","text":"Se necesitarían datos adicionales para comprobar si el patrón de prestación de servicios públicos persiste en el tiempo.","template_group":"es-limitation-5"},{"id":"ari-0235","language":"es","label":"background","domain":"educación","text":"Una literatura creciente destaca la relevancia de escritura académica para estudiantes universitarios.","template_group":"es-background-3"},{"id":"ari-0151","language":"en","label":"limitation","domain":"education","text":"Because the data are observational, causal claims about academic writing should be interpreted cautiously.","template_group":"en-limitation-1"},{"id":"ari-0064","language":"en","label":"objective","domain":"international_relations","text":"The study evaluates whether changes in cross-border cooperation are associated with outcomes among regional organizations.","template_group":"en-objective-4"},{"id":"ari-0002","language":"en","label":"background","domain":"business","text":"Prior scholarship has increasingly examined digital transformation among small firms.","template_group":"en-background-0"},{"id":"ari-0028","language":"en","label":"background","domain":"international_relations","text":"Despite sustained interest, the broader context of cross-border cooperation remains unevenly documented.","template_group":"en-background-4"},{"id":"ari-0135","language":"en","label":"result","domain":"economics","text":"The estimated effect of inflation dynamics remained robust across alternative specifications.","template_group":"en-result-4"},{"id":"ari-0263","language":"es","label":"objective","domain":"marketing","text":"El objetivo de esta investigación es evaluar vinculación con marcas entre consumidores digitales.","template_group":"es-objective-1"},{"id":"ari-0075","language":"en","label":"method","domain":"economics","text":"We analyzed survey data collected from Latin American economies using multivariate regression.","template_group":"en-method-0"},{"id":"ari-0547","language":"pt","label":"result","domain":"educação","text":"Encontramos que níveis mais altos de escrita acadêmica estiveram consistentemente relacionados a melhores resultados entre estudantes universitários.","template_group":"pt-result-1"},{"id":"ari-0042","language":"en","label":"objective","domain":"public_policy","text":"This study aims to examine how public service delivery affects local governments.","template_group":"en-objective-0"},{"id":"ari-0513","language":"pt","label":"method","domain":"economia","text":"O estudo empregou um desenho de métodos mistos combinando entrevistas e análise quantitativa de dinâmica da inflação.","template_group":"pt-method-1"},{"id":"ari-0086","language":"en","label":"method","domain":"business","text":"Data on digital transformation were coded and analyzed using a reproducible statistical workflow.","template_group":"en-method-2"},{"id":"ari-0586","language":"pt","label":"limitation","domain":"relações_internacionais","text":"Como os dados são observacionais, afirmações causais sobre cooperação transfronteiriça devem ser interpretadas com cautela.","template_group":"pt-limitation-1"},{"id":"ari-0405","language":"es","label":"implication","domain":"economía","text":"Los resultados destacan la importancia práctica de incorporar dinámica inflacionaria en estrategias dirigidas a economías latinoamericanas.","template_group":"es-implication-1"},{"id":"ari-0274","language":"es","label":"objective","domain":"relaciones_internacionales","text":"Este artículo busca identificar los principales determinantes de cooperación transfronteriza para organizaciones regionales.","template_group":"es-objective-3"},{"id":"ari-0567","language":"pt","label":"result","domain":"economia","text":"O efeito estimado de dinâmica da inflação permaneceu robusto em especificações alternativas.","template_group":"pt-result-4"},{"id":"ari-0419","language":"es","label":"implication","domain":"marketing","text":"La evidencia respalda una mayor atención a vinculación con marcas en programas orientados a consumidores digitales.","template_group":"es-implication-3"},{"id":"ari-0268","language":"es","label":"objective","domain":"relaciones_internacionales","text":"Investigamos la relación entre cooperación transfronteriza y los resultados observados en organizaciones regionales.","template_group":"es-objective-2"},{"id":"ari-0236","language":"es","label":"background","domain":"negocios","text":"Una literatura creciente destaca la relevancia de transformación digital para pequeñas empresas.","template_group":"es-background-3"},{"id":"ari-0588","language":"pt","label":"limitation","domain":"política_pública","text":"Como os dados são observacionais, afirmações causais sobre prestação de serviços públicos devem ser interpretadas com cautela.","template_group":"pt-limitation-1"},{"id":"ari-0168","language":"en","label":"limitation","domain":"public_policy","text":"Measurement error may affect the estimated relationship involving public service delivery.","template_group":"en-limitation-3"},{"id":"ari-0514","language":"pt","label":"method","domain":"relações_internacionais","text":"O estudo empregou um desenho de métodos mistos combinando entrevistas e análise quantitativa de cooperação transfronteiriça.","template_group":"pt-method-1"},{"id":"ari-0428","language":"es","label":"implication","domain":"negocios","text":"Los hallazgos señalan la necesidad de políticas más específicas que aborden transformación digital.","template_group":"es-implication-5"},{"id":"ari-0608","language":"pt","label":"limitation","domain":"negócios","text":"Seriam necessários dados adicionais para verificar se o padrão de transformação digital persiste ao longo do tempo.","template_group":"pt-limitation-5"},{"id":"ari-0364","language":"es","label":"limitation","domain":"relaciones_internacionales","text":"El estudio está limitado por el tamaño relativamente pequeño de la muestra de organizaciones regionales.","template_group":"es-limitation-0"},{"id":"ari-0521","language":"pt","label":"method","domain":"marketing","text":"Os dados sobre engajamento com marcas foram codificados e analisados por meio de um fluxo estatístico reproduzível.","template_group":"pt-method-2"},{"id":"ari-0408","language":"es","label":"implication","domain":"política_pública","text":"Los resultados destacan la importancia práctica de incorporar prestación de servicios públicos en estrategias dirigidas a gobiernos locales.","template_group":"es-implication-1"},{"id":"ari-0346","language":"es","label":"result","domain":"relaciones_internacionales","text":"Los hallazgos indican que cooperación transfronteriza explica una parte relevante de las diferencias entre organizaciones regionales.","template_group":"es-result-3"},{"id":"ari-0423","language":"es","label":"implication","domain":"economía","text":"Estos resultados pueden orientar decisiones institucionales relacionadas con dinámica inflacionaria y economías latinoamericanas.","template_group":"es-implication-4"},{"id":"ari-0143","language":"en","label":"result","domain":"marketing","text":"Results reveal that changes in brand engagement were concentrated among a subset of online consumers.","template_group":"en-result-5"},{"id":"ari-0453","language":"pt","label":"background","domain":"economia","text":"Uma literatura crescente destaca a relevância de dinâmica da inflação para economias latino-americanas.","template_group":"pt-background-3"},{"id":"ari-0173","language":"en","label":"limitation","domain":"marketing","text":"The geographic scope limits the generalizability of the findings about brand engagement.","template_group":"en-limitation-4"},{"id":"ari-0642","language":"pt","label":"implication","domain":"política_pública","text":"Esses resultados podem orientar decisões institucionais relacionadas a prestação de serviços públicos e governos locais.","template_group":"pt-implication-4"},{"id":"ari-0244","language":"es","label":"background","domain":"relaciones_internacionales","text":"A pesar del interés sostenido, el contexto más amplio de cooperación transfronteriza sigue documentado de forma desigual.","template_group":"es-background-4"},{"id":"ari-0021","language":"en","label":"background","domain":"economics","text":"A growing body of literature highlights the relevance of inflation dynamics for Latin American economies.","template_group":"en-background-3"},{"id":"ari-0077","language":"en","label":"method","domain":"marketing","text":"We analyzed survey data collected from online consumers using multivariate regression.","template_group":"en-method-0"},{"id":"ari-0187","language":"en","label":"implication","domain":"education","text":"The results highlight the practical importance of incorporating academic writing into strategies for university students.","template_group":"en-implication-1"},{"id":"ari-0360","language":"es","label":"result","domain":"política_pública","text":"Los resultados revelan que los cambios en prestación de servicios públicos se concentraron en un subconjunto de gobiernos locales.","template_group":"es-result-5"},{"id":"ari-0110","language":"en","label":"result","domain":"business","text":"The results show a significant association between digital transformation and outcomes for small firms.","template_group":"en-result-0"},{"id":"ari-0111","language":"en","label":"result","domain":"economics","text":"The results show a significant association between inflation dynamics and outcomes for Latin American economies.","template_group":"en-result-0"},{"id":"ari-0269","language":"es","label":"objective","domain":"marketing","text":"Investigamos la relación entre vinculación con marcas y los resultados observados en consumidores digitales.","template_group":"es-objective-2"},{"id":"ari-0047","language":"en","label":"objective","domain":"marketing","text":"The objective of this research is to assess brand engagement among online consumers.","template_group":"en-objective-1"},{"id":"ari-0426","language":"es","label":"implication","domain":"política_pública","text":"Estos resultados pueden orientar decisiones institucionales relacionadas con prestación de servicios públicos y gobiernos locales.","template_group":"es-implication-4"},{"id":"ari-0483","language":"pt","label":"objective","domain":"economia","text":"Investigamos a relação entre dinâmica da inflação e os resultados observados em economias latino-americanas.","template_group":"pt-objective-2"},{"id":"ari-0209","language":"en","label":"implication","domain":"marketing","text":"These results may inform institutional decisions concerning brand engagement and online consumers.","template_group":"en-implication-4"},{"id":"ari-0066","language":"en","label":"objective","domain":"public_policy","text":"The study evaluates whether changes in public service delivery are associated with outcomes among local governments.","template_group":"en-objective-4"},{"id":"ari-0441","language":"pt","label":"background","domain":"economia","text":"dinâmica da inflação tornou-se uma questão relevante para pesquisadores que estudam economias latino-americanas.","template_group":"pt-background-1"},{"id":"ari-0528","language":"pt","label":"method","domain":"política_pública","text":"Realizamos entrevistas semiestruturadas com governos locais e aplicamos análise temática.","template_group":"pt-method-3"},{"id":"ari-0579","language":"pt","label":"limitation","domain":"economia","text":"O estudo é limitado pelo tamanho relativamente pequeno da amostra de economias latino-americanas.","template_group":"pt-limitation-0"},{"id":"ari-0307","language":"es","label":"method","domain":"educación","text":"Realizamos entrevistas semiestructuradas con estudiantes universitarios y aplicamos análisis temático.","template_group":"es-method-3"},{"id":"ari-0141","language":"en","label":"result","domain":"economics","text":"Results reveal that changes in inflation dynamics were concentrated among a subset of Latin American economies.","template_group":"en-result-5"},{"id":"ari-0487","language":"pt","label":"objective","domain":"educação","text":"Este artigo busca identificar os principais determinantes de escrita acadêmica para estudantes universitários.","template_group":"pt-objective-3"},{"id":"ari-0030","language":"en","label":"background","domain":"public_policy","text":"Despite sustained interest, the broader context of public service delivery remains unevenly documented.","template_group":"en-background-4"},{"id":"ari-0318","language":"es","label":"method","domain":"política_pública","text":"Se utilizó un diseño comparativo de estudios de caso para examinar prestación de servicios públicos en varios contextos.","template_group":"es-method-4"},{"id":"ari-0617","language":"pt","label":"implication","domain":"marketing","text":"Esses achados sugerem que formuladores de políticas devem considerar engajamento com marcas ao desenhar intervenções para consumidores digitais.","template_group":"pt-implication-0"}]
index.html CHANGED
@@ -1,19 +1,70 @@
1
  <!doctype html>
2
- <html>
3
- <head>
4
- <meta charset="utf-8" />
5
- <meta name="viewport" content="width=device-width" />
6
- <title>My static Space</title>
7
- <link rel="stylesheet" href="style.css" />
8
- </head>
9
- <body>
10
- <div class="card">
11
- <h1>Welcome to your static Space!</h1>
12
- <p>You can modify this app directly by editing <i>index.html</i> in the Files and versions tab.</p>
13
- <p>
14
- Also don't forget to check the
15
- <a href="https://huggingface.co/docs/hub/spaces" target="_blank">Spaces documentation</a>.
16
- </p>
17
- </div>
18
- </body>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
  </html>
 
1
  <!doctype html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="utf-8"/>
5
+ <meta name="viewport" content="width=device-width,initial-scale=1"/>
6
+ <meta name="theme-color" content="#081422"/>
7
+ <meta name="description" content="Reproducible multilingual model evaluation, experiment tracking and responsible ML diagnostics."/>
8
+ <title>ModelOps Research Lab | Hugdrigo</title>
9
+ <link rel="stylesheet" href="styles.css" />
10
+ </head>
11
+ <body>
12
+ <a class="skip" href="#workspace">Skip to workspace</a>
13
+ <header class="hero">
14
+ <div class="shell hero-layout">
15
+ <div>
16
+ <div class="eyebrow"><span class="pulse"></span> RODRIGO PANTOJA <span class="slash">/</span> REPRODUCIBLE ML ENGINEERING</div>
17
+ <div class="version">PORTFOLIO LAB 013 <span>RESEARCH SERIES</span></div>
18
+ <h1><span class="mark">◈</span> ModelOps<br/><span class="gradient-text">Research Lab</span></h1>
19
+ <p class="hero-copy">From an academic NLP dataset to reproducible machine-learning experiments. Train four transparent text classifiers in your browser, inspect domain shift and errors, and export every result.</p>
20
+ <div class="chips"><span>EN · ES · PT</span><span>Browser-only training</span><span>Dataset & model lineage</span><span>No paid inference</span></div>
21
+ </div>
22
+ <aside class="hero-snapshot" aria-label="Experiment snapshot">
23
+ <div class="snapshot-title"><span class="pulse"></span> RESEARCH DATASET <strong>v2.0</strong></div>
24
+ <div class="snapshot-grid">
25
+ <div><strong>1,026</strong><small>curated synthetic rows</small></div>
26
+ <div><strong>4</strong><small>trained baselines</small></div>
27
+ <div><strong>3</strong><small>languages</small></div>
28
+ <div><strong>6</strong><small>rhetorical labels</small></div>
29
+ </div>
30
+ <div class="snapshot-footer">Train 842 · IID test 130 · Challenge 54</div>
31
+ <div class="snapshot-caution">Synthetic data. These scores do not validate production or real-world academic use.</div>
32
+ </aside>
33
+ </div>
34
+ </header>
35
+ <main id="workspace" class="shell app-shell">
36
+ <div class="app-status"><div id="status" role="status" aria-live="polite"><span class="pulse"></span> Loading local research data and training models…</div><span id="lastRun" class="subtle">No saved experiment yet</span></div>
37
+ <nav class="tabs" aria-label="Research modules">
38
+ <button class="tab active" data-tab="overview" aria-current="page">01 <span>Overview</span></button>
39
+ <button class="tab" data-tab="experiments">02 <span>Experiments</span></button>
40
+ <button class="tab" data-tab="slices">03 <span>Slice analysis</span></button>
41
+ <button class="tab" data-tab="errors">04 <span>Error lab</span></button>
42
+ <button class="tab" data-tab="shift">05 <span>Shift diagnostics</span></button>
43
+ <button class="tab" data-tab="audit">06 <span>Data audit</span></button>
44
+ <button class="tab" data-tab="exports">07 <span>Export & governance</span></button>
45
+ </nav>
46
+ <section class="view active" data-view="overview">
47
+ <div class="section-kicker">AT A GLANCE</div><div class="titlebar"><div><h2>Model evaluation dashboard</h2><p>All displayed scores are recomputed in your browser from the supplied labeled examples.</p></div><div class="title-controls"><label for="overviewSplit">Evaluation set</label><select id="overviewSplit"><option value="challenge">Novel-pattern challenge · 54</option><option value="iid">Controlled IID · 130</option></select></div></div>
48
+ <div class="kpis" id="kpis"><div class="card loading">Training models…</div></div>
49
+ <div class="two-col">
50
+ <div class="card"><div class="cardhead"><div><div class="overline">LEADERBOARD</div><h3>Baseline comparison</h3></div><select id="metricSelect" aria-label="Leaderboard metric"><option value="accuracy">Accuracy</option><option value="macroF1">Macro F1</option></select></div><div id="leaderboard"></div><p class="fine">Four independently scored predictions: word Naive Bayes, word TF-IDF centroid, character TF-IDF centroid, and a weighted word/character hybrid. Scores are diagnostic, not safety ratings.</p></div>
51
+ <div class="card"><div class="cardhead"><div><div class="overline">EXPERIMENT INSIGHT</div><h3>Generalization gap</h3></div><span class="pill">IMPORTANT</span></div><div id="generalization"></div><p class="fine">A large IID-to-challenge gap indicates the limits of a controlled synthetic test set. The original Python model is archived separately, not presented as a head-to-head result.</p></div>
52
+ </div>
53
+ <div class="two-col"><div class="card"><div class="cardhead"><div><div class="overline">ERROR DISTRIBUTION</div><h3>Confusion matrix</h3></div><select id="confModel" aria-label="Confusion matrix method"></select></div><div id="confusion" class="h-scroll"></div></div><div class="card"><div class="overline">RESEARCH DESIGN</div><h3>Fixed evaluation partitions</h3><div id="partitionBars"></div><p class="fine">Partitions come from the original Academic Rhetorical Intent Mini v1.1 dataset, not a freshly randomized split. Train rows never enter evaluation.</p></div></div>
54
+ </section>
55
+ <section class="view" data-view="experiments"><div class="section-kicker">EXPERIMENT CONTROL</div><h2>Reproduce, compare, record</h2><p class="lede">Training is performed locally each time you run an experiment. The seed controls the bootstrap resampling, not the deterministic model fitting.</p>
56
+ <div class="card"><div class="form-row"><label>Evaluation cohort<select id="runSplit"><option value="challenge">Novel-pattern challenge (54)</option><option value="iid">Controlled IID test (130)</option></select></label><label>Bootstrap random seed<input id="runSeed" type="number" min="1" max="999999999" value="2026"/></label><label>Bootstrap iterations<select id="runReps"><option value="200">200</option><option value="500">500</option></select></label><button class="primary" id="runButton">Run four baselines</button></div><p id="runNote" class="fine">Bootstrapped uncertainty describes sampling variation in this specific synthetic evaluation set. It is not a model training or deployment uncertainty estimate.</p></div>
57
+ <div id="experimentOutput" class="card"><p class="muted">Run a benchmark to see the results and 95% sample bootstrap intervals.</p></div>
58
+ <div class="card"><div class="cardhead"><div><div class="overline">LOCAL EXPERIMENT REGISTRY</div><h3>Run history</h3></div><button id="clearHistory" class="ghost">Clear saved history</button></div><p class="fine">Up to 15 summaries are saved in this browser only. No raw academic text is stored in local storage.</p><div id="historyList" class="h-scroll"></div></div>
59
+ </section>
60
+ <section class="view" data-view="slices"><div class="section-kicker">SUBGROUP EVALUATION</div><h2>Where each method succeeds or fails</h2><p class="lede">Inspect performance by language, research domain, and rhetorical label. Small slices are descriptive only, not fairness certifications. Macro F1 averages labels that actually occur in the selected group.</p><div class="card"><div class="form-row"><label>Method<select class="model-select" id="sliceModel"></select></label><label>Cohort<select id="sliceSplit"><option value="challenge">Novel-pattern challenge</option><option value="iid">Controlled IID test</option></select></label></div></div><div class="two-col"><div class="card"><h3>Performance by language</h3><div id="languageSlices" class="h-scroll"></div></div><div class="card"><h3>Performance by domain</h3><div id="domainSlices" class="h-scroll"></div></div></div><div class="card"><h3>Per-label metrics</h3><div id="labelSlices" class="h-scroll"></div></div></section>
61
+ <section class="view" data-view="errors"><div class="section-kicker">FAILURE ANALYSIS</div><h2>Inspect individual mistakes</h2><p class="lede">Unlike a single score, the error explorer shows exactly what the method predicted and what the source label was.</p><div class="card"><div class="form-row"><label>Method<select class="model-select" id="errorModel"></select></label><label>Cohort<select id="errorSplit"><option value="challenge">Novel-pattern challenge</option><option value="iid">Controlled IID test</option></select></label><label>Search examples<input id="errorSearch" type="search" placeholder="Find text or label…"/></label></div><div id="errorsList"></div></div><div class="card"><div class="overline">ON-DEVICE INFERENCE</div><h3>Test your own sentence</h3><p class="fine">Paste one academic-style sentence. No text is uploaded to a server. The displayed model scores are heuristic ranking signals, not calibrated probabilities.</p><textarea id="sampleText" rows="3" placeholder="Example: This study seeks to evaluate how digital literacy affects student engagement."></textarea><div class="form-row"><label>Classifier<select class="model-select" id="sampleModel"></select></label><button class="primary" id="classifyButton">Classify sentence</button></div><div id="sampleOutput" aria-live="polite"></div></div></section>
62
+ <section class="view" data-view="shift"><div class="section-kicker">DOMAIN SHIFT</div><h2>Beyond the easy test score</h2><p class="lede">Contrast controlled test outcomes against unseen patterns. This is a controlled synthetic shift analysis, not a real-world external evaluation.</p><div class="card"><h3>Accuracy across evaluation settings</h3><div id="shiftGrid" class="h-scroll"></div></div><div class="card"><div class="cardhead"><div><div class="overline">PREVIOUS MODEL EVIDENCE</div><h3>Archived Python classifier</h3></div><span class="pill">NOT COMPARABLE DIRECTLY</span></div><div class="kpis small-kpis"><div class="mini"><strong>100%</strong><small>Controlled IID (130)</small></div><div class="mini"><strong>79.6%</strong><small>Novel challenge (54)</small></div><div class="mini"><strong>35.7%</strong><small>PubMed development smoke (28)</small></div></div><p class="fine">These archived figures belong to a different Python TF-IDF classification pipeline and a distinct external biomedical sample. They come from the original project's evaluation results, not from this browser implementation. The 28-sentence sample is too small to establish general deployment performance.</p></div></section>
63
+ <section class="view" data-view="audit"><div class="section-kicker">DATA QUALITY</div><h2>Train/test integrity & provenance</h2><p class="lede">The dataset was synthetically curated for this research portfolio. Audit checks make label coverage, partition sizes, and exact leakage signals visible.</p><div id="auditSummary" class="kpis"></div><div class="two-col"><div class="card"><h3>Dataset composition</h3><div id="datasetComposition" class="h-scroll"></div></div><div class="card"><h3>Leakage & integrity checks</h3><div id="leakageChecks"></div></div></div><div class="card"><h3>Limitations and appropriate use</h3><p>All examples in the original dataset are synthetic-curated. High accuracy can reflect repeated linguistic templates rather than general understanding. The challenge set is a harder fixed test; it is not a substitute for independent human-labeled real-world evaluation.</p><p>Do not use this system to grade students, evaluate academic integrity, screen admissions, determine employment outcomes, or certify scholarly quality.</p></div></section>
64
+ <section class="view" data-view="exports"><div class="section-kicker">REPRODUCIBILITY & GOVERNANCE</div><h2>Take every result with you</h2><p class="lede">This project ships the entire evaluation dataset and executable JavaScript source. No hidden inference API, tokens, or remote server are required.</p><div class="card"><h3>Export a reproducible record</h3><div class="button-row"><button id="downloadPred" class="primary">Prediction CSV</button><button id="downloadMetrics" class="ghost">Metrics JSON</button><button id="downloadReport" class="ghost">Research report (.md)</button><button id="downloadRuns" class="ghost">Saved runs (.json)</button></div><p class="fine">Exports are generated locally, recording the evaluation split, method, seed, source version, and methodology notes.</p></div><div class="two-col"><div class="card"><h3>Release methodology</h3><ol class="methods"><li>Use original fixed train / IID / challenge partitions.</li><li>Fit each classifier to the same 842 training examples.</li><li>Generate one prediction per held-out example.</li><li>Evaluate Accuracy, Macro F1, confusion matrix and language/domain slices.</li><li>Bootstrap accuracy over the chosen held-out sample with a fixed seed.</li><li>Record data lineage and separate synthetic from real-world evidence.</li></ol></div><div class="card"><h3>Model governance</h3><p><strong>Data license:</strong> CC BY 4.0 (attributed to Academic Rhetorical Intent Mini). <strong>Code license:</strong> MIT.</p><p><strong>Privacy:</strong> training and inference run entirely in the browser. History summaries, if enabled, remain in local browser storage.</p><p><strong>Risk:</strong> These are interpretable baselines. The benchmark does not establish robustness outside controlled data or authorize consequential decisions about people.</p></div></div><div class="card"><h3>What is inside the published repository?</h3><div id="fileInventory" class="file-list"><span>Loading release inventory…</span></div></div></section>
65
+ <footer class="foot"><div><strong>ModelOps Research Lab</strong> · Research software by Rodrigo Pantoja · 2026</div><div>MIT code · CC BY 4.0 source dataset · Static hosting · Synthetic research benchmark</div></footer>
66
+ </main>
67
+ <script src="engine.js" defer></script>
68
+ <script src="app.js" defer></script>
69
+ </body>
70
  </html>
preflight.py ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Zero-dependency Hugging Face Static Space preflight."""
3
+ from pathlib import Path
4
+ import json,re,hashlib
5
+ root=Path(__file__).resolve().parent
6
+ readme=(root/'README.md').read_text(encoding='utf-8')
7
+ assert readme.startswith('---\n'), 'Missing README front matter'
8
+ front=readme.split('---',2)[1]
9
+ values=dict(re.findall(r'^([A-Za-z_]+):\s*(.*?)\s*$',front,re.M))
10
+ assert values['sdk']=='static', 'Space needs sdk: static'
11
+ assert values['app_file']=='index.html', 'Wrong app file'
12
+ assert values['license']=='mit', 'Wrong code license'
13
+ assert len(values['short_description'])<=60, 'Short description too long'
14
+ assert values['colorFrom'] in {'red','yellow','green','blue','indigo','purple','pink','gray'}
15
+ assert values['colorTo'] in {'red','yellow','green','blue','indigo','purple','pink','gray'}
16
+ for name in ('index.html','styles.css','engine.js','app.js','train.json','iid_test.json','challenge.json'):
17
+ assert (root/name).is_file(), 'Missing '+name
18
+ manifest=json.loads((root/'dataset_manifest.json').read_text())
19
+ for name,digest in manifest['files_sha256'].items():
20
+ assert hashlib.sha256((root/name).read_bytes()).hexdigest()==digest, f'Data checksum changed: {name}'
21
+ for name,count in [('train.json',842),('iid_test.json',130),('challenge.json',54)]:
22
+ assert len(json.loads((root/name).read_text()))==count,name
23
+ print('PASS: Static SDK, app entry point, license, 60-char description, colors, source files, dataset SHA256 and partition counts.')
styles.css ADDED
@@ -0,0 +1 @@
 
 
1
+ :root{color-scheme:dark;--bg:#07121f;--panel:#101f30;--panel2:#0b1a2a;--line:#284258;--muted:#9eb5c9;--text:#f0f7fc;--teal:#66e3cb;--blue:#82b9ff;--purple:#b1a0ff;--amber:#eebd75;--bad:#f9a4ab;font-family:Inter,ui-sans-serif,system-ui,-apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif}*{box-sizing:border-box}body{margin:0;background:var(--bg);color:var(--text);font-size:14px;line-height:1.55}button,input,select,textarea{font:inherit}button{cursor:pointer}button:disabled{opacity:.5;cursor:wait}a{color:var(--teal)}.skip{position:absolute;left:-9999px}.skip:focus{left:16px;top:16px;z-index:99;background:#fff;color:#000;padding:12px}.shell{max-width:1220px;margin:auto;padding:0 26px}.hero{background:radial-gradient(ellipse at 82% 25%,#174669 0,transparent 60%),linear-gradient(112deg,#0b3a39,#0b233e 48%,#0d1735 100%);border-bottom:1px solid #315268;padding:50px 0 43px}.hero-layout{display:grid;grid-template-columns:minmax(0,1.65fr) minmax(270px,.9fr);gap:64px;align-items:center}.eyebrow,.version,.overline,.section-kicker,.snapshot-title{letter-spacing:.18em;text-transform:uppercase;font-size:10px;font-weight:800}.eyebrow{color:#a5f9ec}.pulse{display:inline-block;width:7px;height:7px;border-radius:100%;background:var(--teal);box-shadow:0 0 15px #36fdd9;margin-right:8px}.slash{padding:0 10px;color:#6fb6b5}.version{color:#9cc8de;margin:22px 0 4px}.version span{margin-left:13px;color:#79c5e2}h1{font-size:clamp(47px,5.8vw,77px);line-height:1.04;letter-spacing:-.075em;margin:8px 0 17px;font-weight:850}.mark{color:var(--teal);font-size:.65em;vertical-align:7px;margin-right:9px}.gradient-text{background:linear-gradient(90deg,#79edcf,#93bfff,#c6a1ff);background-clip:text;color:transparent}.hero-copy{color:#c9e0ef;font-size:15px;max-width:660px;margin:0 0 21px}.chips{display:flex;gap:7px;flex-wrap:wrap}.chips span{border:1px solid #46788a;border-radius:30px;padding:5px 10px;font-size:11px;color:#d8edf6}.hero-snapshot{border:1px solid #437385;background:#091b2aab;border-radius:16px;padding:24px 25px;box-shadow:0 16px 36px #050b2030}.snapshot-title{color:#ace9eb;display:flex;align-items:center;letter-spacing:.1em}.snapshot-title strong{font-size:10px;margin-left:auto;color:#9bb3ca}.snapshot-grid{display:grid;grid-template-columns:1fr 1fr;gap:17px;margin-top:27px}.snapshot-grid div{display:flex;flex-direction:column;gap:0}.snapshot-grid strong{font-size:31px;letter-spacing:-.04em}.snapshot-grid small{color:#b3c7d8}.snapshot-footer{margin-top:24px;padding-top:13px;border-top:1px solid #305067;font-size:12px;color:#a8d8df}.snapshot-caution{color:#a5b7c7;font-size:11px;margin-top:9px}.app-shell{padding-bottom:35px}.app-status{display:flex;justify-content:space-between;align-items:center;gap:12px;padding:22px 2px 13px;font-size:12px;color:#b5cbd8}.app-status #status{color:#a9e8db}.subtle{color:#718da3}.tabs{display:flex;overflow:auto;gap:0;border-bottom:1px solid var(--line);scrollbar-width:thin;margin-bottom:28px}.tab{border:0;border-bottom:2px solid transparent;background:transparent;color:#a8bbcc;padding:12px 13px;white-space:nowrap;font-size:11px;letter-spacing:.045em;font-weight:700}.tab span{margin-left:7px;font-size:12px;letter-spacing:0}.tab:hover,.tab.active{color:var(--teal)}.tab.active{border-color:var(--teal)}.view{display:none}.view.active{display:block}.section-kicker,.overline{color:var(--teal)}h2{font-size:26px;letter-spacing:-.037em;margin:4px 0 4px}h3{font-size:17px;letter-spacing:-.03em;margin:3px 0 14px}.lede,.titlebar p{color:#a8bed0;margin:0 0 22px}.titlebar{display:flex;justify-content:space-between;align-items:end;gap:20px;margin-bottom:17px}.title-controls{display:flex;gap:10px;align-items:center;color:#a1bdc9;font-size:12px}.kpis{display:grid;grid-template-columns:repeat(4,minmax(0,1fr));gap:12px;margin:16px 0}.kpi,.mini{border:1px solid var(--line);border-radius:12px;background:var(--panel);padding:16px 19px}.kpi strong,.mini strong{display:block;font-size:28px;letter-spacing:-.05em}.kpi small,.mini small{color:#a3bacb;font-size:12px;display:block}.kpi em{font-size:10px;color:#72dac9;font-style:normal}.two-col{display:grid;grid-template-columns:1fr 1fr;gap:13px;margin:13px 0}.card{border:1px solid var(--line);border-radius:14px;background:var(--panel2);padding:19px 20px;margin:13px 0;min-width:0}.two-col .card{margin:0}.cardhead{display:flex;justify-content:space-between;align-items:start;gap:8px;margin-bottom:16px}.overline{font-size:9px}.cardhead h3{margin-bottom:0}.fine{font-size:11px;color:#96b0c3;margin-top:14px}.muted{color:#91aabd}.pill{border:1px solid #52748b;border-radius:30px;font-size:10px;padding:4px 8px;color:#d9d1ff;white-space:nowrap}select,input,textarea{color:#eff6fc;background:#14263a;border:1px solid #3b5972;border-radius:9px;padding:9px 12px;max-width:100%;min-height:39px}select:focus-visible,button:focus-visible,input:focus-visible,textarea:focus-visible{outline:2px solid #88cafa;outline-offset:2px}option{background:#14263a}.form-row{display:flex;align-items:end;flex-wrap:wrap;gap:12px}.form-row label{display:flex;flex-direction:column;gap:5px;font-size:12px;color:#afd0db;flex:1 1 160px}.form-row button{min-height:41px}.primary,.ghost{border:1px solid #486986;border-radius:9px;color:#e7f6ff;background:#182d42;padding:9px 14px;font-size:12px;font-weight:700}.primary{background:#3476c7;border-color:#498ee2;color:#fff}.ghost:hover,.primary:hover{filter:brightness(1.15)}.button-row{display:flex;gap:9px;flex-wrap:wrap}.bar-row{display:grid;grid-template-columns:170px 1fr 58px;align-items:center;gap:10px;margin:13px 0}.bar-name{font-size:12px;color:#daeef6}.bar-track{height:9px;background:#27465d;border-radius:50px;overflow:hidden}.bar-fill{height:100%;background:linear-gradient(90deg,#3bbbaf,#82cbff);border-radius:50px}.bar-fill.alt{background:linear-gradient(90deg,#598bd5,#bdabff)}.bar-value{font-weight:800;text-align:right;font-size:12px}.insight-number{font-size:32px;font-weight:800;line-height:1.2}.gap-warn{color:var(--amber)}.good{color:var(--teal)}.bad{color:var(--bad)}.split-bar{height:22px;background:#182f47;border-radius:7px;display:flex;overflow:hidden}.split-part{height:100%;text-align:center;font-size:10px;padding:2px 0;color:#071522;font-weight:800}.split-train{background:#68e0c6}.split-iid{background:#86bbff}.split-challenge{background:#b8a5ff}.legend{display:flex;justify-content:space-between;gap:10px;flex-wrap:wrap;color:#bfd1dd;font-size:12px;margin:13px 0}.legend i{display:inline-block;width:8px;height:8px;border-radius:50%;margin-right:6px}.h-scroll{overflow-x:auto}.table{border-collapse:collapse;width:100%;font-size:12px}.table th{text-align:left;color:#a9ccdc;font-size:11px;font-weight:700;padding:10px 8px;border-bottom:1px solid #34546b;white-space:nowrap}.table td{padding:9px 8px;border-bottom:1px solid #253c4d;vertical-align:top}.table tbody tr:hover{background:#152b40}.number{text-align:right;font-variant-numeric:tabular-nums}.table td.number,.table th.number{text-align:right}.labelchip{display:inline-block;padding:3px 7px;border:1px solid #406379;border-radius:18px;font-size:11px}.matrix td{padding:9px 8px;text-align:center;min-width:48px}.matrix th{font-size:10px;text-align:center}.matrix td.matrixcell{background:color-mix(in srgb,var(--teal) calc(var(--strength) * 85%),#16293a);color:#ecf7fe;font-weight:800}.test-detail{padding:10px 12px;border:1px solid #314f68;border-radius:8px;margin:9px 0;background:#10253b}.test-detail summary{cursor:pointer;font-weight:700}.test-detail p{margin-bottom:7px}.badge{display:inline-block;border:1px solid #45667f;background:#1c3852;color:#c8f0eb;border-radius:99px;padding:3px 8px;font-size:10px;font-weight:700}.badge.badge-error{border-color:#795258;color:#ffc8c8}.badge.badge-success{border-color:#326e6b;color:#a5f0cd}.mini-kpis,.small-kpis{display:grid;grid-template-columns:repeat(3,minmax(0,1fr));gap:12px}.small-kpis{margin:12px 0}.report{background:#0c2237;padding:14px;border:1px solid #31546c;border-radius:10px;margin-top:16px}.methods{padding-left:22px;color:#b9d3df}.methods li{margin-bottom:10px}.file-list{display:grid;grid-template-columns:repeat(2,1fr);gap:6px}.file-list span{padding:7px 11px;border:1px solid #28455b;border-radius:8px;background:#0b1e2f;color:#b5d9e8;font:11px ui-monospace,monospace}.foot{display:flex;justify-content:space-between;gap:12px;margin-top:30px;border-top:1px solid var(--line);padding:20px 2px;color:#829daf;font-size:10px}.foot strong{color:#c4e9e3}textarea{width:100%;resize:vertical}#errorsList .test-detail{margin-bottom:8px}.toast{color:#e2f8ed}.loading{color:var(--muted)}@media(max-width:880px){.hero-layout{grid-template-columns:1fr;gap:24px}.hero{padding:35px 0}.hero-snapshot{max-width:560px}.two-col{grid-template-columns:1fr}.foot{flex-direction:column}.tabs{margin-bottom:20px}.titlebar{align-items:start;flex-direction:column}.kpis{grid-template-columns:repeat(2,1fr)}}@media(max-width:520px){.shell{padding:0 15px}.hero{padding:28px 0}h1{font-size:47px}.hero-copy{font-size:13px}.hero-snapshot{padding:17px}.version{margin-top:19px}.app-status{flex-direction:column;align-items:start}.kpis{gap:7px}.kpi{padding:13px}.kpi strong{font-size:23px}.card{padding:14px}.bar-row{grid-template-columns:94px 1fr 45px;gap:7px}.bar-name{font-size:10px}.mini-kpis,.small-kpis{grid-template-columns:1fr}.file-list{grid-template-columns:1fr}.snapshot-grid strong{font-size:25px}.foot{font-size:9px}}
test_browser.py ADDED
@@ -0,0 +1,82 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from playwright.sync_api import sync_playwright
2
+ import json,os,re
3
+ BASE=os.path.dirname(os.path.abspath(__file__))
4
+ html=open(BASE+'/index.html').read()
5
+ html=re.sub(r'<link rel="stylesheet"[^>]*>', '', html)
6
+ html=re.sub(r'<script src="[^"]+" defer></script>', '', html)
7
+ css=open(BASE+'/styles.css').read()
8
+ engine=open(BASE+'/engine.js').read()
9
+ app=open(BASE+'/app.js').read()
10
+ data={x:json.load(open(BASE+'/'+x)) for x in ['train.json','iid_test.json','challenge.json']}
11
+ def initialize(page):
12
+ page.set_content(html,wait_until='load')
13
+ page.add_style_tag(content=css)
14
+ page.evaluate('data => { window.fetch = async (url) => ({ok:true, json:async()=>data[url.replace("./","")]}); }',data)
15
+ page.add_script_tag(content=engine)
16
+ page.add_script_tag(content=app)
17
+ errs=[]
18
+ with sync_playwright() as p:
19
+ browser=p.chromium.launch(headless=True, executable_path='/usr/bin/chromium',args=['--no-sandbox','--disable-dev-shm-usage'])
20
+ page=browser.new_page(viewport={'width':1440,'height':1040},accept_downloads=True)
21
+ page.on('pageerror',lambda e: errs.append(str(e)))
22
+ initialize(page)
23
+ page.wait_for_function("document.querySelector('#status').textContent.includes('Ready.')",timeout=30000)
24
+ print('01 Boot ready:',page.locator('#status').inner_text())
25
+ print('02 Best challenge:',page.locator('#kpis').inner_text().replace('\n',' ')[:120])
26
+ assert page.locator('#leaderboard .bar-row').count()==4
27
+ assert page.locator('#confusion tbody tr').count()==6
28
+ page.locator('#overviewSplit').select_option('iid')
29
+ assert '130' in page.locator('#kpis').inner_text()
30
+ print('03 130 IID data loaded')
31
+ page.locator('button[data-tab="experiments"]').click()
32
+ page.locator('#runSeed').fill('2026')
33
+ page.locator('#runReps').select_option('200')
34
+ page.locator('#runButton').click()
35
+ assert page.locator('#experimentOutput tbody tr').count()==4
36
+ assert page.locator('#historyList tbody tr').count()>=1
37
+ print('04 Experiment runner + history PASS')
38
+ page.locator('button[data-tab="slices"]').click()
39
+ assert page.locator('#languageSlices tbody tr').count()==3
40
+ page.locator('#sliceSplit').select_option('iid')
41
+ assert page.locator('#domainSlices tbody tr').count()>=3
42
+ print('05 Language and domain slices PASS')
43
+ page.locator('button[data-tab="errors"]').click()
44
+ assert 'errors in 54' in page.locator('#errorsList').inner_text()
45
+ page.locator('#errorSearch').fill('method')
46
+ print('06 Error inspector and search PASS')
47
+ page.locator('#sampleText').fill('This study seeks to evaluate the effects of technology on learning.')
48
+ page.locator('#classifyButton').click()
49
+ assert 'PREDICTED RHETORICAL FUNCTION' in page.locator('#sampleOutput').inner_text()
50
+ print('07 Sample inference PASS')
51
+ page.locator('button[data-tab="shift"]').click()
52
+ assert page.locator('#shiftGrid tbody tr').count()==4
53
+ print('08 Shift comparison PASS')
54
+ page.locator('button[data-tab="audit"]').click()
55
+ assert 'PASS' in page.locator('#leakageChecks').inner_text()
56
+ print('09 Data audit PASS')
57
+ page.locator('button[data-tab="exports"]').click()
58
+ for button in ['downloadPred','downloadMetrics','downloadReport','downloadRuns']:
59
+ with page.expect_download(timeout=10000) as d:
60
+ page.locator('#'+button).click()
61
+ download=d.value
62
+ print('10 Download',download.suggested_filename,'PASS')
63
+ # screenshot at overview after setup
64
+ page.locator('button[data-tab="overview"]').click()
65
+ page.locator('#overviewSplit').select_option('challenge')
66
+ page.screenshot(path=os.path.join(BASE,'ModelOps_Research_Lab_DESKTOP_PREVIEW.png'),full_page=True)
67
+ print('11 Desktop screenshot PASS')
68
+ # Mobile responsive
69
+ mobile=browser.new_page(viewport={'width':390,'height':844},device_scale_factor=1)
70
+ mobile.on('pageerror',lambda e: errs.append(str(e)))
71
+ initialize(mobile)
72
+ mobile.wait_for_function("document.querySelector('#status').textContent.includes('Ready.')",timeout=30000)
73
+ mobile.screenshot(path=os.path.join(BASE,'ModelOps_Research_Lab_MOBILE_PREVIEW.png'),full_page=True)
74
+ width=mobile.evaluate('document.documentElement.scrollWidth')
75
+ print('12 Mobile viewport scroll width:',width)
76
+ assert width<=400, 'mobile horizontal overflow on document'
77
+ mobile.locator('button[data-tab="slices"]').click()
78
+ assert mobile.locator('#languageSlices tbody tr').count()==3
79
+ print('13 Mobile interface PASS')
80
+ print('14 JavaScript errors:',errs)
81
+ assert not errs
82
+ browser.close()
test_engine.js ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // Local test suite (Node >=18). No third-party packages. Run: node --test test_engine.js
2
+ 'use strict';
3
+ const test=require('node:test'),assert=require('node:assert/strict'),fs=require('node:fs'),path=require('node:path');
4
+ const E=require('./engine.js');const data=n=>JSON.parse(fs.readFileSync(path.join(__dirname,n),'utf8'));
5
+ const train=data('train.json'),iid=data('iid_test.json'),challenge=data('challenge.json');
6
+ const model=E.train(train);
7
+ const preds={};for(const split of ['iid','challenge']){preds[split]={};for(const method of ['nb','word','char','hybrid'])preds[split][method]=E.evaluate(model,split==='iid'?iid:challenge,method);}
8
+ test('source counts and partitions match original',()=>{assert.equal(train.length,842);assert.equal(iid.length,130);assert.equal(challenge.length,54);});
9
+ test('balanced six class universe',()=>{assert.equal(E.LABELS.length,6);for(const label of E.LABELS)assert.ok(train.some(r=>r.label===label));});
10
+ test('three source languages represented',()=>{for(const language of ['en','es','pt'])assert.ok(train.some(r=>r.language===language));});
11
+ test('all records have stable unique IDs',()=>{const all=[...train,...iid,...challenge];assert.equal(new Set(all.map(r=>r.id)).size,1026);});
12
+ test('no missing text and labels across partitions',()=>{for(const r of [...train,...iid,...challenge]){assert.ok(r.text?.trim());assert.ok(E.LABELS.includes(r.label));}});
13
+ test('words normalize Unicode accents',()=>{const f=E.features('Educación MÉTODO educação','word');assert.ok(f.has('educacion'));assert.ok(f.has('metodo'));assert.ok(f.has('educacao'));});
14
+ test('character trigram counts are deterministic',()=>{assert.deepEqual([...E.features('abc abc','char')],[...E.features('abc abc','char')]);});
15
+ test('invalid training label is rejected',()=>{assert.throws(()=>E.train([{text:'works',label:'broken'}]));});
16
+ test('empty train rejected',()=>{assert.throws(()=>E.train([]));});
17
+ test('classifier rejects unknown model',()=>{assert.throws(()=>E.predict(model,'hello','bogus'));});
18
+ test('four predictions are valid labels',()=>{for(const method of ['nb','word','char','hybrid'])assert.ok(E.LABELS.includes(E.predict(model,'Our results suggest progress.',method).label));});
19
+ test('all predictions have correct row metadata',()=>{for(const split of ['iid','challenge'])for(const method of ['nb','word','char','hybrid']){assert.equal(preds[split][method].length,split==='iid'?130:54);assert.ok(preds[split][method].every(p=>p.id&&p.language&&p.domain&&p.text));}});
20
+ test('benchmark results are reproducible byte-for-byte',()=>{assert.deepEqual(E.evaluate(model,challenge,'hybrid'),preds.challenge.hybrid);});
21
+ test('NB IID accuracy matches measured baseline',()=>{assert.equal(E.evaluation(preds.iid.nb).accuracy,1);});
22
+ test('hybrid challenge accuracy is 42/54',()=>{assert.equal(E.evaluation(preds.challenge.hybrid).correct,42);});
23
+ test('all models show challenge degradation',()=>{for(const method of ['nb','word','char','hybrid'])assert.ok(E.evaluation(preds.iid[method]).accuracy>E.evaluation(preds.challenge[method]).accuracy);});
24
+ test('macro F1 is within 0..1',()=>{for(const split of ['iid','challenge'])for(const method of ['nb','word','char','hybrid'])assert.ok(E.evaluation(preds[split][method]).macroF1>=0&&E.evaluation(preds[split][method]).macroF1<=1);});
25
+ test('confusion entries sum to sample size',()=>{const m=E.evaluation(preds.challenge.hybrid);assert.equal(E.LABELS.reduce((s,x)=>s+E.LABELS.reduce((t,y)=>t+m.confusion[x][y],0),0),54);});
26
+ test('macro F1 of perfect predictions equals 1',()=>{const rows=E.LABELS.map(label=>({truth:label,pred:label}));assert.equal(E.evaluation(rows).macroF1,1);});
27
+ test('macro F1 zero for two mismatches with unsupported labels',()=>{const rows=[{truth:'background',pred:'method'},{truth:'objective',pred:'method'}];assert.equal(E.evaluation(rows).macroF1,0);});
28
+ test('invalid evaluation label rejected',()=>{assert.throws(()=>E.evaluation([{truth:'not-label',pred:'method'}]));});
29
+ test('evaluation of no rows rejected',()=>{assert.throws(()=>E.evaluation([]));});
30
+ test('by-language slice totals cover challenge',()=>{const slices=E.summarizeSlices(preds.challenge.hybrid,'language');assert.equal(slices.reduce((s,x)=>s+x.n,0),54);assert.equal(slices.length,3);});
31
+ test('by-domain slices cover IID',()=>{const slices=E.summarizeSlices(preds.iid.word,'domain');assert.equal(slices.reduce((s,x)=>s+x.n,0),130);});
32
+ test('clean original test split has no exact duplicate',()=>{assert.equal(E.audit(train,iid).clean,true);});
33
+ test('clean novel challenge has no exact duplicate',()=>{assert.equal(E.audit(train,challenge).clean,true);});
34
+ test('duplicated ID and text detected',()=>{const r=E.audit(train,[train[0]]);assert.equal(r.exactTextOverlap,1);assert.equal(r.duplicateIds,1);assert.equal(r.clean,false);});
35
+ test('bootstrap is seeded and reproducible',()=>{assert.deepEqual(E.bootstrapAccuracy(preds.challenge.hybrid,2026,200),E.bootstrapAccuracy(preds.challenge.hybrid,2026,200));});
36
+ test('bootstrap confidence range within valid values',()=>{const c=E.bootstrapAccuracy(preds.challenge.hybrid,2026,200);assert.ok(c.low<=c.high&&c.low>=0&&c.high<=1);});
37
+ test('invalid bootstrap repetitions rejected',()=>{assert.throws(()=>E.bootstrapAccuracy(preds.challenge.hybrid,2026,0));assert.throws(()=>E.bootstrapAccuracy(preds.challenge.hybrid,2026,5001));});
38
+ test('different model predictions can be compared',()=>{const x=E.compareRuns(preds.challenge.nb,preds.challenge.hybrid);assert.ok(x.changed>=0&&x.bWins>=0&&x.aWins>=0);});
39
+ test('CSV escape commas and quoted fields',()=>{const csv=E.toCSV([{a:'"x,y"',b:'ok'}],['a','b']);assert.equal(csv,'a,b\r\n"""x,y""",ok');});
40
+ test('test labels are not features in model input',()=>{const one=challenge[0];const relabeled={...one,label:one.label==='result'?'background':'result'};assert.equal(E.evaluate(model,[one],'word')[0].pred,E.evaluate(model,[relabeled],'word')[0].pred);});
41
+ test('model version tracks release',()=>{assert.match(E.version(),/^2\.0\.0$/);});
train.json ADDED
The diff for this file is too large to render. See raw diff