Spaces:
Sleeping
Sleeping
Commit History
-
feat(training): add retrain-on-drift CLI
New module sentiment.training.retrain_on_drift with argparse CLI: --backend {catboost|lora}, --drift-report PATH (or '-' for stdin), --gate-min-observed N, --dry-run. Reads a /metrics/drift response dict, applies the domain gate (Step 1), invokes run_baseline / run_lora_training if the gate fires, and appends to reports/retrain-decisions.jsonl (gitignored). Dataset loading is wrapped in _load_splits() so tests can stub it without monkeypatching HARDDataset (no shared load_dataset_splits helper exists in src/sentiment/training/; both backends inline the HARDDataset(...).load() call in their own main()). 6 CLI integration tests with monkeypatched training entrypoints. 233 tests green.89b50af -
feat(domain): add retrain-on-drift gate (pure decision)
New module sentiment/domain/retrain_policy.py with RetrainGate, RetrainDecision, should_retrain. Pure function over the /metrics/drift response dict; domain boundary preserved (the existing test_fitness.py rglob sweep over src/sentiment/domain/ auto-includes the new file — no fitness edit needed). Default gate fires on any signal at drift_level=='significant' with observed_count>=200 and insufficient_data==false. 7 gate truth-table tests. 227 tests green.23f9162
-
feat(mlflow): swap default tracking URI to sqlite:///mlflow.db
Closes ADR-0003's deferred position on the file-backed registry deprecation. The five default-constant sites flip from "file:./mlruns" to "sqlite:///mlflow.db": src/sentiment/training/mlflow_logging.py src/sentiment/training/baseline.py src/sentiment/training/lora.py src/sentiment/adapters/mlflow_registry_classifier.py (two sites) src/api/main.py Test fixtures in test_mlflow_logging.py, test_baseline_training.py, and test_lora_training.py build their per-test tracking URIs as sqlite:///<tmp>/mlflow.db instead of file://<tmp>/mlruns. test_api.py and test_mlflow_registry_classifier.py update their literal defaults. One new test pins the cross-module invariant (test_default_tracking_uri_is_sqlite_across_modules in test_mlflow_logging.py): asserts the constant in three training modules, both MLflowRegistryAdapter and load_from_registry_or_fallback signature defaults, and the os.environ.get default in api/main.py all equal "sqlite:///mlflow.db" with no remaining "file:./mlruns…5f2258e -
feat(api): make MLflow registry the serving source-of-truth
Phase 7 (ADR-0004) — /health.model_version now reflects what was actually loaded into memory, not whatever the report file last said. _build_classifier now returns (port, RegistryVersionInfo | None) and delegates catboost/lora dispatch to load_from_registry_or_fallback. Stub branch returns (StubClassifier(), None). Lifespan unpacks both onto app.state. /health serializes RegistryVersionInfo via dataclasses.asdict (name, version, run_id, source='registry') or null on filesystem fallback / stub. MODEL_VERSION env var pins a specific registry version; unset → latest non-archived. MLFLOW_TRACKING_URI honored (default file:./mlruns). *_MODEL_DIR env vars retained as the fallback path. _extract_model_version removed; _load_reference drops its third return value (drift signals only). The four Phase-6 report-driven /health tests are replaced by six Phase-7 loader-driven ones; existing lifespan tests now monkeypatch the loader rather than the adapter classes (which are no longer imported by …9b4857c -
feat(adapters): add MLflowRegistryAdapter + load_from_registry_or_fallback
Phase 7 (ADR-0004) — wraps a registry-loaded SentimentPyfunc as a SentimentClassifierPort and exposes a RegistryVersionInfo for /health. The loader resolves version via search_model_versions DESC (deprecation of get_latest_versions) or get_model_version when pinned; on MlflowException / OSError / empty registry it falls back to the existing filesystem adapter (returning None for version_info). 11 unit tests cover both happy and fallback paths; mlflow.pyfunc.load_model and MlflowClient are monkeypatched so no real mlruns/ is required.bc20464 -
feat(api): surface registry version in /health
4c20ba0 -
feat(training): register AraBERT LoRA in MLflow Model Registry
6a304ee -
feat(training): register CatBoost baseline in MLflow Model Registry
8ec0b42 -
feat(training): add polymorphic SentimentPyfunc + register_as in log_run
d24d5bf -
chore(phase): close Phase 5 — dialect breakdown (Gulf vs MSA F1, lexicon-v1)
Real-backend smoke on the full HARD test split (n_gulf=420, gulf_share=1.025%): - CatBoost: Gulf macro 0.7024, MSA macro 0.7655, delta +0.063 - LoRA: Gulf macro 0.7668, MSA macro 0.8384, delta +0.072 LoRA fine-tuning helps MSA (+0.073 macro) more than Gulf (+0.064 macro); the diglossia gap widens with the better backend. Adds `_preserve_existing_confidence_histogram` to both training scripts so retrains no longer wipe the Phase-4 artifact owned by the separate `build_confidence_reference` CLI. CatBoost report f1_macro byte-equivalent to pre-slice (full determinism); LoRA report f1_macro drift +0.003 from pre-slice (MPS float noise).cd5a699 -
feat(training): include dialect_breakdown in AraBERT LoRA report
68237d7 -
feat(training): include dialect_breakdown in CatBoost baseline report
1473fd8 -
feat(training): add compute_dialect_breakdown utility
7565fd5 -
feat(domain): add Dialect enum + lexicon-based classify_dialect
6d332b7
-
feat(training): backfill confidence_histogram into reports JSON
466b5b5 -
feat(api): wire /predict drift recording + /metrics/drift endpoint
9811dac -
feat(adapters): add InMemoryDriftMonitor with predicted-class + confidence buckets
eefbb1a -
feat(domain): add DriftMonitorPort + PSI implementation + confidence bucketing
8b14399 -
feat(api): rewire /predict through Depends(get_classifier) with hardened error handling
- Add get_classifier(request) dependency so /predict receives the live classifier via FastAPI DI; tests override via app.dependency_overrides - Wrap classifier.predict in try/except: - ValueError → HTTPException 422 (detail echoes the message) - Any other exception → logger.exception(...) server-side and HTTPException 500 with fixed body "internal inference error" - Response carries no input text, no stack trace, no model internals on error (AGENTS.md review-checklist #5) - Releases D1 lock — /predict no longer pins to the module-level stub Tests: - test_predict_uses_overridden_classifier (DI override exercised) - test_predict_returns_500_when_classifier_raises_unexpected — body is the generic message; user text, exception message, traceback, and type name all absent from the raw response body - test_predict_returns_422_when_classifier_raises_value_error — body detail matches the adapter's exception string Phase 3 / spec 003 / Step 2.976efc5 -
refactor(api): introduce create_app() + lifespan with SENTIMENT_BACKEND dispatch
- Add create_app() factory and async lifespan that constructs the chosen classifier once at startup and stores it on app.state.classifier - SENTIMENT_BACKEND env var selects stub | catboost | lora (default stub); LORA_MODEL_DIR / CATBOOST_MODEL_DIR override paths - /health now reads app.state.backend_name (stub / catboost-baseline-v1 / arabert-lora-v1) for ops liveness probes - Unknown backend value raises ValueError listing valid set; missing model dir surfaces FileNotFoundError at startup (no silent fallback) - Drop module-global _classifier; predict reads from request.app.state Tests: - Add 5 lifespan tests (default stub, lora-env, catboost-env, fail-fast on bogus, fail-fast on missing lora dir); patch adapter classes via monkeypatch so unit tests stay offline - Tighten body["model"] == "stub" assertion on existing health test - Switch existing tests to a TestClient(create_app()) context-managed fixture so lifespan fires Phase 3 / spec 003 / Step 1.f88a246 -
chore(phase): close Phase 2 — AraBERT LoRA beats CatBoost baseline
F1 macro 0.8344 on the HARD test split (D3 gate >0.765 cleared by +0.069). Per-class: positive 0.945, negative 0.826, neutral 0.733. CatBoost baseline was 0.7648; LoRA delta +9.1% relative. - src/sentiment/training/lora.py: switch Trainer(tokenizer=…) → Trainer(processing_class=…) for transformers>=5.0 - .gitignore: carve out reports/*.{json,md,log} so metric receipts and demo outputs persist into git (heavy artifacts — mlruns/, *.safetensors, models/, data/ — remain ignored) - reports/: arabert-lora-v1.json + arabert-lora-v1-demo.md + arabert-lora-v1-training.log + Phase 1's catboost-baseline-v1 receipts API still on StubClassifier — D1 lock holds; rewire is Phase 3.aeb6c20 -
style: apply ruff format to phase-02 adapter + training + tests
d0c2eee -
feat(demo): add --backend flag to showcase LoRA adapter alongside CatBoost
990670d -
feat(training): add AraBERT LoRA training pipeline with MLflow tracking
5defcb8 -
feat(training): add mlflow_logging helper for file-backed run tracking
c263f6f -
feat(adapters): add AraBERT LoRA adapter implementing SentimentClassifierPort
a6893b2 -
chore(phase): close Phase 1 — CatBoost baseline meets F1 bar
- Loader prefers local HARD TSV (unbalanced split has rating=3; balanced does not). - Training: eval_metric=MultiClass + auto_class_weights=Balanced; 200 iter × 20k feats. - Test F1 macro 0.765 (positive 0.901 / negative 0.717 / neutral 0.676). - Adds docs/phase-01-baseline-results.md and updates README Status.9e8c5f3 -
feat(demo): add CLI showcase for trained CatBoost adapter
4a0e17f -
feat(training): add CatBoost baseline training pipeline with F1+confusion matrix
d7af5d0 -
feat(adapters): add CatBoost+TF-IDF adapter implementing SentimentClassifierPort
8714644 -
feat(adapters): add HARD dataset loader with 5★→3-class mapping
a53dc31 -
chore(repo): bootstrap walking skeleton and governance
4e398cc