Spaces:
Sleeping
Sleeping
Commit History
-
feat: goal_relevance, priority_score dampening, recommendation labels (T-16, D-02, D-03)
7bfea87
-
Wire context_block: upload UI + llm.py injection + doc updates
pages/upload.py: add optional Stakeholder context expander in Phase 1 with free-text area and PDF upload (pypdf). Word-count the combined input, truncate to 500 words with a warning if exceeded, store in st.session_state['context_block']. Pass context_block to build_ost in Phase 2. pipeline/llm.py: add context_block param to build_ost and _build_user_message (default ''). Appended after cluster evidence in the user message as 'Stakeholder context:' section. Omitted entirely when empty so prompt is unchanged for existing callers. docs/data_contracts.md: add context_block to llm.py inputs and session_state keys.9636e4b
-
feat(clusterer): T-09 replace KMeans with BERTopic + HDBSCAN
Dmitrii ShumakherSwitches clustering from KMeans (silhouette-sweep k=3..8 over raw 384D embeddings) to BERTopic + HDBSCAN over UMAP-reduced 5D space. HDBSCAN is density-based, finds variable-size themes without a predefined k, and produces membership probabilities natively - which feed directly into T-10 hybrid chunk selection. Prototype validation (notebooks/bertopic_hdbscan_prototype.ipynb): Corpus | Method | Clusters | Silhouette (5D) | Noise -----------|----------|---------:|----------------:|------: Notion 527 | KMeans | 4 | ~0.47 | 0% Notion 527 | HDBSCAN | 8 | 0.480 | 7.6% Revolut | KMeans | 4 | ~0.40 | 0% Revolut | HDBSCAN | 4 | 0.392 | 17.6% HDBSCAN finds more themes of more varied sizes, which is what discovery data actually looks like. Silhouette is computed in the UMAP-5D space where HDBSCAN operates (a 384D-only comparison is unfair to the reduced-space algorithm). Implementation: …1497ef5 -
Add jtbd_confidence signal, solution count validation, and fix doc discrepancies
pipeline/llm.py: add MIN_CLUSTER_SIZE=3, _VALID_CONFIDENCE_VALUES, _apply_confidence_override, Check 5b (2-3 solutions per opportunity), Checks 7-8 (jtbd_confidence enum and non-empty reason) prompts/system_prompt.txt: add jtbd_confidence and jtbd_confidence_reason to schema block, Rule 9 rubric, all four few-shot examples updated docs/llm_output_schema.json: add jtbd_confidence rules, rubric, and 2026-05-14 changelog entry docs/data_contracts.md: add jtbd_confidence fields to both OST structs; fix avg_sentiment comment from VADER to lxyuan; fix LLM fallback model noteceff64c -
Add job_type classification to JTBD output (functional/emotional/social)
Add job_type field to LLM output schema, system prompt, and data contracts. Rule 8 in system_prompt.txt defines the classification rubric. Four few-shot examples now cover functional (x2), emotional (Revolut transfer anxiety), and social (Revolut freelancer credibility). Test script Check 7 validates job_type is present and within enum. PM sign-off: Lucas (assigned task).4b52c4f
-
Merge pull request #13 from Lookus22/fix/source-type-diversity-denominator
Unverified signaturefix(odi_scorer): use fixed denominator for source_type_diversityce60de9 -
fix: expand source_type enum from 4 to 6 values per May 13 decision
Dmitrii ShumakherSource type enum expanded per docs/decisions.md (May 13 2026 architecture session). Adds two new source types: - social — Reddit threads, Twitter/X, community forums - internal — Sales call notes, CS/AM notes, escalation reports All four files updated in lockstep so the enum stays consistent across: - pipeline/odi_scorer.py — KNOWN_SOURCE_TYPES (denominator now 6) - pipeline/extractor.py — ALLOWED_SOURCE_TYPES validation - pipeline/chunker.py — ALLOWED_SOURCE_TYPES validation - docs/data_contracts.md — contract documentation Diversity values re-verified on synthetic single-cluster scenarios: Single-source (1 type) -> 0.1667 (1/6) Mixed-source (2 types) -> 0.3333 (2/6) All 6 types -> 1.0 (6/6) Closes T-02. Addresses Lucas's review feedback on PR #13.27dda00
-
Add OST structural validation and fix stale data contract doc
Add _validate_ost() to llm.py: checks opportunity count, hallucinated cluster_ids, duplicates, missing clusters, risk values, and forbidden score fields. Fallback model now triggers on validation failure as well as JSONDecodeError. Final RuntimeError names missing cluster IDs. Fix stale line in data_contracts.md: model names and trigger condition were left over from before the primary/fallback swap.ee9b1eb -
docs(data_contracts): document fixed denominator for source_type_diversity
Dmitrii ShumakherMatch the data contract to the code change in 472c6fb. The denominator is now KNOWN_SOURCE_TYPES_COUNT (=4) — the size of the source_type enum — not the unique source types in the current upload. Adds a Notes bullet explaining the rationale and the single-source ceiling (0.25 max), with PM sign-off date. Part of T-02.5513176
-
Remove AI context files from repo
c54c4a5 -
feat: three-score scoring system, llm.py implementation, source_map module
c24e531
-
Add system prompt and LLM output schema
- JTBD framing, 7 rules, 2 few-shot examples - ODI fields removed from LLM output, injected post-parse - Validated against 3 Lidl Plus clusters, all checks pass662852b
-
feat: initial repo — pipeline, data contracts, LLM schema, system prompt, pages
ada8316