Move JD extraction from data/artifacts/ (gitignored) to data/ so the demo-critical artifact ships with the repo/image. Increase n_ctx from 4096→6144 to handle longer JDs safely. Reduce parse_attempts from 2→1 (mitigated via improved prompt clarity). Retire target_industries/exclude_companies from LLM prompt to free budget for skill/title recall, but keep as [] in schema for stability. Major prompt refinements: simplify role/task structure, emphasize skill recall priority with explicit scan guidance, consolidate constraints. Add data/jd_extraction.json fixture (parsed from sample job_description.md) and empty secrets file. Update tests & docs to reflect retired prompt fields.
Enrich the embed/rerank query with extracted skills + roles
Hemprasad Badgujar
1cbb761
potion / granite / cross-encoder previously compared candidate text against only the
JD ideal_summary; the extracted must_have_skills and job_titles fed BM25 but not the
embedders. embed_query() now composes ideal_summary + "Core skills: ..." + "Relevant
roles: ..." (summary leads so it survives the embedders' 128-token truncation and
leaves the cross-encoder room for the doc; skills/titles capped 15/8). Wired into the
pipeline and the explorer (which now shows the enriched query per embed phase).
Quality-neutral on the full 100K run (mean matched skills 4.31 vs 4.37 baseline, within
temp-0.5 parse noise). Also fix a band assertion made flaky by the true-random phrasing.
Drop disqualifiers field — remove dead JD signal
Hemprasad Badgujar
86788de
disqualifiers was stored on JDSpec and displayed but never consumed by
any filter (hard_filter doesn't read it). Removed it from the OBJECTIVE
prompt, _coerce_objective, JDSpec + _spec_from_mapping, the Streamlit JD
panel, and the test assertion, so there's no orphaned always-empty field.
Add full pipeline, submission & reasoning support
Hemprasad Badgujar
e73e29e
Introduce an end-to-end pipeline and deliverable handling: add pipeline.run_full and a CLI --out flag to run the full funnel and write submission.csv. Add a deterministic reasoning step (reasoning.add_reasoning) to generate one-line recruiter blurbs and a submission writer (rank.write_submission) that normalizes scores, enforces the 100-row validator contract, and atomically writes the CSV. Move JD country hard-drop from soft_filter into composite_narrow and replace the legacy accept_outside_india boolean with an acceptable_countries list across jd.parse, jd.spec, config models, and tests. Add ReasoningSettings and SubmissionSettings to config and update settings.yaml and .gitignore. Wire reasoning into the Streamlit app, export write_submission from rank.__init__, and add tests for reasoning and submission; adjust tests that relied on the old location behavior.
Add retrieval & soft/composite ranking stages
Hemprasad Badgujar
08c3203
Introduce broad retrieval (potion Model2Vec) + soft_filter scoring + composite_narrow (BM25 + z-blend) pipeline steps. Adds retrieval implementations (potion static embedder, BM25 tokenizer/scorer), ranking modules (soft_filter, composite_narrow), and tests for bm25, potion, composite, soft_filter and hard_filter behavior. Update config/settings to expose new tuning blocks (broad_retrieval, soft_filter, composite_narrow) and default embedder, refine hard_filter inactive logic and integrity toggles, and export new rank/retrieve APIs. Streamlit explorer updated to include rank stages and show JD spec; pyproject dependencies added model2vec and numpy. These changes narrow the candidate pool deterministically before expensive embedding and blend semantic + lexical + quality signals for the top-N cut.
Add JD parsing, central config, checkpoints & LLM
Hemprasad Badgujar
700e896
Introduce a central Settings system (config/settings.yaml + src/config.py) and enable live JD extraction (problem/job_description.md) via a new JD parsing pipeline (src/jd/*) that emits a unified jd_extraction.json. Add a local GGUF LLM client and helper (src/models/llm.py, src/models/__init__.py) to drive the JD calls and model prefetching. Implement safe atomic file IO and per-phase Parquet checkpoints (src/common/io.py, src/common/checkpoint.py) and wire them into common exports. Expand preprocess functionality (schema overrides, date helper, pipeline exports) and update the CLI to run preprocess and accept a --checkpoints flag. Update Dockerfile to include central config and data, and refresh pyproject dependencies for the new modules. Add a suite of tests for the new components.
