Replace references and defaults for the JD-parse / T1 reasoning polish from Qwen3.5-2B to Qwen3-1.7B across docs, config, Dockerfile and code. Make T1 LLM polish conditional on reasoning.mode == "llm" (default "template" keeps full NLG, no LLM). Update model repo/gguf names, README benchmarking notes, prefetch logging, and explanatory comments to reflect the new preferred model and behavior.
Respect --limit; LLM streaming heartbeat
Hemprasad Badgujar
b24d6e4
Fixes a silent bug where CLI --limit was ignored by piping the limit through to pipeline.run_full/run_preprocess and recording it in metrics. Adds a streaming LLM generator with periodic heartbeat logs to distinguish slow-but-progressing generations from hung calls and integrates it into LlmClient (preserves timeout behavior). Streamlit: eagerly warms models once per server process to avoid first-request cold loads. Config: reasoning mode default switched to "template". README and job_description adjusted to reflect usage and content changes.
Add LLM call timeout and streaming pipeline phases
Hemprasad Badgujar
26f86bb
Implement hard per-call timeout (45s) via threading to prevent runaway LLM generations that were observed taking 1996s with no way to interrupt. Convert pipeline phases to generators to yield results as they complete instead of batching at the end, enabling live status updates on slow hardware. Add per-call timing metrics to JD parse artifact. Introduce post-hoc reasoning audit to catch empty/duplicate/leaked-name issues for hackathon grading. Update polish acceptance gate with sentence-count bounds (2-4) and contradiction markers to ensure top-10 notes read as strengths-only.
Add observability logging for LLM and embedder
Hemprasad Badgujar
fbbd1c5
Log LLM generation start/completion events at info level with timing and character metrics. Add warning logging when PotionPicker initialization fails, improving observability for model availability and performance monitoring.
Add timing and error logging throughout
Hemprasad Badgujar
f1347ae
Instrument critical paths with performance metrics and error tracking. Add time.perf_counter() timing to model loading (_load_llama, _load_potion, load_onnx_session) and LLM generation. Wrap key operations in try-except blocks to log failures with context. Track progress of explanation polish with per-row and per-batch timing. Improves observability for debugging performance issues and troubleshooting failures in production.
Cache-first model loading for zero-network ranking
Hemprasad Badgujar
d68637e
Plain hf_hub_download fires a live network call on every load (even for fully-cached files) to validate etags, breaking the 'no network during ranking' contract. Add cached_hf_download() that checks local cache first via try_to_load_from_cache() and only calls the Hub on genuine cache-miss. Update all loaders (llm.py, onnx.py, potion.py) and prefetch to use this strategy. Also fix prefetch to pass cpu_threads to ONNX loaders so @lru_cache keys match the actual runtime. Update README and Dockerfile to clarify cache-first behavior replaces 'auto-download fallback' framing. Tests: 93→101.
Add CPU threading, T1-only LLM polish, experiment script
Hemprasad Badgujar
a41b023
Introduce process-level CPU thread budgeting and cgroup-aware resolution (config.compute.cpu_threads, resolve_cpu_threads, configure_compute, onnx_session_options). Wire cpu_threads through ONNX loaders and retrievers. Switch Step-11 LLM polish to T1-only (top 10) and update tiers/acceptance logic; add batched prompt handling and stability fixes. Add scripts/explain_prompt_experiment.py for fast prompt iteration. Improve settings loading so RERANKER_* env vars override YAML (Yaml-backed source). Minor UX/CI touches: CLI/Streamlit call order, hashing fixes in preprocess, model prefetch updates, README updates, and new runtime tests.
Reuse Qwen3.5 for T2; drop LFM2.5 & factor ONNX
Hemprasad Badgujar
190f0fc
Remove the LFM2.5/T2 polish and related SmolLM2 fallback; T1 and T2 now reuse the Qwen3.5-2B JD-parse model. Update configs, comments and README to reflect the smaller prefetch set and adjusted baked size (~2.3 GB). Factor ONNX session loading into a new src/retrieve/onnx.py and update granite/reranker to use it. Remove orjson-backed JSON/text atomic writers and their exports, simplify models/llm API (drop polish-related helpers), and clean up imports/logging. Add presentation (docs/Redrob_Submission_SoloLeveling.pptx) and various doc/UI text updates to match the new model policy.
Two deployment modes now work together:
- Pre-download (build): rank.py --setup bakes all 5 models into the image
- Auto-download (runtime): HF_HUB_OFFLINE left unset so a missing weight
fetches on first use (safety net); opt into strict offline with
`docker build --build-arg OFFLINE=1`
Fix offline loading: llama.cpp Llama.from_pretrained always lists the repo
over the network (even for a cached file), so strict offline broke. Pin
EXACT gguf filenames (Qwen3.5-2B-Q4_K_M.gguf, LFM2.5-350M-Q4_K_M.gguf) and
load via hf_hub_download + Llama(model_path=...), which is cache-first and
needs no Hub API call. Verified: all 5 models load with HF_HUB_OFFLINE=1.
- .dockerignore: exclude data/artifacts (build downloads fresh; lean
context mirrors a HF Space build instead of shipping the local cache)
- README: deployment section documents pre-download + auto-download
Rebuild Step 11 into a package (src/rank/explain/) with a three-layer
pipeline: FactEngine -> advanced rule-based NLG composer -> tiered LLM
polish. Carries ~90 of 100 finalists with zero LLM.
- bands.py: numbers -> subjective band labels (never raw floats)
- engine.py: JD-grounded FactEngine (missing must-haves, nice-to-have,
3-tier location, country, exclude-company, certs, notice/recency,
widened dominant-signal across all funnel legs)
- compose.py: clause aggregation + connectives (and/but/though), opener
variation, potion-base-32M grounded top-k variant selection, T1=4 /
others 3-4 sentences, >=6 variants/slot
- signals.py: redrob_signals behavioral bank, adaptive best-first,
sentinel skip, >=2-flag soft-pattern gate
- tiers.py: per-tier model (T1 Qwen3.5-2B reused, T2 LFM2.5-350M with
SmolLM2-360M fallback); grounding gate + over-compression guard;
Qwen3-1.7B dropped (-1.1 GB disk)
- Negatives gated to T3/T4 CSV only (top picks read positively);
explanation + exp…
Enhance JD extraction (Qwen3.5-2B + richer prompt) and Step-11 reasoning
Hemprasad Badgujar
eb18ae9
JD extraction:
- Switch JD parse to Qwen3.5-2B (standard-instruct GGUF); reasoning stays Qwen3-1.7B (separate handle)
- Gate Qwen's /no_think soft switch to Qwen models only (LlmClient._soft_no_think)
- Enrich the OBJECTIVE prompt with related skills (nice_to_have), target_industries,
exclude_companies, and all welcomed locations; guard out JD-rejected domains
- best-of-N now targets must_have recall only; _as_str_list dedups keywords case-insensitively
- City->country gazetteer fallback for acceptable_countries
Step-11 reasoning:
- Enrich grounded facts: industry / education / recency / title-fit + dominant-signal tail
- Revive the recent_days knob; LLM polish on Qwen3-1.7B
Config/cleanup: hard-filter graded checks off at funnel entry; doc fixes; remove stray files.
Tests: +dedup / related-field / explain coverage (93 pass, ruff + mypy clean).
Add reasoning LLM, perf tweaks, and UI timing
Hemprasad Badgujar
f85f907
Reduce ONNX truncation lengths to 128 for precision and rerank to cut O(n^2) cost. Add a configurable reasoning step (reasoning.mode: template|llm) with small Gemma GGUF client and token cap; implement safe LLM rephrase with cleaning and deterministic fallback. Parallelize JD parse and preprocessing to overlap model load / polars work. Make candidate location visible to BM25 and include JD locations in composite BM25 queries. Update JD parse prompt wording and add a Streamlit timing summary; tests updated to use template reasoning settings.
Add JD parsing, central config, checkpoints & LLM
Hemprasad Badgujar
700e896
Introduce a central Settings system (config/settings.yaml + src/config.py) and enable live JD extraction (problem/job_description.md) via a new JD parsing pipeline (src/jd/*) that emits a unified jd_extraction.json. Add a local GGUF LLM client and helper (src/models/llm.py, src/models/__init__.py) to drive the JD calls and model prefetching. Implement safe atomic file IO and per-phase Parquet checkpoints (src/common/io.py, src/common/checkpoint.py) and wire them into common exports. Expand preprocess functionality (schema overrides, date helper, pipeline exports) and update the CLI to run preprocess and accept a --checkpoints flag. Update Dockerfile to include central config and data, and refresh pyproject dependencies for the new modules. Add a suite of tests for the new components.
