Add CPU threading, T1-only LLM polish, experiment script
Hemprasad Badgujar
a41b023
Introduce process-level CPU thread budgeting and cgroup-aware resolution (config.compute.cpu_threads, resolve_cpu_threads, configure_compute, onnx_session_options). Wire cpu_threads through ONNX loaders and retrievers. Switch Step-11 LLM polish to T1-only (top 10) and update tiers/acceptance logic; add batched prompt handling and stability fixes. Add scripts/explain_prompt_experiment.py for fast prompt iteration. Improve settings loading so RERANKER_* env vars override YAML (Yaml-backed source). Minor UX/CI touches: CLI/Streamlit call order, hashing fixes in preprocess, model prefetch updates, README updates, and new runtime tests.
Reuse Qwen3.5 for T2; drop LFM2.5 & factor ONNX
Hemprasad Badgujar
190f0fc
Remove the LFM2.5/T2 polish and related SmolLM2 fallback; T1 and T2 now reuse the Qwen3.5-2B JD-parse model. Update configs, comments and README to reflect the smaller prefetch set and adjusted baked size (~2.3 GB). Factor ONNX session loading into a new src/retrieve/onnx.py and update granite/reranker to use it. Remove orjson-backed JSON/text atomic writers and their exports, simplify models/llm API (drop polish-related helpers), and clean up imports/logging. Add presentation (docs/Redrob_Submission_SoloLeveling.pptx) and various doc/UI text updates to match the new model policy.
Add ONNX granite/reranker and compute config
Hemprasad Badgujar
66cf40d
Introduce ONNX-based precision embedding (granite) and cross-encoder reranker with CPU-tuned runtime settings. Adds src/common/runtime.py to set thread-pool env vars and provide onnx session options, and new retrieve modules (granite.py, reranker.py) implementing embedding and pair-scoring via onnxruntime + tokenizers. Wire up configure_compute() in entrypoints (rank.py, src/cli.py, src/streamlit_app.py), expose new retrieve functions in src/retrieve/__init__.py, and extend config (src/config.py and config/settings.yaml) with precision/rerank settings and model defaults. Reduce composite_narrow.top_n to 500 and add precision.top_n=300. Add lightweight tests for passthrough guards (tests/test_granite.py, tests/test_reranker.py) and update pyproject.toml to include onnxruntime and tokenizers dependencies.
