Instructions to use AnamK/jdi-ptr14-0.8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AnamK/jdi-ptr14-0.8b with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("AnamK/jdi-ptr14-0.8b") model = AutoModelForCausalLM.from_pretrained("AnamK/jdi-ptr14-0.8b", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Download engine/VALIDATION.md from AnamK/jdi-ptr14-0.8b: direct link, hf CLI and curl.
- Browser
- Download file 2.5 kB
-
https://huggingface.co/AnamK/jdi-ptr14-0.8b/resolve/main/engine/VALIDATION.md
- Command line
-
hf download hf://AnamK/jdi-ptr14-0.8b/engine/VALIDATION.md
-
curl -L -o VALIDATION.md https://huggingface.co/AnamK/jdi-ptr14-0.8b/resolve/main/engine/VALIDATION.md
Validation: jdi_ptr14 engine vs. the evaluator behind the submitted run
Date: 2026-10-18. GPU: RTX 3090 (shared), CUDA_VISIBLE_DEVICES=0, mem_gb=4.5. torch 2.9.1+cu128, transformers 5.8.0.
Rows: 200 suite rows from the public 0.3 suite, stratified over all 43 in-edition benchmarks with a fixed seed
(work/make_val200.py).
- 1,091 questions in total.
- 47 rows are multi-question and take the shared-prefix path.
- 40 questions have more than 9 options.
Three result sets:
engine:python -m decision_index run --engine jdi_ptr14.engine:Ptr14Engine --option model=<ckpt> --option mem_gb=4.5 --rows val200_rows.jsonl, i.e. the kit runner, which validates every response withdecision_index.engines.base.validate.e2e_fresh: a freshe2e_prefix.py eval <ckpt> <payloads> --layer 14 --batch-tokens 8192 --mem-gb 4.5on the same rows.submitted: the same rows taken fromruns/ptr14nosyn-soup3/results.jsonl.gz.
| Pair | Questions | Argmax agreement | Max abs prob diff |
|---|---|---|---|
| engine vs e2e_fresh | 1,091 | 1,091 (100%) | 0.0 |
| engine vs submitted | 1,091 | 1,091 (100%) | 0.0 |
| e2e_fresh vs submitted | 1,091 | 1,091 (100%) | 0.0 |
Kit answer validator:
- All 200 engine rows have status
ok(0 unsupported, 0 errors). The runner'svalidatepassed on every one, andcompare.pyre-validated all three sets with 0 failures. - All 140,620 rows of the full submitted results file also pass
validate(work/assemble.py).
Equivalence: the outputs are bit-identical. This is expected, because the engine performs exactly the same tensor
operations as e2e_prefix.py: same truncation, same prompt, same split, same batch packing and same head. It also
confirms that the submitted run was produced by bigind_ptr14nosyn_soup3.
Per-request latency (kit runner, this shared 3090, the stratified 200 rows; not the board's sample or hardware): median 48 ms, mean 107 ms, p80 116 ms, max 2.2 s. The maximum is a long multi-question row; the mean is pulled up by ToolRet, BRIGHT and API-Bank rows.
bf16 note: different bf16 code paths can move probabilities in near-ties. For example, flash-linear-attention kernels instead of the torch fallback, or another GPU. Earlier checks found that bf16 vs fp32 changes about 1% of answers. Agreement on the maintainers' hardware should therefore be measured as an argmax-agreement rate, not as exact equality.
Raw outputs: work/val_engine/, work/val_e2e/, engine/val_compare.json.