Update README.md

README.md

@@ -285,6 +285,19 @@ CompactifAI was applied to produce a smaller, efficient model (16B parameters) w
 | IFBench | 72.31 | 70.79 | 68.46 | 39.20 | 32.80 |
 | MMLU-Pro | 78.90 | 74.78 | 76.65 | 85.01 | 70.09 |
 | LiveCodeBench | 71.11 | 68.04 | 64.65 | 66.35 | 29.84 |
+
+### Quantizations
+![Quantization results](assets/quantization_comparisons.png)
+
+| Benchmark | Nemotron 3 Nano 30B A3B | Nemotron 3 Nano Slim 16B (BF16) | Nemotron 3 Nano Slim 16B (fp8) | Nemotron 3 Nano Slim 16B (nvfp4) |
+| --- | ---: | ---: | ---: | ---: |
+| AIME | 87.66 | 87.22 | 86.67 | 82.00 |
+| GPQA | 74.04 | 71.41 | 70.61 | 71.11 |
+| IFBench | 72.31 | 70.79 | 69.60 | 69.90 |
+| MMLU-Pro | 78.90 | 74.78 | 74.76 | 74.19 |
+| LiveCodeBench | 71.11 | 68.04 | 68.68 | 65.60 |
+
+
 ### Performance
 ![Performance results](assets/performance.png)
 - **Framework:** [guidellm](https://github.com/vllm-project/guidellm)
@@ -303,56 +316,40 @@ Nemotron 3 Nano Slim 16B preserves strong long-context behavior after compressio
 
 Benchmark scores were obtained with the following setups. Methodology varies by benchmark family.
 
-#### MMLU-Pro, AIME25, GPQA:d, LiveCodeBench, IFBench, AA-LCR, SciCode
-
-- **Framework**: [Nemo-Skills](https://github.com/NVIDIA-NeMo/Skills)
-- **Inference**: vLLM 0.18.0
-- **Decode**: temp 1.0, top_p 1.0
-
-#### BFCL v4
+### Inference:
+- **Backend:** VLLM 0.18.0
+- **Nemotron models:**  `temp 1.0', `top_p 1.0`
+- **GPT-OSS-20B:** `temp: 1.0`, ``top_p: 1.0`, `reasoning_effort: high`
+- **Qwen3-14B:** `temp: 1.0`, ``top_p: 1.0`, `reasoning_effort: high`
+- **Ministral-3-14B-Instruct-2512:** `temp: 0.15`
 
-- **Framework**: [EvalScope](https://github.com/EvalScope/EvalScope) 1.4.1
-- **Inference**: vLLM 0.18.0
-- **Decode**: temp 0.6, top_p 0.95, parallel_tool_calls true
 
-#### Tau2-bench (Telecom)
+#### MMLU-Pro, AIME25, GPQA:d, LiveCodeBench, IFBench
 
-- **Framework**: [EvalScope](https://github.com/EvalScope/EvalScope) 1.4.1
-- **Inference**: vLLM 0.18.0
-- **Decode (agent)**: temp 0.6, top_p 0.95
-- **Decode (judge / user sim)**: temp 0.7, top_p 0.8, top_k 20, min_p 0
-- **Repro**: telecom subset; max steps 100; 3 runs; judge parser hermes
+- **Framework**: [Nemo-Skills](https://github.com/NVIDIA-NeMo/Skills)
 
 #### LongBench v1
 
 - **Framework**: CompactifAI evaluation interface with the LongBench harness, benchmark version v1, full suite.
 - **Dataset**: `Xnhyacinth/LongBench`, all LongBench v1 tasks, no subset limit.
-- **Inference**: vLLM 0.18.0
-- **Decode**: max_tokens 1024, temperature 1.0, top_p 0.95, chat template enabled.
 
 #### AA-LCR
 
 - **Framework**: [EvalScope](https://github.com/EvalScope/EvalScope) 1.4.1
 - **Dataset**: `evalscope/AA-LCR` from ModelScope, default test split, 300 examples, 3 repeats.
-- **Inference**: vLLM 0.18.0
-- **Decode**: temperature 1.0, top_p 1.0
 - **Decode (judge)**: judge top_p 0.8, top_k 20, min_p 0.0; pattern scoring.
 
 #### NIAH
-
-- **Framework**: [Nemo-Skills](https://github.com/NVIDIA-NeMo/Skills)
-- **Dataset / tasks**: RULER NIAH variants: single needle, multi-key, multi-query, and multi-value.
-- **Context lengths**: 4096, 8192, 16384, 32768, 65536, 131072, and 262144 tokens.
-- **Inference**: vLLM 0.18.0
-- **Decode**: max_tokens 128, temperature 1.0, top_p 1.0, top_k 20, min_p 0.0, tokenizer `meta-llama/Llama-3.1-8B`.
+- **Framework:** EvalScope 1.4.1.
+- **Dataset / tasks:** : **needle retrieval** with configurable **`needles`**, **`retrieval_question`**, and haystack **`subset_list`** (`english` and/or `chinese` corpora).
+- **Context lengths:** **`numpy.linspace(context_lengths_min, context_lengths_max, context_lengths_num_intervals)`**
+- **Decode:**  tokenizer meta-llama/Llama-3.1-8B.
 
 #### RULER
 
 - **Framework**: [Nemo-Skills](https://github.com/NVIDIA-NeMo/Skills)
 - **Dataset / tasks**: RULER task families including VT, CWE, FWE, QA, and NIAH; CWE included.
-- **Context lengths**: 4096, 8192, 16384, 32768, 65536, 131072, and 262144 tokens.
-- **Inference**: vLLM 0.18.0
-- **Decode**: temperature 1.0, top_p 1.0, top_k 20, min_p 0.0; task-specific output limits: NIAH 128, VT 30, CWE 120, FWE 50, QA 32.
+- **Context lengths**: 131072 and 262144 tokens.
 
 ---