Publish detailed Mango model card, measured results and visual analysis
Browse files- BENCHMARK_METHODOLOGY.md +0 -5
- MODEL_CARD_STATUS.json +9 -9
- README.md +5 -1
BENCHMARK_METHODOLOGY.md
CHANGED
|
@@ -68,8 +68,3 @@ The frozen suite SHA-256 is
|
|
| 68 |
6ba848464b4685e496b5ac83a115e7efb3cf0c8f7a6afcdc49553af49a5df421.
|
| 69 |
BENCHMARK_REPORT.json records exact scores, uncertainty, raw artifact hashes, grader hashes and timing
|
| 70 |
coverage without publishing prompts, generated answers, credentials or real user traffic.
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
## Corrected Qwen effort coverage
|
| 74 |
-
|
| 75 |
-
The separate full404 Qwen effort study now grades403 valid questions plus12 agents per effort. Invalid annotation ifeval-1122 (letter="#") triggers random alphabetic substitution in the pinned reference and is unscored for both. Strict IFEval36/39 medium,37/39 xhigh; language-detection seed0. All200 historical strict grades and78 valid effort grades are unchanged. Timing retains all404 collected requests. Historical202 cross-model comparisons and44 frozen subjective ratings are unchanged.
|
|
|
|
| 68 |
6ba848464b4685e496b5ac83a115e7efb3cf0c8f7a6afcdc49553af49a5df421.
|
| 69 |
BENCHMARK_REPORT.json records exact scores, uncertainty, raw artifact hashes, grader hashes and timing
|
| 70 |
coverage without publishing prompts, generated answers, credentials or real user traffic.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
MODEL_CARD_STATUS.json
CHANGED
|
@@ -3,22 +3,22 @@
|
|
| 3 |
"state": "documentation_refresh",
|
| 4 |
"source_repo": "z-lab/Qwen3.8-27B-DFlash2",
|
| 5 |
"source_revision": "50307d4c4cde6860d4eee73e2547cd786fe8e8a4",
|
| 6 |
-
"prior_repository_commit": "
|
| 7 |
-
"reviewed_plan_sha256": "
|
| 8 |
-
"created_utc": "2026-10-
|
| 9 |
"scope": "Card Markdown, aggregate benchmark data, methodology and original visual assets only.",
|
| 10 |
"files": [
|
| 11 |
{
|
| 12 |
"path": "BENCHMARK_METHODOLOGY.md",
|
| 13 |
-
"bytes":
|
| 14 |
-
"sha256": "
|
| 15 |
-
"git_blob": "
|
| 16 |
},
|
| 17 |
{
|
| 18 |
"path": "README.md",
|
| 19 |
-
"bytes":
|
| 20 |
-
"sha256": "
|
| 21 |
-
"git_blob": "
|
| 22 |
},
|
| 23 |
{
|
| 24 |
"path": "assets/mango-header.png",
|
|
|
|
| 3 |
"state": "documentation_refresh",
|
| 4 |
"source_repo": "z-lab/Qwen3.8-27B-DFlash2",
|
| 5 |
"source_revision": "50307d4c4cde6860d4eee73e2547cd786fe8e8a4",
|
| 6 |
+
"prior_repository_commit": "e383e96a1ac8822a9012a57d43c40d488993a7ad",
|
| 7 |
+
"reviewed_plan_sha256": "cbf387a3f7fa167c94737f30ef6ae1d17f3ebdb11c700bc652b401fa402d249d",
|
| 8 |
+
"created_utc": "2026-10-30T06:52:20.481414+00:00",
|
| 9 |
"scope": "Card Markdown, aggregate benchmark data, methodology and original visual assets only.",
|
| 10 |
"files": [
|
| 11 |
{
|
| 12 |
"path": "BENCHMARK_METHODOLOGY.md",
|
| 13 |
+
"bytes": 4560,
|
| 14 |
+
"sha256": "d8c2f04abf706531ce0d27c67837e1cbfdfe461f1ffe97e5c419141f294cb5c3",
|
| 15 |
+
"git_blob": "bc62ad64cfbd710ae5828f97824d62fe8657c79f"
|
| 16 |
},
|
| 17 |
{
|
| 18 |
"path": "README.md",
|
| 19 |
+
"bytes": 13279,
|
| 20 |
+
"sha256": "fde646285c874fb30ecced1104bdb0e7f2c1096c11c9dd4552b1114423691762",
|
| 21 |
+
"git_blob": "e15ca2ee586f1bc243f19924cc1016bf07b39efd"
|
| 22 |
},
|
| 23 |
{
|
| 24 |
"path": "assets/mango-header.png",
|
README.md
CHANGED
|
@@ -25,6 +25,8 @@ z-lab's Qwen3.8 27B DFlash2 draft model, containing its config and safetensors w
|
|
| 25 |
> Get model support, request models and quants, discuss benchmarks, and follow new releases and model improvements.
|
| 26 |
>
|
| 27 |
> [Join the Infinity community](https://discord.gg/5n5YDbQCMy)
|
|
|
|
|
|
|
| 28 |
<!-- mango-community:end -->
|
| 29 |
|
| 30 |
<!-- mango-qwen-effort:start -->
|
|
@@ -57,6 +59,8 @@ These results describe the linked configurations, **not this artifact**. A chang
|
|
| 57 |
|
| 58 |
| Different configuration | Its weighted score | Its workload min |
|
| 59 |
| --- | --- | --- |
|
|
|
|
|
|
|
| 60 |
| [Qwen3.8 27B 路 MLX affine 4-bit 路 OrcaRouter](https://huggingface.co/isilicon/mango-qwen38-27b-uncensored-mlx4) | 80.64 | 23.98 |
|
| 61 |
|
| 62 |
A completed score requires a native preflight, matching frozen prompts, 202 suite answers, 12 agent cases, archive verification and the same reference graders. The [benchmark methodology](BENCHMARK_METHODOLOGY.md) describes the protocol; publisher-side results are retained separately below.
|
|
@@ -92,7 +96,7 @@ python -m pip install -U huggingface_hub
|
|
| 92 |
hf download isilicon/mango-dflash2 \
|
| 93 |
config.json \
|
| 94 |
model.safetensors \
|
| 95 |
-
--revision
|
| 96 |
--local-dir ./weights/dflash2
|
| 97 |
```
|
| 98 |
|
|
|
|
| 25 |
> Get model support, request models and quants, discuss benchmarks, and follow new releases and model improvements.
|
| 26 |
>
|
| 27 |
> [Join the Infinity community](https://discord.gg/5n5YDbQCMy)
|
| 28 |
+
>
|
| 29 |
+
> [All model tests, per-question scores, experiment history and scoring methods](https://huggingface.co/datasets/isilicon/mango-evaluation-results)
|
| 30 |
<!-- mango-community:end -->
|
| 31 |
|
| 32 |
<!-- mango-qwen-effort:start -->
|
|
|
|
| 59 |
|
| 60 |
| Different configuration | Its weighted score | Its workload min |
|
| 61 |
| --- | --- | --- |
|
| 62 |
+
| [Qwen3.8 27B Cyber 路 IQ4_XS imatrix 路 embedded MTP](https://huggingface.co/isilicon/mango-qwen27-cyber-iq4xs-mtp) | 70.27 | 29.74 |
|
| 63 |
+
| [Qwen3.8 27B Cyber 路 Q6_K 路 Q8 MTP + projector](https://huggingface.co/isilicon/mango-qwen27-cyber-q6k) | 66.98 | 45.60 |
|
| 64 |
| [Qwen3.8 27B 路 MLX affine 4-bit 路 OrcaRouter](https://huggingface.co/isilicon/mango-qwen38-27b-uncensored-mlx4) | 80.64 | 23.98 |
|
| 65 |
|
| 66 |
A completed score requires a native preflight, matching frozen prompts, 202 suite answers, 12 agent cases, archive verification and the same reference graders. The [benchmark methodology](BENCHMARK_METHODOLOGY.md) describes the protocol; publisher-side results are retained separately below.
|
|
|
|
| 96 |
hf download isilicon/mango-dflash2 \
|
| 97 |
config.json \
|
| 98 |
model.safetensors \
|
| 99 |
+
--revision e383e96a1ac8822a9012a57d43c40d488993a7ad \
|
| 100 |
--local-dir ./weights/dflash2
|
| 101 |
```
|
| 102 |
|