isilicon commited on
Commit
fe2e918
路
verified 路
1 Parent(s): e383e96

Publish detailed Mango model card, measured results and visual analysis

Browse files
Files changed (3) hide show
  1. BENCHMARK_METHODOLOGY.md +0 -5
  2. MODEL_CARD_STATUS.json +9 -9
  3. README.md +5 -1
BENCHMARK_METHODOLOGY.md CHANGED
@@ -68,8 +68,3 @@ The frozen suite SHA-256 is
68
  6ba848464b4685e496b5ac83a115e7efb3cf0c8f7a6afcdc49553af49a5df421.
69
  BENCHMARK_REPORT.json records exact scores, uncertainty, raw artifact hashes, grader hashes and timing
70
  coverage without publishing prompts, generated answers, credentials or real user traffic.
71
-
72
-
73
- ## Corrected Qwen effort coverage
74
-
75
- The separate full404 Qwen effort study now grades403 valid questions plus12 agents per effort. Invalid annotation ifeval-1122 (letter="#") triggers random alphabetic substitution in the pinned reference and is unscored for both. Strict IFEval36/39 medium,37/39 xhigh; language-detection seed0. All200 historical strict grades and78 valid effort grades are unchanged. Timing retains all404 collected requests. Historical202 cross-model comparisons and44 frozen subjective ratings are unchanged.
 
68
  6ba848464b4685e496b5ac83a115e7efb3cf0c8f7a6afcdc49553af49a5df421.
69
  BENCHMARK_REPORT.json records exact scores, uncertainty, raw artifact hashes, grader hashes and timing
70
  coverage without publishing prompts, generated answers, credentials or real user traffic.
 
 
 
 
 
MODEL_CARD_STATUS.json CHANGED
@@ -3,22 +3,22 @@
3
  "state": "documentation_refresh",
4
  "source_repo": "z-lab/Qwen3.8-27B-DFlash2",
5
  "source_revision": "50307d4c4cde6860d4eee73e2547cd786fe8e8a4",
6
- "prior_repository_commit": "d20e9e937e1e02413a5e3716daa100981a34d56e",
7
- "reviewed_plan_sha256": "7d2948f34a9ce61358f5190af36e2f552a9ec988c5a1af4c9a41f66fc20edddf",
8
- "created_utc": "2026-10-30T05:12:28.482102+00:00",
9
  "scope": "Card Markdown, aggregate benchmark data, methodology and original visual assets only.",
10
  "files": [
11
  {
12
  "path": "BENCHMARK_METHODOLOGY.md",
13
- "bytes": 5091,
14
- "sha256": "de1f4cda257f43b476b56da1387fb87a0abc1ef7e384f9c2019803bdf71648b0",
15
- "git_blob": "2fc7deae72ab4456feebf116ef9ff0fd6d96f36c"
16
  },
17
  {
18
  "path": "README.md",
19
- "bytes": 12867,
20
- "sha256": "79f65a5f9adc82cabff6da3b264bc746c6645cbb026c52d0c99caa368bf30a79",
21
- "git_blob": "c8b48d6913c7e065796aacf4004f5f54530f4ef7"
22
  },
23
  {
24
  "path": "assets/mango-header.png",
 
3
  "state": "documentation_refresh",
4
  "source_repo": "z-lab/Qwen3.8-27B-DFlash2",
5
  "source_revision": "50307d4c4cde6860d4eee73e2547cd786fe8e8a4",
6
+ "prior_repository_commit": "e383e96a1ac8822a9012a57d43c40d488993a7ad",
7
+ "reviewed_plan_sha256": "cbf387a3f7fa167c94737f30ef6ae1d17f3ebdb11c700bc652b401fa402d249d",
8
+ "created_utc": "2026-10-30T06:52:20.481414+00:00",
9
  "scope": "Card Markdown, aggregate benchmark data, methodology and original visual assets only.",
10
  "files": [
11
  {
12
  "path": "BENCHMARK_METHODOLOGY.md",
13
+ "bytes": 4560,
14
+ "sha256": "d8c2f04abf706531ce0d27c67837e1cbfdfe461f1ffe97e5c419141f294cb5c3",
15
+ "git_blob": "bc62ad64cfbd710ae5828f97824d62fe8657c79f"
16
  },
17
  {
18
  "path": "README.md",
19
+ "bytes": 13279,
20
+ "sha256": "fde646285c874fb30ecced1104bdb0e7f2c1096c11c9dd4552b1114423691762",
21
+ "git_blob": "e15ca2ee586f1bc243f19924cc1016bf07b39efd"
22
  },
23
  {
24
  "path": "assets/mango-header.png",
README.md CHANGED
@@ -25,6 +25,8 @@ z-lab's Qwen3.8 27B DFlash2 draft model, containing its config and safetensors w
25
  > Get model support, request models and quants, discuss benchmarks, and follow new releases and model improvements.
26
  >
27
  > [Join the Infinity community](https://discord.gg/5n5YDbQCMy)
 
 
28
  <!-- mango-community:end -->
29
 
30
  <!-- mango-qwen-effort:start -->
@@ -57,6 +59,8 @@ These results describe the linked configurations, **not this artifact**. A chang
57
 
58
  | Different configuration | Its weighted score | Its workload min |
59
  | --- | --- | --- |
 
 
60
  | [Qwen3.8 27B 路 MLX affine 4-bit 路 OrcaRouter](https://huggingface.co/isilicon/mango-qwen38-27b-uncensored-mlx4) | 80.64 | 23.98 |
61
 
62
  A completed score requires a native preflight, matching frozen prompts, 202 suite answers, 12 agent cases, archive verification and the same reference graders. The [benchmark methodology](BENCHMARK_METHODOLOGY.md) describes the protocol; publisher-side results are retained separately below.
@@ -92,7 +96,7 @@ python -m pip install -U huggingface_hub
92
  hf download isilicon/mango-dflash2 \
93
  config.json \
94
  model.safetensors \
95
- --revision 6f03dc12a69104696203b5b775577a64a5fe8a8a \
96
  --local-dir ./weights/dflash2
97
  ```
98
 
 
25
  > Get model support, request models and quants, discuss benchmarks, and follow new releases and model improvements.
26
  >
27
  > [Join the Infinity community](https://discord.gg/5n5YDbQCMy)
28
+ >
29
+ > [All model tests, per-question scores, experiment history and scoring methods](https://huggingface.co/datasets/isilicon/mango-evaluation-results)
30
  <!-- mango-community:end -->
31
 
32
  <!-- mango-qwen-effort:start -->
 
59
 
60
  | Different configuration | Its weighted score | Its workload min |
61
  | --- | --- | --- |
62
+ | [Qwen3.8 27B Cyber 路 IQ4_XS imatrix 路 embedded MTP](https://huggingface.co/isilicon/mango-qwen27-cyber-iq4xs-mtp) | 70.27 | 29.74 |
63
+ | [Qwen3.8 27B Cyber 路 Q6_K 路 Q8 MTP + projector](https://huggingface.co/isilicon/mango-qwen27-cyber-q6k) | 66.98 | 45.60 |
64
  | [Qwen3.8 27B 路 MLX affine 4-bit 路 OrcaRouter](https://huggingface.co/isilicon/mango-qwen38-27b-uncensored-mlx4) | 80.64 | 23.98 |
65
 
66
  A completed score requires a native preflight, matching frozen prompts, 202 suite answers, 12 agent cases, archive verification and the same reference graders. The [benchmark methodology](BENCHMARK_METHODOLOGY.md) describes the protocol; publisher-side results are retained separately below.
 
96
  hf download isilicon/mango-dflash2 \
97
  config.json \
98
  model.safetensors \
99
+ --revision e383e96a1ac8822a9012a57d43c40d488993a7ad \
100
  --local-dir ./weights/dflash2
101
  ```
102