bebrws commited on
Commit
e39d5ab
·
verified ·
1 Parent(s): fef61ed

Model card: add Foundation-Sec-1.1-8B-Instruct comparison (honest standings)

Browse files
Files changed (1) hide show
  1. README.md +6 -4
README.md CHANGED
@@ -84,10 +84,12 @@ style, upsampled) produced churn without net gain and a chat-mode regression —
84
  diagnosed as format conflict + template overfitting + step-share dilution. Round 7
85
  fixed all three (bare-letter format-matched training, GLM-5.2 paraphrases,
86
  multi-turn packing to 38% of gradient steps): chat CTI-Bench **+1.6** to 60.4%,
87
- CyberMetric completion **87.8%** and SecQA completion **98.6%** (k3 records), and
88
- **v7cti-completion ties Foundation-Sec overall (79.26% vs 79.09%)** while beating it
89
- on CyberMetric by 3.6 pts. Foundation-Sec still leads CTI-Bench (66.0%) — the
90
- remaining target for the next round (failure-targeted DPO or light domain CPT).
 
 
91
 
92
  **Qualitative** (6 open-ended security prompts): Foundation-Sec gives terse,
93
  mostly-correct completion-style answers; k3 gives structured markdown answers with
 
84
  diagnosed as format conflict + template overfitting + step-share dilution. Round 7
85
  fixed all three (bare-letter format-matched training, GLM-5.2 paraphrases,
86
  multi-turn packing to 38% of gradient steps): chat CTI-Bench **+1.6** to 60.4%,
87
+ CyberMetric completion **87.8%** and SecQA completion **98.6%** (k3 records).
88
+ Against the base Foundation-Sec, v7cti-completion ties overall (79.26% vs 79.09%);
89
+ against the stronger **Foundation-Sec-1.1-8B-Instruct (80.50% overall)** k3 leads
90
+ only on CyberMetric (+2.0) and trails on CTI-Bench (62.6% vs 67.4%) — CTI/ATT&CK
91
+ knowledge remains a pretraining-data advantage and the target of the next round
92
+ (light domain CPT, then failure-targeted DPO).
93
 
94
  **Qualitative** (6 open-ended security prompts): Foundation-Sec gives terse,
95
  mostly-correct completion-style answers; k3 gives structured markdown answers with