zechen-nlp commited on
Commit
bb617ad
Β·
verified Β·
1 Parent(s): fcb2969

Update Automated MNLP evaluation report (2026-06-02)

Browse files
Files changed (1) hide show
  1. EVAL_REPORT.md +8 -8
EVAL_REPORT.md CHANGED
@@ -2,7 +2,7 @@
2
 
3
  - **Model repo:** [`cs-552-2026-camykaz/general_knowledge_model`](https://huggingface.co/cs-552-2026-camykaz/general_knowledge_model)
4
  - **Owner(s):** group **camykaz**
5
- - **Generated at:** 2026-06-01T06:13:57+00:00 (UTC)
6
  - **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
7
 
8
  _This PR is opened automatically by the course CI. It is **non-blocking** β€” you do not need to merge it. The next nightly run will refresh this file._
@@ -12,7 +12,7 @@ _This PR is opened automatically by the course CI. It is **non-blocking** β€” yo
12
  | Benchmark | Accuracy | Status |
13
  |---|---:|---|
14
  | Math | β€” | not run |
15
- | Knowledge | 0.3400 | ok |
16
  | Multilingual | β€” | not run |
17
  | Safety | β€” | not run |
18
 
@@ -24,22 +24,22 @@ _Prompts are intentionally omitted to avoid revealing benchmark contents. For mu
24
 
25
  **Correct** (1 shown)
26
 
27
- - **reference**: `I`
28
  - **overall** (1/1 completions correct)
29
- - **extracted** (βœ“): `I`
30
  - **completion**:
31
 
32
  ```text
33
- \boxed{I}
34
  ```
35
 
36
  **Incorrect** (1 shown)
37
 
38
- - **reference**: `C`
39
  - **overall** (0/1 completions correct)
40
- - **extracted** (βœ—): `<no answer>`
41
  - **completion**:
42
 
43
  ```text
44
- Answer: C. 44%
45
  ```
 
2
 
3
  - **Model repo:** [`cs-552-2026-camykaz/general_knowledge_model`](https://huggingface.co/cs-552-2026-camykaz/general_knowledge_model)
4
  - **Owner(s):** group **camykaz**
5
+ - **Generated at:** 2026-06-02T23:31:13+00:00 (UTC)
6
  - **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
7
 
8
  _This PR is opened automatically by the course CI. It is **non-blocking** β€” you do not need to merge it. The next nightly run will refresh this file._
 
12
  | Benchmark | Accuracy | Status |
13
  |---|---:|---|
14
  | Math | β€” | not run |
15
+ | Knowledge | 0.4400 | ok |
16
  | Multilingual | β€” | not run |
17
  | Safety | β€” | not run |
18
 
 
24
 
25
  **Correct** (1 shown)
26
 
27
+ - **reference**: `A`
28
  - **overall** (1/1 completions correct)
29
+ - **extracted** (βœ“): `A`
30
  - **completion**:
31
 
32
  ```text
33
+ \boxed{A}
34
  ```
35
 
36
  **Incorrect** (1 shown)
37
 
38
+ - **reference**: `F`
39
  - **overall** (0/1 completions correct)
40
+ - **extracted** (βœ—): `C`
41
  - **completion**:
42
 
43
  ```text
44
+ \boxed{C}
45
  ```