zechen-nlp commited on
Commit
ded105d
Β·
verified Β·
1 Parent(s): cca4a69

Update Automated MNLP evaluation report (2026-06-11)

Browse files
Files changed (1) hide show
  1. EVAL_REPORT.md +8 -14
EVAL_REPORT.md CHANGED
@@ -2,7 +2,7 @@
2
 
3
  - **Model repo:** [`cs-552-2026-mnlplus/general_knowledge_model`](https://huggingface.co/cs-552-2026-mnlplus/general_knowledge_model)
4
  - **Owner(s):** group **mnlplus**
5
- - **Generated at:** 2026-06-10T21:20:55+00:00 (UTC)
6
  - **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
7
 
8
  _This PR is opened automatically by the course CI. It is **non-blocking** β€” you do not need to merge it. The next nightly run will refresh this file._
@@ -18,7 +18,7 @@ _This PR is opened automatically by the course CI. It is **non-blocking** β€” yo
18
  | Benchmark | Accuracy | Status |
19
  |---|---:|---|
20
  | Math | β€” | not run |
21
- | Knowledge | 0.3400 | ok |
22
  | Multilingual | β€” | not run |
23
  | Safety | β€” | not run |
24
 
@@ -30,35 +30,29 @@ _Prompts are intentionally omitted to avoid revealing benchmark contents. For mu
30
 
31
  **Correct** (1 shown)
32
 
33
- - **reference**: `G`
34
  - **overall** (1/1 completions correct)
35
- - **extracted** (βœ“): `G`
36
  - **completion**:
37
 
38
  ```text
39
  <think>
40
- Maria pays $9 for lunch on each workday.
41
- She uses 5 Γ— 9 to determine her lunch cost for a 5-day work week.
42
- To find the total amount Maria spends on lunch during that week, we need to multiply the cost of lunch per day by the number of days she works.
43
- So, we have:
44
- Total lunch cost = $9/day Γ— 5 days
45
- = $45
46
- The answer is (G).
47
  </think>
48
 
49
- \boxed{G}
50
  ```
51
 
52
  **Incorrect** (1 shown)
53
 
54
- - **reference**: `I`
55
  - **overall** (0/1 completions correct)
56
  - **extracted** (βœ—): `B`
57
  - **completion**:
58
 
59
  ```text
60
  <think>
61
- An executory contract is a contract where one party has begun performance and the other party has completed performance.
62
  </think>
63
 
64
  \boxed{B}
 
2
 
3
  - **Model repo:** [`cs-552-2026-mnlplus/general_knowledge_model`](https://huggingface.co/cs-552-2026-mnlplus/general_knowledge_model)
4
  - **Owner(s):** group **mnlplus**
5
+ - **Generated at:** 2026-06-11T04:19:46+00:00 (UTC)
6
  - **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
7
 
8
  _This PR is opened automatically by the course CI. It is **non-blocking** β€” you do not need to merge it. The next nightly run will refresh this file._
 
18
  | Benchmark | Accuracy | Status |
19
  |---|---:|---|
20
  | Math | β€” | not run |
21
+ | Knowledge | 0.3800 | ok |
22
  | Multilingual | β€” | not run |
23
  | Safety | β€” | not run |
24
 
 
30
 
31
  **Correct** (1 shown)
32
 
33
+ - **reference**: `F`
34
  - **overall** (1/1 completions correct)
35
+ - **extracted** (βœ“): `F`
36
  - **completion**:
37
 
38
  ```text
39
  <think>
40
+ The athlete's contract with the baseball team is a future event, and the assignment is a promise to assign the right expected to arise from that future event.
 
 
 
 
 
 
41
  </think>
42
 
43
+ \boxed{F}
44
  ```
45
 
46
  **Incorrect** (1 shown)
47
 
48
+ - **reference**: `C`
49
  - **overall** (0/1 completions correct)
50
  - **extracted** (βœ—): `B`
51
  - **completion**:
52
 
53
  ```text
54
  <think>
55
+ The correct answer is B.
56
  </think>
57
 
58
  \boxed{B}