ya9919 commited on
Commit
13d2f1e
·
verified ·
1 Parent(s): d5b5384

Add four held-out ConsumptionQA examples

Browse files
Files changed (2) hide show
  1. README.md +45 -0
  2. qa_examples.json +62 -0
README.md CHANGED
@@ -36,6 +36,51 @@ S²-MAS is a society-scale multi-agent simulation framework for studying how hou
36
 
37
  The benchmark groups twelve systems into six decision-model routes: encoder option scoring; language models with learned decision heads; next-token decision SFT; dynamic-schema classification; multimodal encoder decisions; and provider decision interfaces. All receive text inputs in the common benchmark. The **17 model-repository entries** comprise **10 fixed-budget trained variants**—five native Laya and five Qwen variants—plus **six checkpoint/runtime entries** and **one compatibility alias**.
38
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
39
  ## This model's role
40
 
41
  This checkpoint follows the **language-model plus learned decision-head route**. It applies the development-selected LoRA method to the independently initialized, official post-trained Qwen3.5-9B backbone. The 200k one-pass protocol permits a direct comparison with the adapted 4B model; the upstream training histories also differ between those initializations.
 
36
 
37
  The benchmark groups twelve systems into six decision-model routes: encoder option scoring; language models with learned decision heads; next-token decision SFT; dynamic-schema classification; multimodal encoder decisions; and provider decision interfaces. All receive text inputs in the common benchmark. The **17 model-repository entries** comprise **10 fixed-budget trained variants**—five native Laya and five Qwen variants—plus **six checkpoint/runtime entries** and **one compatibility alias**.
38
 
39
+ <!-- consumptionqa-examples:start -->
40
+ ## Test-set QA examples
41
+
42
+ These four QA excerpts come from the existing held-out consumer financial-choice test set. They retain the original English questions, candidate menus and recorded reference answers. Full household states and original identifiers are omitted. The answers belong to the households in the original records; provide a household state when running inference.
43
+
44
+ [Download the examples as JSON](qa_examples.json).
45
+
46
+ ### 1. Active credit-card use
47
+
48
+ Does your household use credit cards? Exclude cards that have not been activated.
49
+
50
+ - `1` — Yes
51
+ - `2` — No
52
+
53
+ **Recorded reference answer:** `2` — No
54
+
55
+ ### 2. Demand-deposit account
56
+
57
+ Does your household currently have a renminbi demand-deposit account?
58
+
59
+ - `1` — Yes
60
+ - `2` — No
61
+
62
+ **Recorded reference answer:** `1` — Yes
63
+
64
+ ### 3. Stock trading account
65
+
66
+ Does your household currently have a stock trading account?
67
+
68
+ - `1` — Yes
69
+ - `2` — No
70
+
71
+ **Recorded reference answer:** `2` — No
72
+
73
+ ### 4. Unmatured time deposits
74
+
75
+ Does your household currently have any renminbi time deposits that have not yet matured?
76
+
77
+ - `1` — Yes
78
+ - `2` — No
79
+
80
+ **Recorded reference answer:** `2` — No
81
+
82
+ <!-- consumptionqa-examples:end -->
83
+
84
  ## This model's role
85
 
86
  This checkpoint follows the **language-model plus learned decision-head route**. It applies the development-selected LoRA method to the independently initialized, official post-trained Qwen3.5-9B backbone. The 200k one-pass protocol permits a direct comparison with the adapted 4B model; the upstream training histories also differ between those initializations.
qa_examples.json ADDED
@@ -0,0 +1,62 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "title": "Selected held-out ConsumptionQA excerpts",
3
+ "content": "Question, option menu and recorded reference answer. Household states and original identifiers are omitted. These are QA excerpts, not complete inference requests.",
4
+ "examples": [
5
+ {
6
+ "example_id": "example-01",
7
+ "topic": "Active credit-card use",
8
+ "split": "test",
9
+ "task": "ConsumptionQA",
10
+ "question_type": "choice",
11
+ "question": "Does your household use credit cards? Exclude cards that have not been activated.",
12
+ "options": {
13
+ "1": "Yes",
14
+ "2": "No"
15
+ },
16
+ "reference_choice": "2",
17
+ "reference_answer": "No"
18
+ },
19
+ {
20
+ "example_id": "example-02",
21
+ "topic": "Demand-deposit account",
22
+ "split": "test",
23
+ "task": "ConsumptionQA",
24
+ "question_type": "choice",
25
+ "question": "Does your household currently have a renminbi demand-deposit account?",
26
+ "options": {
27
+ "1": "Yes",
28
+ "2": "No"
29
+ },
30
+ "reference_choice": "1",
31
+ "reference_answer": "Yes"
32
+ },
33
+ {
34
+ "example_id": "example-03",
35
+ "topic": "Stock trading account",
36
+ "split": "test",
37
+ "task": "ConsumptionQA",
38
+ "question_type": "choice",
39
+ "question": "Does your household currently have a stock trading account?",
40
+ "options": {
41
+ "1": "Yes",
42
+ "2": "No"
43
+ },
44
+ "reference_choice": "2",
45
+ "reference_answer": "No"
46
+ },
47
+ {
48
+ "example_id": "example-04",
49
+ "topic": "Unmatured time deposits",
50
+ "split": "test",
51
+ "task": "ConsumptionQA",
52
+ "question_type": "choice",
53
+ "question": "Does your household currently have any renminbi time deposits that have not yet matured?",
54
+ "options": {
55
+ "1": "Yes",
56
+ "2": "No"
57
+ },
58
+ "reference_choice": "2",
59
+ "reference_answer": "No"
60
+ }
61
+ ]
62
+ }