Instructions to use prereasoner/prereasoner-weights with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use prereasoner/prereasoner-weights with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
Readers v11: the choice (readers/choice.json) replaces the refusal; cards describe the four readers
Browse files- README.md +6 -5
- readers/choice.json +32 -0
- readers/refusal.json +0 -26
README.md
CHANGED
|
@@ -48,9 +48,9 @@ the complete bundle, and only then installs it.
|
|
| 48 |
| `primitives.npz` | Learned primitive-composition head |
|
| 49 |
| `schema_property_head.pt` | Calibrated Schema.org named-property evidence head |
|
| 50 |
| `readers/stage1/` | Own-data schema-item model (XLM-R base): scores each table's and column's use |
|
| 51 |
-
| `readers/stage2_a/`
|
| 52 |
-
| `readers/rescorer/` | Re-scorer (XLM-R base):
|
| 53 |
-
| `readers/
|
| 54 |
|
| 55 |
The source repository contains the small ontology, calibration, taxonomy, and manifest artifacts.
|
| 56 |
The base Qwen model is downloaded separately from its publisher. The readers' directories hold their
|
|
@@ -107,7 +107,8 @@ servable. Unsupported and under-calibrated coordinates abstain.
|
|
| 107 |
The Prereasoner weight bundle is released under Apache-2.0. The base model and source datasets retain
|
| 108 |
their own licenses and terms. The readers in `readers/` are fine-tuned from XLM-RoBERTa (MIT) on Spider's
|
| 109 |
train and dev questions (Yu et al., EMNLP 2018; CC BY-SA 4.0), BIRD train (Li et al., NeurIPS 2023;
|
| 110 |
-
CC BY-SA 4.0) and Spider-Syn (Gan et al., ACL 2021; MIT)
|
| 111 |
-
|
|
|
|
| 112 |
Spider TRAIN (Yu et al., EMNLP 2018; CC BY-SA 4.0); keep that attribution when redistributing it. See
|
| 113 |
[`THIRD_PARTY.md`](https://github.com/ManiDoraisamy/prereasoner-data/blob/main/THIRD_PARTY.md).
|
|
|
|
| 48 |
| `primitives.npz` | Learned primitive-composition head |
|
| 49 |
| `schema_property_head.pt` | Calibrated Schema.org named-property evidence head |
|
| 50 |
| `readers/stage1/` | Own-data schema-item model (XLM-R base): scores each table's and column's use |
|
| 51 |
+
| `readers/stage2_a/` to `readers/stage2_d/` | Four own-data reading models (XLM-R large); each `reader.json` says whether it reads the question's values beside their columns |
|
| 52 |
+
| `readers/rescorer/` | Re-scorer (XLM-R base): scores the readings' queries that ran |
|
| 53 |
+
| `readers/choice.json` | The readers' choice: a logistic model over the re-scorer's logit and how many readers build a query, fitted on held-out calibration databases; it ranks the queries and refuses below its threshold |
|
| 54 |
|
| 55 |
The source repository contains the small ontology, calibration, taxonomy, and manifest artifacts.
|
| 56 |
The base Qwen model is downloaded separately from its publisher. The readers' directories hold their
|
|
|
|
| 107 |
The Prereasoner weight bundle is released under Apache-2.0. The base model and source datasets retain
|
| 108 |
their own licenses and terms. The readers in `readers/` are fine-tuned from XLM-RoBERTa (MIT) on Spider's
|
| 109 |
train and dev questions (Yu et al., EMNLP 2018; CC BY-SA 4.0), BIRD train (Li et al., NeurIPS 2023;
|
| 110 |
+
CC BY-SA 4.0) and Spider-Syn (Gan et al., ACL 2021; MIT), and on questions Gemini on Vertex AI wrote
|
| 111 |
+
over those training databases; keep those attributions when redistributing them. The Spider test set
|
| 112 |
+
appears in no training step. The retired `sql_proposer/` adapter in earlier revisions was trained on
|
| 113 |
Spider TRAIN (Yu et al., EMNLP 2018; CC BY-SA 4.0); keep that attribution when redistributing it. See
|
| 114 |
[`THIRD_PARTY.md`](https://github.com/ManiDoraisamy/prereasoner-data/blob/main/THIRD_PARTY.md).
|
readers/choice.json
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"calibration": {
|
| 3 |
+
"choice_cross_validated": 664,
|
| 4 |
+
"pool": 788,
|
| 5 |
+
"questions": 1028,
|
| 6 |
+
"readers": 4,
|
| 7 |
+
"rescorer_alone": 657
|
| 8 |
+
},
|
| 9 |
+
"coef": [
|
| 10 |
+
0.40996411721148307,
|
| 11 |
+
-0.21938053399837695,
|
| 12 |
+
-0.5981433326616837,
|
| 13 |
+
1.0922106495653106,
|
| 14 |
+
0.48803509361172165
|
| 15 |
+
],
|
| 16 |
+
"features": [
|
| 17 |
+
"rescorer_logit",
|
| 18 |
+
"score/10",
|
| 19 |
+
"best_rank/16",
|
| 20 |
+
"readers_share",
|
| 21 |
+
"empty"
|
| 22 |
+
],
|
| 23 |
+
"intercept": -1.1371059946383688,
|
| 24 |
+
"matched": {
|
| 25 |
+
"answered": 943,
|
| 26 |
+
"baseline_wrong": 294,
|
| 27 |
+
"questions": 1053,
|
| 28 |
+
"right": 649,
|
| 29 |
+
"wrong": 294
|
| 30 |
+
},
|
| 31 |
+
"threshold": 0.1421
|
| 32 |
+
}
|
readers/refusal.json
DELETED
|
@@ -1,26 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"coef": [
|
| 3 |
-
2.517458605608872,
|
| 4 |
-
0.429020566395794,
|
| 5 |
-
0.2848816003601845,
|
| 6 |
-
0.07420142722472789,
|
| 7 |
-
-0.11174800968699541
|
| 8 |
-
],
|
| 9 |
-
"features": [
|
| 10 |
-
"rescorer_probability",
|
| 11 |
-
"score",
|
| 12 |
-
"margin",
|
| 13 |
-
"rank",
|
| 14 |
-
"empty"
|
| 15 |
-
],
|
| 16 |
-
"intercept": -0.7305985111800368,
|
| 17 |
-
"matched": {
|
| 18 |
-
"answered": 881,
|
| 19 |
-
"baseline_wrong": 294,
|
| 20 |
-
"questions": 1053,
|
| 21 |
-
"right": 587,
|
| 22 |
-
"wrong": 294
|
| 23 |
-
},
|
| 24 |
-
"target": null,
|
| 25 |
-
"threshold": 0.2511
|
| 26 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|